Edge-RT: OS Support for Controlled Latency in the Multi-Tenant, Real-Time Edge

Embedded and real-time devices in many domains are increasingly dependent on network connectivity. The ability to offload computations encourages Cost, Size, Weight and Power (C-SWaP) optimizations, while coordination over the network effectively enables systems to sense the environment beyond their own local sensors, and to collaborate globally. The promise is significant: Autonomous Vehicles (AVs) coordinating with each other through infrastructure, factories aggregating data for global optimization, and power-constrained devices leveraging offloaded inference tasks. Low-latency wireless (e.g., 5G) technologies paired with the edge cloud, are further enabling these trends. Unfortunately, computation at the edge poses significant challenges due to the challenging combination of limited resources, required high performance, security due to multi-tenancy, and real-time latency.

This paper introduces Edge-RT, a set of OS extensions for the edge designed to meet the end-to-end (packet reception to transmission) deadlines across chains of computations. It supports strong security by executing a chain per-client device, thus isolating tenant and device computations. Despite a practical focus on deadlines and strong isolation, it maintains high system efficiency. To do so, Edge-RT focuses on per-packet deadlines inherited by the computations that operate on it. It introduces mechanisms to avoid per-packet system overheads, while trading only bounded impacts on predictable scheduling. Results show that compared to Linux and EdgeOS, Edge-RT can both maintain higher throughput and meet significantly more deadlines both for systems with bimodal workloads with utilization above 60%, in the presence of malicious tasks, and as the system scales up in clients. Edge Configurations Deadline-aware Preemptivity Client Isolation Computation Chain Dynamic Workloads Scalability CFS ( §II-B) not deadline-aware preemptive process-based per-client chain supported > 2000 DPDK+OVS/SR-IOV ( §II-A) not deadline-aware non-preemptive process-based no chain supported ∼ 256 SCHED DEADLINE ( §II-B) per-thread preemptive process-based no chain not supported < 1000 eBPF+XDP ( §II-A) not deadline-aware non-preemptive no isolation no chain not supported -EdgeOS ( §II-C) not deadline-aware preemptive FWP-based per-client chain supported > 2000 Edge-RT ( §III) per-packet preemptive FWP-based per-client chain supported > 2000 TABLE I: A summary of edge-cloud configurations in §II. Entries labelled with bullets from fully supported ( ), complicated (please refer to the text for details) ( ), and not supported ( ).

pedestrians are mobile, the client number and frequency of service requests vary over time.

• Network processing -basestations traditionally focus on network processing including properly accounting for bandwidth, and slicing the network [5], [6], [7], [8]

across carriers. This network processing is done by network functions (NFs) that transform and filter packets, and are often composed into chains that process packets, and pass them on to the next NF. Chains of isolated NFs enable multiple applications to process on packets, and enable NFs to provide limitations on each other. For example, the first and last NFs can provide firewall-like functionality to limit which packets can be processed and transmitted by NFs in the middle of the chain. The core question this paper seeks to answer is: is it possible to practically meet end-to-end deadlines of packets while still maintaining high-throughput and strong isolation in a multi-tenant, edge-cloud for dynamic and dense workloads?

One tempting answer is to directly adapt existing deadlinedriven scheduling systems (e.g., EDF) to edge-clouds. We argue that this is not sufficient because (1) many edge cloud infrastructures do not support preemptive scheduling ( §II-A), (2) to optimize for meeting end-to-end deadlines across chains of computations, normal per-thread prioritization is not a good fit for dynamic workloads ( §III), and (3) high-throughput network systems seek to avoid per-message overheads, which is a bad match for OS abstractions that require locks and Inter-Processor Interrupts (IPIs) for coordination ( §III).

This paper presents Edge-RT, an OS infrastructure built on the public EdgeOS [9], that focuses on packet-or messagebased deadline scheduling across chains of computations, while maintaining high performance, density, and isolation between client computations. Edge-RT focuses on practical mechanisms to meet deadlines while minimising per-message system overheads: (1) it associates deadlines with packets, and threads inherit these message deadlines as packets flow through the computation chains to provide end-to-end, deadline-based scheduling, and (2) creates mechanisms for coordination and execution that avoid per-message overheads for scheduling, batching, and inter-FWP, inter-core coordination while bounding interference. Contributions. Edge-RT's contributions cente