ISAC: In-Switch Approximate Cache for IoT Object Detection and Recognition

In object detection and recognition, similar but nonidentical sensing data probably maps to the same result. Therefore, a cache preserving popular results to support approximate match for similar input requests can accelerate the task by avoiding the otherwise expensive deep learning model inferences. However, the current software and hardware practices carried on edge or cloud servers are inefficient in either cost or performance. Taking advantage of the on-path programmable switches, we propose In-Switch Approximate Cache (ISAC) to reduce the server workload and latency. The unique approximate matching requirement sets ISAC apart from a conventional exact-match cache. Equipped with efficient encoding and qualifying algorithms, ISAC in an on-path switch can fulfill most of the input requests with high accuracy. When adapting to a P4 programmable switch, it can sustain up to 194M frames per second and fulfill 60.3% of them, achieving a considerable reduction on detection latency, server cost, and power consumption. Readily deployable in existing network infrastructure, ISAC is the first-of-its-kind approximate cache that can be completely implemented in a switch to support a class of IoT applications.