An Efficient Deep Learning Accelerator for Compressed Video Analysis

Previous neural network accelerators tailored to video analysis only accept data of RGB/YUV domain, requiring decompressing the video that are often compressed before transmitted from the edge sensors. A compressed video processing accelerator can remove the decoding overhead, and gain performance speedup by operating on more compact input data. This work proposes a novel deep learning accelerator architecture, Alchemist, which predicts results directly from the compressed video bitstream instead of reconstructing the full RGB images. By utilizing the metadata of motion vector and critical blocks extracted from bitstream, Alchemist contributes to remarkable performance speedup of 5x with negligible accuracy loss.