Compressed Volumetric Heatmaps for Multi-Person 3D Pose Estimation

In this paper we present a novel approach for bottomup multi-person 3D human pose estimation from monocular RGB images. We propose to use high resolution volumetric heatmaps to model joint locations, devising a simple and effective compression method to drastically reduce the size of this representation. At the core of the proposed method lies our Volumetric Heatmap Autoencoder, a fully-convolutional network tasked with the compression of ground-truth heatmaps into a dense intermediate representation. A second model, the Code Predictor, is then trained to predict these codes, which can be decompressed at test time to re-obtain the original representation. Our experimental evaluation shows that our method performs favorably when compared to state of the art on both multi-person and single-person 3D human pose estimation datasets and, thanks to our novel compression strategy, can process full-HD images at the constant runtime of 8 fps regardless of the number of subjects in the scene. Code and models available at https://github.com/fabbrimatteo/LoCO . + × e-c2d e-c3d d -c3d d -c2d feature extractor f-c2d L2 Encoder e d Decoder Code Predictor f Code Predictor (train) f e VHA (train and eval.) d e f D'×H ''×W '' N×D '×H ''×W '' N×D '×H ''×W '' D '×H ''×W '' D '×H ''×W '' 3×H×W N×D×H '×W ' N×D×H '×W ' + concat. × deconcat. Code Predictor (eval.