FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation

In this work, we present FFB6D, a Full Flow Bidirectional fusion network designed for 6D pose estimation from a single RGBD image. Our key insight is that appearance information in the RGB image and geometry information from the depth image are two complementary data sources, and it still remains unknown how to fully leverage them. Towards this end, we propose FFB6D, which learns to combine appearance and geometry information for representation learning as well as output representation selection. Specifically, at the representation learning stage, we build bidirectional fusion modules in the full flow of the two networks, where fusion is applied to each encoding and decoding layer. In this way, the two networks can leverage local and global complementary information from the other one to obtain better representations. Moreover, at the output representation stage, we designed a simple but effective 3D keypoints selection algorithm considering the texture and geometry information of objects, which simplifies keypoint localization for precise pose estimation. Experimental results show that our method outperforms the state-of-the-art by large margins on several benchmarks. Code and video are available at https://github.com/ethnhe/FFB6D.git . Pose Estimation Dense Fusion CNN Encoder CNN Decoder Point Cloud Decoder Point Cloud Encoder (a) The DenseFusion [65] Network. The two networks extract features from different modalities of data separately without any communication, util the final layers of the encoding-decoding architecture. Fusion module Pose Estimation Concatenate CNN Encoder CNN Decoder Point Cloud Decoder Point Cloud Encoder