ManiVideo: Generating Hand-Object Manipulation Video with Dexterous and Generalizable Grasping

Occlusion-free normal maps Occlusion confidence maps Confidence occlusion occlusion-free Unseen objects Generated results Human-centered results Segmentation Figure 1. ManiVideo: We propose a novel framework for generalizable and dexterous hand-object manipulation video generation. Left: Given several reference images of unseen objects, our method generates realistic and plausible manipulation videos driven by hand-object signals. By integrating multiple datasets, ManiVideo supports applications such as human-centered manipulation video generation. Right:

To ensure hand-object consistency, we introduce a multi-layer occlusion representation capable of learning 3D occlusion relationships from occlusion-free normal maps and occlusion confidence maps.