OakInk2 : A Dataset of Bimanual Hands-Object Manipulation in Complex Task Completion

Use the knife to cut the apple; then use the clamp to grip the sugar cubes into the bowl; afterwards, use the microwave oven to heat the bowl.

L2 L3 L9 OPEN CLOSE OPEN CLOSE CLOSE OPEN <heat, sth> Figure 1. An overview of the data and content of our proposed OAKINK2 dataset. OAKINK2 dataset focuses on bimanual object manipulation tasks for complex daily activities. 1) The top row shows the data collection process, including the task setup (top-left panel), human demonstration (top-center), and annotation (top-right).

2. The second row shows the three levels of abstraction constructed by OAKINK2 for complex tasks, including the Affordance, Primitive Task, and Complex Task. OAKINK2 dataset provides allocentric and egocentric videos of human manipulation process, as well as the corresponding 3D-pose annotation and task specification.