Ego4o: Egocentric Human Motion Capture and Understanding from Multi-Modal Input

Output: Simultaneous Motion Capture and Understanding with Various Multi-Modal Inputs …walking in the bedroom, then she turns left and bends over to grab the… … is leaning forward while standing in the living area as she grabs and … The person is standing in the living area, then leans forward to grab the clothes. I am walking in my room… Motion Description Figure 1. Our method can use an egocentric image and 1-3 IMU sensors from wearable devices to accurately predict human motion and generate motion descriptions. Motion descriptions, when available, can also enhance motion capture accuracy. Ego4o supports flexible input combinations, functioning with or without images, or with varied IMU placements.