Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives

Ego-Exo4D studies skilled human activity through simultaneously captured egocentric and exocentric video. Its scenarios include sports, music, dance, and bicycle repair, where technique becomes clearer when the performer's viewpoint and the wider scene are available together.

The expanded dataset reports 1,286 hours of combined video from 740 participants in 13 cities and 123 natural scene contexts. Individual recordings run from one to 42 minutes, preserving extended procedures rather than isolated actions.

The collection adds multichannel audio, eye gaze, 3D point clouds, camera poses, IMU data, and paired language descriptions. Expert commentary from coaches and teachers provides domain-specific explanation alongside the recorded activity.

Its benchmark suite covers fine-grained activity understanding, proficiency estimation, cross-view translation, and 3D hand and body pose. These tasks connect capture design directly to how skilled performance is analyzed.

The cited arXiv record provides the complete dataset protocol, benchmark definitions, author list, and open resources.