LucidAction: A Hierarchical and Multi-model Dataset for Comprehensive Action Quality Assessment

Action Quality Assessment (AQA) research confronts formidable obstacles due to 1 limited, mono-modal datasets sourced from one-shot competitions, which hinder 2 the generalizability and comprehensiveness of AQA models. To address these 3 limitations, we present LucidAction, the first systematically collected multi-view 4 AQA dataset structured on curriculum learning principles. LucidAction features 5 a three-tier hierarchical structure, encompassing eight diverse sports events with 6 four curriculum levels, facilitating sequential skill mastery and supporting a wide 7 range of athletic abilities. The dataset encompasses multi-modal data, including 8 multi-view RGB video, 2D and 3D pose sequences, enhancing the richness of 9 information available for analysis. Leveraging a high-precision multi-view Motion 10 Capture (MoCap) system ensures precise capture of complex movements. Meticu-11 lously annotated data, incorporating detailed penalties from professional gymnasts, 12 ensures the establishment of robust and comprehensive ground truth annotations. 13 Experimental evaluations employing diverse contrastive regression baselines on 14 LucidAction elucidate the dataset’s complexities. Through ablation studies, we 15 investigate the advantages conferred by multi-modal data and fine-grained annota-16 tions, offering insights into improving AQA performance. The data and code will 17 be openly released to support advancements in the AI sports field. 18