CVPR Findings2026
Count What Repeats: Period-Adaptive Multi-Scale Consistency for Self-Supervised Repetitive Action Counting
Shizhao Gao, Jun Li, Qiming Li
Abstract
Repetitive action counting is a fundamental yet challenging task in video understanding. Most existing approaches rely on full supervision with per-video count labels, which limits scalability and generalization. To overcome these limitations, we present a fully self-supervised framework that learns the periodic structure of human motion directly from unlabeled skeleton sequences. The core of our approach is the Period-Adaptive Multi-Scale Temporal Cycle Consistency (PAMS TCC) objective, which jointly aligns cycle phases across multiple temporal scales within a video and separates features across videos. This innovative objective successfully yields a highly robust, tempo-invariant embedding, achieving this result in a fully self-supervised manner without manual labels. A multi-expert consensus algorithm further enhances the counting robustness during inference by dynamically adapting peak-detection parameters and aggregating predictions across scales. This algorithm can substantially stabilize the counting process even under large speed variations. Extensive experiments demonstrate state-of-the-art performance: our method achieves 0.129 Mean Absolute Error (MAE) / 0.842 Off-By-One (OBO) on RepCount and 0.098 MAE / 0.879 OBO on Countix, outperforming all previous supervised baselines. These results establish a label-free, tempo-robust, and scalable solution that advances self-supervised repetitive action counting.
Ask Lune
Know this paper in minutes, not an afternoon.
Lune can read every page of it and the top-tier papers that cite it, and gives you its claims, its evidence and its limits, each one cited to the paper it comes from. No skimming, no AI slop.
Lineage
- Spatiotemporal Contrastive Video Representation Learning cites A Simple Framework for Contrastive Learning of Visual Representations
- Spatiotemporal Contrastive Video Representation Learning cites SlowFast Networks for Video Recognition
- TransRAC: Encoding Multi-scale Temporal Correlation with Transformers for Repetitive Action Counting cites Counting Out Time: Class Agnostic Video Repetition Counting in the Wild
Builds on8
- A Simple Framework for Contrastive Learning of Visual RepresentationsTing Chen, Simon Kornblith, Mohammad Norouzi, Geoffrey E. HintonICML 2020 · 24,064 citations
- SlowFast Networks for Video RecognitionChristoph Feichtenhofer, Haoqi Fan, Jitendra Malik, Kaiming HeICCV 2019 · 4,104 citations
- TransRAC: Encoding Multi-scale Temporal Correlation with Transformers for Repetitive Action CountingHuazhang Hu, Sixun Dong, Yiqun Zhao, Dongze Lian et al.CVPR 2022 · 57 citations
- Spatiotemporal Contrastive Video Representation LearningRui Qian, Tianjian Meng, Boqing Gong, Ming-Hsuan Yang et al.CVPR 2021
- Counting Out Time: Class Agnostic Video Repetition Counting in the WildDebidatta Dwibedi, Yusuf Aytar, Jonathan Tompson, Pierre Sermanet et al.CVPR 2020
Related papers
- Self-supervised 3D Skeleton Action Representation Learning with Motion Consistency and ContinuityYukun Su, Guosheng Lin, Qingyao WuICCV 2021 · 86 citations
- D^2-STX: Decoupling Spatial-Temporal Cross-Attention for Dual-branch Repetitive Action CountingXiaoai Wang, Hang Wang, Yan Liu, Huan Hu, Bruce X.B. YuCVPR Findings 2026 · 1 citation
- Learning to Predict Activity Progress by Self-Supervised Video AlignmentGerard Donahue, Ehsan ElhamifarCVPR 2024
- Context-Aware and Scale-Insensitive Temporal Repetition CountingHuaidong Zhang, Xuemiao Xu, Guoqiang Han, Shengfeng HeCVPR 2020
- CountLLM: Towards Generalizable Repetitive Action Counting via Large Language ModelZiyu Yao, Xuxin Cheng, Zhiqi Huang, Lei LiCVPR 2025
