Abstract
Given an untrimmed video, repetitive actions counting aims to estimate the number of repetitions of class-agnostic actions. To handle the various length of videos and repetitive actions, also optimization challenges in end-to-end video model training, down-sampling is commonly utilized in recent state-of-the-art methods, leading to ignorance of several repetitive samples. In this paper, we attempt to understand repetitive actions from a full temporal resolution view, by combining offline feature extraction and temporal convolution networks. The former step enables us to train repetition counting network without down-sampling while preserving all repetitions regardless of the video length and action frequency, and the later network models all frames in a flexible and dynamically expanding temporal receptive field to retrieve all repetitions with a global aspect. Besides, temporal self-similarity matrix is used in our model to represent the correlation of action, which contains much cycle information in time series. We experimentally demonstrate that our method achieves better or comparable performance in three public datasets, i.e., TransRAC, UCFRep and QUVA. We expect this work will encourage our community to think about the importance of full temporal resolution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Azy, O., Ahuja, N.: Segmentation of periodically moving objects. In: 2008 19th International Conference on Pattern Recognition, pp. 1–4. IEEE (2008)
Briassouli, A., Ahuja, N.: Extraction and analysis of multiple periodic motions in video sequences. IEEE Trans. Pattern Anal. Mach. Intell. 29(7), 1244–1261 (2007)
Cutler, R., Davis, L.S.: Robust real-time periodic motion detection, analysis, and applications. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 781–796 (2000)
Dwibedi, D., Aytar, Y., Tompson, J., Sermanet, P., Zisserman, A.: Counting out time: class agnostic video repetition counting in the wild. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10387–10396 (2020)
Farha, Y.A., Gall, J.: MS-TCN: multi-stage temporal convolutional network for action segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3575–3584 (2019)
Feichtenhofer, C.: X3D: expanding architectures for efficient video recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 203–213 (2020)
Hu, H., Dong, S., Zhao, Y., Lian, D., Li, Z., Gao, S.: TransRAC: encoding multi-scale temporal correlation with transformers for repetitive action counting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19013–19022 (2022)
Lea, C., Flynn, M.D., Vidal, R., Reiter, A., Hager, G.D.: Temporal convolutional networks for action segmentation and detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 156–165 (2017)
Levy, O., Wolf, L.: Live repetition counting. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3020–3028 (2015)
Liu, Z., et al.: Video Swin transformer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3202–3211 (2022)
Liu, Z., Wang, L., Wu, W., Qian, C., Lu, T.: Tam: temporal adaptive module for video recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 13708–13718 (2021)
Pogalin, E., Smeulders, A.W., Thean, A.H.: Visual quasi-periodicity. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2008)
Soomro, K., Zamir, A.R., Shah, M.: UCF101: a dataset of 101 human actions classes from videos in the wild. ar**v preprint ar**v:1212.0402 (2012)
Thangali, A., Sclaroff, S.: Periodic motion detection and estimation via space-time sampling. In: 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION 2005)-Volume 1, vol. 2, pp. 176–182. IEEE (2005)
Tsai, P.S., Shah, M., Keiter, K., Kasparis, T.: Cyclic motion detection for motion based recognition. Pattern Recogn. 27(12), 1591–1603 (1994)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Yao, Z., Cheng, X., Zou, Y.: PoseRAC: pose saliency transformer for repetitive action counting. ar**v preprint ar**v:2303.08450 (2023)
Zhang, H., Xu, X., Han, G., He, S.: Context-aware and scale-insensitive temporal repetition counting. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 670–678 (2020)
Zhang, Y., Shao, L., Snoek, C.G.: Repetitive activity counting by sight and sound. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14070–14079 (2021)
Acknowledgements
This work is supported in part by the National Natural Science Foundation of China under Grant 62261160652; in part by the National Natural Science Foundation of China under Grant 61733011; in part by the National Natural Science Foundation of China under Grant 62206075; in part by the National Natural Science Foundation of China under Grant 52275013; in part by the GuangDong Basic and Applied Basic Research Foundation under Grant 2021A1515110438; in part by the Guangdong Basic and Applied Basic Research Foundation under Grant 2020B1515120064; in part by the Shenzhen Science and Technology Program under Grant JCYJ20210324120214040; in part by the National Key Research and Development Program of China under Grant 2022YFC3601700.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Li, J., Chen, B., Wang, Z., Liu, H. (2023). Full Resolution Repetition Counting. In: Yang, H., et al. Intelligent Robotics and Applications. ICIRA 2023. Lecture Notes in Computer Science(), vol 14267. Springer, Singapore. https://doi.org/10.1007/978-981-99-6483-3_48
Download citation
DOI: https://doi.org/10.1007/978-981-99-6483-3_48
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-6482-6
Online ISBN: 978-981-99-6483-3
eBook Packages: Computer ScienceComputer Science (R0)