Abstract
This paper takes initial strides at designing and evaluating a vision-based system for privacy ensured activity monitoring. The proposed technology utilizing Artificial Intelligence (AI)-empowered proactive systems offering continuous monitoring, behavioral analysis, and modeling of human activities. To this end, this paper presents Single Run Action Detector (S-RAD) which is a real-time privacy-preserving action detector that performs end-to-end action localization and classification. It is based on Faster-RCNN combined with temporal shift modeling and segment based sampling to capture the human actions. Results on UCF-Sports and UR Fall dataset present comparable accuracy to State-of-the-Art approaches with significantly lower model size and computation demand and the ability for real-time execution on edge embedded device (e.g. Nvidia Jetson Xavier).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Alaoui, A.Y., El Fkihi, S., Thami, R.O.H.: Fall detection for elderly people using the variation of key points of human skeleton. IEEE Access 7, 154786–154795 (2019)
Atallah, L., Lo, B., King, R., Yang, G.Z.: Sensor positioning for activity recognition using wearable accelerometers. IEEE Trans. Biomed. Circ. Syst. 5(4), 320–329 (2011)
Cameiro, S.A., da Silva, G.P., Leite, G.V., Moreno, R., GuimarĂ£es, S.J.F., Pedrini, H.: Multi-stream deep convolutional network using high-level features applied to fall detection in video sequences. In: 2019 International Conference on Systems, Signals and Image Processing (IWSSIP), pp. 293–298 (2019)
Chen, K., et al.: MMDetection: Open mmlab detection toolbox and benchmark. ar**v preprint ar**v:1906.07155 (2019)
Duarte, K., Rawat, Y.S., Shah, M.: Videocapsulenet: a simplified network for action detection. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS 2018, pp. 7621–7630. Curran Associates Inc., Red Hook (2018)
Everingham, M., Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes (voc) challenge. Int. J. Comput. Vision 88(2), 303–338 (2010). https://doi.org/10.1007/s11263-009-0275-40275-4
Gkioxari et al., Georgia, J.M.: Finding action tubes. CoRR abs/1411.6031 (2014). http://arxiv.org/abs/1411.6031
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR abs/1512.03385 (2015). http://arxiv.org/abs/1512.03385
Hou, R., Chen, C., Shah, M.: Tube convolutional neural network (t-cnn) for action detection in videos. In: The IEEE International Conference on Computer Vision (ICCV), October 2017
Kalogeiton, V., Weinzaepfel, P., Ferrari, V., Schmid, C.: Action tubelet detector for spatio-temporal action localization. CoRR abs/1705.01861 (2017). http://arxiv.org/abs/1705.01861
Kay, W., et al.: The kinetics human action video dataset. ar**v preprint ar**v:1705.06950 (2017)
Kwolek, B., Kepski, M.: Human fall detection on embedded platform using depth maps and wireless accelerometer. Comput. Methods Programs Biomed. 117(3), 489–501 (2014)
Leite, G., Silva, G., Pedrini, H.: Fall detection in video sequences based on a three-stream convolutional neural network. In: 2019 18th IEEE International Conference On Machine Learning and Applications (ICMLA), pp. 191–195 (2019)
Lin, J., Gan, C., Han, S.: Temporal shift module for efficient video understanding. CoRR abs/1811.08383 (2018). http://arxiv.org/abs/1811.08383
Liu, W., et al.: SSD: single shot multibox detector. CoRR abs/1512.02325 (2015). http://arxiv.org/abs/1512.02325
Lu, N., Wu, Y., Feng, L., Song, J.: Deep learning for fall detection: three-dimensional CNN combined with LSTM on video kinematic data. IEEE J. Biomed. Health Inform. 23(1), 314–323 (2019)
Mirzadeh, S.I., Ghasemzadeh, H.: Optimal policy for deployment of machine learning models on energy-bounded systems. In: Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI) (2020)
Neff, C., Mendieta, M., Mohan, S., Baharani, M., Rogers, S., Tabkhi, H.: Revamp2t: real-time edge video analytics for multicamera privacy-aware pedestrian tracking. IEEE Internet Things J. 7(4), 2591–2602 (2020)
Pagan, J., et al.: Toward ultra-low-power remote health monitoring: an optimal and adaptive compressed sensing framework for activity recognition. IEEE Trans. Mobile Comput. (TMC) 18(3), 658–673 (2018)
Peng et al., **aojiang, S.C.: Multi-region two-stream R-CNN for action detection. Lecture Notes in Computer Science, vol. 9908, pp. 744–759. Springer, Amsterdam, Netherlands, October 2016. https://doi.org/10.1007/978-3-319-46493-0_45, https://hal.inria.fr/hal-01349107
Ren, S., He, K., Girshick, R.B., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. CoRR abs/1506.01497 (2015). http://arxiv.org/abs/1506.01497
Saeedi, R., Purath, J., Venkatasubramanian, K., Ghasemzadeh, H.: Toward seamless wearable sensing: automatic on-body sensor localization for physical activity monitoring. In: 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 5385–5388. IEEE (2014)
Simonyan, K., Zisserman, A.: Two-stream convolutional networks for action recognition in videos. CoRR abs/1406.2199 (2014). http://arxiv.org/abs/1406.2199
Soomro, K., Zamir, A.R.: Action recognition in realistic sports videos (2014)
Tran, D., Bourdev, L.D., Fergus, R., Torresani, L., Paluri, M.: C3D: generic features for video analysis. CoRR abs/1412.0767 (2014). http://arxiv.org/abs/1412.0767
Wang, L., **ong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., Gool, L.V.: Temporal segment networks: Towards good practices for deep action recognition. CoRR abs/1608.00859 (2016). http://arxiv.org/abs/1608.00859
Weinzaepfel, P., Harchaoui, Z., Schmid, C.: Learning to track for spatio-temporal action localization. CoRR abs/1506.01929 (2015). http://arxiv.org/abs/1506.01929
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Saravanan, A., Sanchez, J., Ghasemzadeh, H., Macabasco-O’Connell, A., Tabkhi, H. (2021). Single Run Action Detector over Video Stream - A Privacy Preserving Approach. In: Li, X., Wu, M., Chen, Z., Zhang, L. (eds) Deep Learning for Human Activity Recognition. DL-HAR 2021. Communications in Computer and Information Science, vol 1370. Springer, Singapore. https://doi.org/10.1007/978-981-16-0575-8_7
Download citation
DOI: https://doi.org/10.1007/978-981-16-0575-8_7
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-0574-1
Online ISBN: 978-981-16-0575-8
eBook Packages: Computer ScienceComputer Science (R0)