SpringerLink
Log in

Crowd Movement Type Estimation in Video by Integral Optical Flow and Convolution Neural Network

  • SELECTED PAPERS
  • Published:
Pattern Recognition and Image Analysis Aims and scope Submit manuscript

Abstract

The paper proposes a new approach for crowd movement type estimation in video by combining convolutional neural network and integral optical flow. At first, main notions of crowd detection and tracking are given. Secondly, crowd movement features and parameters are defined. Three rules are proposed to identify direct crowd motion. Signs are presented for identifying chaotic crowd movement. Region movement indicators are introduced to analyze the movement of a group of people or a crowd. Thirdly, an algorithm of crowd movement types estimation using convolutional neural network and integral optical flow is proposed. We calculate crowd movement trajectories and show how they can be used to analyze behavior and divide crowds into groups of people. Experimental results show that with the help of convolutional neural network and integral optical flow crowd movement parameters can be calculated more accurately and quickly. The algorithm demonstrates stronger robustness to noise and the ability to get more accurate boundaries of moving objects.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.

REFERENCES

  1. S. Ali and M. Shah, “A Lagrangian particle dynamics approach for crowd flow segmentation and stability analysis,” in 2007 IEEE Conf. on Computer Vision and Pattern Recognition, Minneapolis, Minn., 2007 (IEEE, 2007), pp. 1–6. https://doi.org/10.1109/cvpr.2007.382977

  2. M. R. Bhuiyan, J. Abdullah, N. Hashim, and F. Al Farid, “Video analytics using deep learning for crowd analysis: a review,” Multimedia Tools Appl. 81, 27895–27922 (2022). https://doi.org/10.1007/s11042-022-12833-z

    Article  Google Scholar 

  3. H. Chen, O. Nedzvedz, S. Ye, and S. Ablameyko, “Crowd abnormal behaviour identification based on integral optical flow in video surveillance systems,” Informatica 29, 211–232 (2018). https://doi.org/10.15388/informatica.2018.164

    Article  MathSciNet  Google Scholar 

  4. C. Chen and Yu. Shao, “Anomalous crowd behavior detection and localization in video surveillance,” in 2014 IEEE Int. Conf. on Control Science and Systems Engineering, Yantai, China, 2014 (IEEE, 2014), pp. 190–194. https://doi.org/10.1109/ccsse.2014.7224535

  5. M. Hu, S. Ali, and M. Shah, “Learning motion patterns in crowded scenes using motion flow field,” in 2008 19th Int. Conf. on Pattern Recognition, Tampa, Fla., 2008 (IEEE, 2008), pp. 1–5. https://doi.org/10.1109/icpr.2008.4761183

  6. T-W. Hui and C. C. Loy, “LiteFlowNet3: Resolving correspondence ambiguity for more accurate optical flow estimation,” in Computer Vision–ECCV 2020, Ed. by A. Vedaldi, H. Bischof, T. Brox, and J. M. Frahm, Lecture Notes in Computer Science, Vol. 12365 (Springer, Cham, 2020), pp. 169–184. https://doi.org/10.1007/978-3-030-58565-5_11

    Book  Google Scholar 

  7. R. Mehran, A. Oyama, and M. Shah, “Abnormal crowd behavior detection using social force model,” in Proc. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, Miami, 2009 (IEEE, 2009), pp. 935–942. https://doi.org/10.1109/CVPR.2009.5206641

  8. M. Ravanbakhsh, M. Nabi, H. Mousavi, E. Sangineto, and N. Sebe, “Plug-and-play cnn for crowd motion analysis: An application in abnormal event detection,” ar**v preprint at https://arxiv.org/abs/1610.00307v1, Cornell University Library, April, 25, (2017). https://doi.org/10.48550/ar**v.1610.00307

  9. J. Shao, C. C. Loy, K. Kang, and X. Wang, “Slicing convolutional neural network for crowd video understanding,” in Proc. IEEE Conf. on Computer Vision and Pattern Recognition, Las Vegas, 2016 (2016), pp. 5620–5628. https://doi.org/10.1109/CVPR.2016.606

  10. P. Shreedevi and H. S. Mohana, “Video analysis to recognize unusual crowd behavior for surveillance systems: A review,” in Third Congress on Intelligent Systems, Ed. by S. Kumar, H. Sharma, K. Balachandran, J. H. Kim, and J. C. Bansal, Lecture Notes in Networks and Systems, Vol. 613 (Springer, Singapore, 2023), pp. 285–304. https://doi.org/10.1007/978-981-19-9379-4_22

  11. N. A. Sjarif, S. M. Shamsuddin, and S. Z. Hashim, “Detection of abnormal behaviors in crowd scene: A review,” Int. J. Adv. Software Comput. Appl. 4 (1), 1–33 (2012).

    Google Scholar 

  12. Q. Wang, Q. Ma, C.-H. Luo, H.-Y. Liu, and C.‑L. Zhang, “Hybrid histogram of oriented optical flow for abnormal behavior detection in crowd scenes,” Int. J. Pattern Recognit. Artif. Intell. 30, 210–224 (2016). https://doi.org/10.1142/S0218001416550077

    Article  MathSciNet  Google Scholar 

  13. L. Zhang and J. Han, “Recognition of abnormal behavior of crowd based on spatial location feature,” in IEEE 9th Joint Int. Information Technology and Artificial Intelligence Conf. (ITAIC), Chongqing, China, 2020 (IEEE, 2020), pp. 736–741. https://doi.org/10.1109/ITAIC49862.2020.9338944

  14. W. Zhang, Y. Hou, and S. Wang, “Event recognition of crowd video using corner optical flow and convolutional neural network,” Proc. SPIE 10033, 100335K (2016). https://doi.org/10.1117/12.2245305

  15. R. Zhao, D. Dong, Y. Wang, C. Li, Y. Ma, and V. F. Enríquez, “Image-Based crowd stability analysis using improved multi-column convolutional neural network,” IEEE Trans. Intell. Transp. Syst. 23, 5480–5489 (2022). https://doi.org/10.1109/TITS.2021.3054376

    Article  Google Scholar 

  16. Z. Zhao and R. Li, “Bayesian cellular automata fusion model based on dual-stream strategy for video anomaly action detection,” Pattern Recognit. Image Anal. 31, 688–698 (2021). https://doi.org/10.1134/S1054661821040301

    Article  Google Scholar 

  17. L. Zhu, C. Li, Y. Zhongguo, K. Yuan, and S. Wang, “Crowd density estimation based on classification activation map and patch density level,” Neural Comput. Appl. 32, 5105–5116 (2020). https://doi.org/10.1007/s00521-018-3954-7

Download references

Funding

This work is supported by the National High-End Foreign Experts’ Program (G2021016028L), Zhejiang Provincial Natural Science Foundation of China under grant no. LGF19F020016 and Zhejiang Shuren University Basic Scientific Research Special Funds.

Author information

Authors and Affiliations

  1. Zhejiang Shuren University, 310015, Hangzhou, China

    Huafeng Chen & Shi** Ye

  2. International Science and Technology Cooperation Base of Zhejiang Province: Remote Sensing Image Processing and Application, 310000, Hangzhou, China

    Huafeng Chen

  3. Belarusian State University, 220030, Minsk, Republic of Belarus

    Angelina Pashkevich, Shi** Ye & Sergey Ablameyko

  4. Polotsk State University, 211440, Novopolotsk, Republic of Belarus

    Rykhard Bohush

  5. United Institute for Informatics Problems, National Academy of Sciences of Belarus, 220012, Minsk, Republic of Belarus

    Sergey Ablameyko

Authors
  1. Huafeng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Angelina Pashkevich
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shi** Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rykhard Bohush
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sergey Ablameyko
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Huafeng Chen, Angelina Pashkevich, Shi** Ye, Rykhard Bohush or Sergey Ablameyko.

Ethics declarations

The authors of this work declare that they have no conflicts of interest.

Additional information

Huafeng Chen. Born in 1982. Associate Professor of Zhejiang Shuren University. Graduated from Zhejiang University in 2003. In 2009 he got his PhD in the field of Earth Exploration and Information Technology at the Institute of Space Information and Technique, Zhejiang University. His scientific interests include remote sensing image processing, GIS application, image and video processing, multiagent system. He has published more than 20 academic articles.

Angelina Pashkevich. Graduated from Belarussian State University in 2020. In 2022 she received Master’s degree in the field of computer mathematics and systems analysis in Belarussian State University. Currently she is postgraduate student in Belarussian State University. Her scientific interests are detection and tracking of objects during video surveillance.

Shi** Ye. Born in 1967. Graduated from Zhejiang University in 1988. He received his Master’s degree in Computer Science and Technology from Zhejiang University in 2003. Professor and Vice President of Zhejiang Shuren University. His scientific interests include the application of computer graphics and images, GIS. Author of more than 70 scientific articles. He has taken part in four research projects and was awarded second prize of Zhejiang Provincial Scientific and Technological Achievement. Two of his teaching research programs won the first prize and second prize of Zhejiang Provincial Teaching Achievement.

Rykhard Bohush. Graduated from Polotsk State University in 1997. In 2002, he received his Candidate of Sciences degree, and in 2022, he received his Doctor of Sciences degree. Head of Computer Systems and Networks Department of Polotsk State University. His scientific interests include image and video processing, intelligent systems, and machine learning.

Sergey Ablameyko. Born in 1956, DipMath in 1978, PhD in 1984, DSc in 1990, Prof. in 1992. Professor of Belarusian State University. His scientific interests are: image analysis, pattern recognition, digital geometry, knowledge-based systems, geographical information systems, medical imaging. He is in Editorial Board of Pattern Recognition and Image Analysis and many other international and national journals. He is a Fellow of IAPR, Fellow of Belarusian Engineering Academy, Academician of National Academy of Sciences of Belarus, Academician of the European Academy, and others. He was a First Vice-President of International Association for Pattern Recognition IAPR (2006–2008), President of Belarusian Association for Image Analysis and Recognition.

Publisher’s Note.

Pleiades Publishing remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huafeng Chen, Pashkevich, A., Ye, S. et al. Crowd Movement Type Estimation in Video by Integral Optical Flow and Convolution Neural Network. Pattern Recognit. Image Anal. 34, 266–274 (2024). https://doi.org/10.1134/S1054661824700068

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S1054661824700068

Keywords:

Search

Navigation