SpringerLink
Log in

Video anomaly localization using modified faster RCNN with soft NMS algorithm

  • Regular Paper
  • Published:
International Journal of Data Science and Analytics Aims and scope Submit manuscript

Abstract

Localization of anomalies in surveillance videos is a critical component of smart and intelligent surveillance systems. The goal of anomaly detection is to automatically detect the presence of anomalies in a short amount of time. The proposed system developed an efficient and improved faster RCNN-based system for accurate detection of anomalies. The modified version of Faster RCNN extracts the features at different levels by using a feature pyramid network and is able to detect small-scale anomalies by adding a Soft NMS algorithm. The proposed model is experimentally evaluated using three benchmarked datasets UCSD Ped1, UCSD Ped2 and Avenue, and gets better detection performance. Finally, a comparison study with different YOLO series is conducted, and our system outperforms them.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Data availability

Publically available benchmarked dataset are used. 1.UCSD Ped1 and UCSD Ped2: http://www.svcl.ucsd.edu/projects/anomaly UCSD\(_A\)nomaly\(_D\)ataset.tar.gz. 2.Avenue: http://www.cse.cuhk.edu.hk/leojia/projects/detectabnormal/dataset.html.

References

  1. Zhang, X., et al.: Video anomaly detection and localization using motion-field shape description and homogeneity testing. Pattern Recognit. 105, 107394 (2020)

    Article  Google Scholar 

  2. Pimentel, M.A.F., Clifton, D.A., Clifton, L., Tarassenko, L.: A review of novelty detection. Signal Process. 99, 215–249 (2014)

    Article  Google Scholar 

  3. Popoola, O.P., Wang, K.: Video-based abnormal human behavior recognition—a review. IEEE Trans. Syst. Man Cybern. C Appl. Rev 42(6), 865–878 (2012)

    Article  Google Scholar 

  4. Morris, B.T., Trivedi, M.M.: A survey of vision-based trajectory learning and analysis for surveillance. IEEE Trans. Circuits Syst. Video Technol. 18(8), 1114–1127 (2008)

    Article  Google Scholar 

  5. Xu, K., Jiang, X., Sun, T.: Anomaly detection based on stacked sparse coding with intraframe classification strategy. IEEE Trans. Multim. 20(5), 1062–1074 (2018)

    Article  Google Scholar 

  6. Ren, S., et al.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, vol. 28 (2015)

  7. Bodla, N., et al.: Soft-NMS—improving object detection with one line of code. In: Proceedings of the IEEE International Conference on Computer Vision (2017)

  8. Tripathi, G., Singh, K.: Convolutional neural networks for crowd behaviour analysis: a survey. Vis. Comput. 1, 24 (2018)

    Google Scholar 

  9. Piciarelli, C., Micheloni, C., Foresti, G.L.: Trajectory-based anomalous event detection. IEEE Trans. Circuits Syst. Video Technol. 18(11), 1544–1554 (2008)

    Article  Google Scholar 

  10. Sabokrou, M., Fayyaz, M., Fathy, M., Klette, R.: Deep-cascade: cascading 3D deep neural networks for fast anomaly detection and localization in crowded scenes. IEEE Trans. Image Process. 26(4), 1992–2004 (2017)

    Article  MathSciNet  Google Scholar 

  11. Anoopa, S., Salim, A.: Survey on anomaly detection in surveillance videos. Mater. Today: Proc. 58, 162–167 (2022)

    Google Scholar 

  12. Nawaratne, R., Bandaragoda,T., Adikari, A., Alahakoon, D., De Silva, D., Yu, X.: Incremental knowledge acquisition and selflearning for autonomous video surveillance. In: IECON 2017—43rd Annual Conference of the IEEE Industrial Electronics Society, pp. 4790–4795 (2017)

  13. Hu, X., Hu, S., Huang, Y., Zhang, H., Wu, H.: Video anomaly detection using deep incremental slow feature analysis network. IET Comput. Vis. 10(4), 258–265 (2016)

    Article  Google Scholar 

  14. Zhou, S., Shen, W., Zeng, D., Fang, M., Wei, Y., Zhang, Z.: Spatial–temporal convolutional neural networks for anomaly detection and localization in crowded scenes. Signal Process.: Image Commun. 47, 358–368 (2016)

    Google Scholar 

  15. Feng, Y., Yuan, Y., Lu, X.: Learning deep event models for crowd anomaly detection. Neurocomputing 219, 548–556 (2017)

    Article  Google Scholar 

  16. Ravanbakhsh, M., et al.: Plug-and-play CNN for crowd motion analysis: an application in abnormal event detection. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE (2018)

  17. Sabokrou, M., et al.: Deep-anomaly: fully convolutional neural network for fast anomaly detection in crowded scenes. Comput. Vis. Image Underst. 172, 88–97 (2018)

    Article  Google Scholar 

  18. Cai, Y., et al.: Video anomaly detection with multi-scale feature and temporal information fusion. Neurocomputing 423, 264–273 (2021)

    Article  Google Scholar 

  19. Song, J.-F., et al.: Video anomaly detection based on optical flow feature enhanced spatio-temporal feature network FusionNet-LSTM-G. IEEE Access 10, 130314–130325 (2022)

    Article  Google Scholar 

  20. Li, Q., et al.: Attention-based anomaly detection in multi-view surveillance videos. Knowl.-Based Syst. 252, 109348 (2022)

    Article  Google Scholar 

  21. Tang, J., et al.: SAE-PPL: self-guided attention encoder with prior knowledge-guided pseudo labels for weakly supervised video anomaly detection. J. Vis. Commun. Image Represent. 97, 103967 (2023)

    Article  Google Scholar 

  22. He, P., et al.: Adversarial and focused training of abnormal videos for weakly-supervised anomaly detection. Pattern Recognit. 147, 110119 (2024)

    Article  Google Scholar 

  23. Varadarajan, J., et al.: Active online anomaly detection using Dirichlet process mixture model and gaussian process classification. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE (2017)

  24. **e, S., Zhang, X., Cai, J.: Video crowd detection and abnormal behavior model detection based on machine learning method. Neural Comput. Appl. 31(1), 175–184 (2019)

    Article  Google Scholar 

  25. Farneback, G.: Very high accuracy velocity estimation using orientation tensors, parametric motion, and simultaneous segmentation of the motion field. In: Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, vol. 1. IEEE (2001)

  26. Anoopa, S., Salim, A., Beevi, N.: Advanced video anomaly detection using 2D CNN and stacked LSTM with deep active learning-based model: 10.48129/kjs. splml. 19159. Kuwait J. Sci. (2022)

  27. He, K., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)

  28. Adelson, E.H., et al.: Pyramid methods in image processing. RCA Eng. 29.6, 33–41 (1984)

    Google Scholar 

  29. Felzenszwalb, P.F., Girshick, R.B., McAllester, D., et al.: Object detection with discriminatively trained part-based models. In: Proceedings of IEEE Transactions on Pattern Analysis and Machine Intelligence [S. 1.], pp. 201–205. TPAMI Press (2010)

  30. Bodla, N., Singh, B., Chellappa, R., et al.: Improving object detection with one line of code. In: 2017 IEEE International Conference on Computer Vision (ICCV) (2017)

  31. Mahadevan, V., Li, W., Bhalodia, V., Vasconcelos, N.: Anomaly detection in crowded scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1975–1981 (2010)

  32. Lab, S.V.C.: UCSD anomaly data set (2014)

  33. Lu, C., Shi, J., Jia, J.: Abnormal event detection at 150 fps in Matlab. In: Proceedings of the IEEE International Conference on Computer Vision (2013). http://www.cse.cuhk.edu.hk/leojia/ projects/detectabnormal/dataset.html

Download references

Funding

This work received no specific grant from any funding agency, commercial or not-for-profit sectors

Author information

Author notes

  1. A. Salim and S. Nadera Beevi have contributed equally to this work.

Authors and Affiliations

  1. Computer Science and Engineering, College of Engineering Trivandrum, Sreekaryam, Trivandrum, Kerala, 695006, India

    S. Anoopa & A. Salim

  2. Computer Application, TKM College of Engineering, Karicode, Kollam, Kerala, 691005, India

    S. Nadera Beevi

Authors
  1. S. Anoopa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Salim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S. Nadera Beevi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

S. Anoopa wrote the manuscript and did experiments. A. Salim helped in the experiment section, prepared all figures and reviewed the manuscript. S. Nadera Beevi helped in the experiment section and reviewed the manuscript.

Corresponding author

Correspondence to S. Anoopa.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Anoopa, S., Salim, A. & Nadera Beevi, S. Video anomaly localization using modified faster RCNN with soft NMS algorithm. Int J Data Sci Anal (2024). https://doi.org/10.1007/s41060-024-00591-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s41060-024-00591-0

Keywords

Search

Navigation