Abstract
There is a strong demand of smart vision based surveillance system owing to the increase in crime at a frightening rate at various public places like Banks, Airport, Shop** malls and its application in human activity recognition ranges from patient fall detection, irregular pattern recognition or Human computer Interaction. As the crime increases at a disturbing rate, public security violations and high cost of security personals have motivated the author to do the strategic survey of existing vision and image processing based techniques in the past literature. The paper begins with discussing the common approach towards suspicious activity detection and recognition followed by summarizing the supervised and unsupervised machine learning methodologies mainly based on SVM, HMM and ANN classifiers, which were adopted by the researchers previously varying from single human behavior modeling to crowded scenes. Next, this paper discusses system model for human’s normal and abnormal activities recognition along with various feature selectors and detectors used in previous literature. This was followed by conducting a review of benchmark researches which covered a comprehensive state of art methodologies in the related fields, key points owned, feature learning and applications. At last experimental aspects of various papers have been discussed with essential performance matrices like accuracy along with the major issues, common problems, challenges and future scope in the related field.
Similar content being viewed by others
References
Gowsikhaa D, Abirami S, Baskaran R (2012) Automated human behavior analysis from video surveillance: a survey. Artif Intell Rev pp 1–19
Lavee G, Khan L, Thuraisingham B (2007) A framework for a video analysis tool for suspicious event detection. Multimed Tools Appl 35:109–123
Wiliem A, Madasu V, Boles W, Yarlagadda P (2008) Detecting uncommon trajectories. In: Proceedings of Digital image computing: techniques and applications, pp 398–404
Benezeth Y, Jodoin P, Saligrama V, Rosenberger C (2009) Abnormal events detection based on spatio-temporal co-occurrences. In: Proceedings of IEEE conference on computer vision pattern recognition workshops, pp 2458–2465
Chen Y, Liang G, Ka KL, Xu Y (2007) Abnormal behavior detection by multi-SVM-based Bayesian network. In: Proceedings of international conference information acquisition, pp 298–303
Duque D, Santos H, Cortez P (2007) Prediction of abnormal behaviors for intelligent video surveillance systems. In: Computational intelligence and data mining, 2007. CIDM 2007. IEEE Symposium on, pp 362–367
Vaswani N, Roy-Chowdhury AK, Chellappa R (2005) Shape activity: a continuous-state HMM for moving/deforming shapes with application to abnormal activity detection. IEEE Trans Image Process 14(10):1603–1616
Duong TV, Bui HH, Phung DQ, Venkatesh S (2005) Activity recognition and abnormality detection with the switching hidden semi-Markov model. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition, vol 1, pp 838–845
Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput Surv 38
Bakar ZA, Mohemad R, Ahmad A, Deris MM (2006) A comparative study for outlier detection techniques in data mining. In: Proceedings of IEEE conference on cybernetics and intelligent systems, pp 1–6
Pantic M, Pentland A, Nijholt A, Huang TS (2006) Human computing and machine understanding of human behavior: a survey. In: Proceedings of ICMI 2006 and IJCAI 2007 international conference on artificial intelligence human computing (ICMI’06/IJCAI’07). Huang TS, Nijholt A, Pantic M, Pentland A (eds) Springer, Berlin, pp 47–71
Gandhi T, Trivedi MM (2007) Pedestrian protection systems: issues, survey, and challenges. IEEE Trans Intell Transp Syst 8(3):413–430
Volker K (2007) The meaning of action: a review on action recognition and map**. Adv Rob 21:1473–1501
Poppe R (2007) Vision-based human motion analysis: an over-view. Comput Vis Image Understanding 108:4–18
Zhou Y, Yan S, Huang TS (2007) Detecting anomaly in videos from trajectory similarity analysis. In: Proceedings of the IEEE international conference on multimedia expo, pp 1087–1090
Chen-Chiung H, Shu-Shuo H (2007) A simple and fast surveillance system human tracking and behavior analysis. In: 3rd IEEE international conference on signal-image technologies and internet based system, pp 812–818. https://doi.org/10.1109/sitis.2007.128
Wang Z, Zhang J (2008) Detecting pedestrian abnormal behavior based on fuzzy associative memory. In: 2008, IEEE 4th International conference on natural computation, vol 6, pp 143–147
Zhang J, Liu Z (2008) Detecting abnormal motion of pedestrian in video. In: Information and automation, 2008. ICIA 2008. IEEE, International conference on, pp 81–85
Zhou G, Wu Y (2009) Anomalous event detection based on self-organizing map for supermarket monitoring. In: 2009 International conference on information engineering and computer science, IEEE, pp 1–4
Yin G, Bruckner D (2009) Gaussian models and fast learning algorithm for persistence analysis of tracked video objects. In 2009 2nd conference on human system interactions, IEEE, pp 60–63
Li CL, Hao ZB, Li JJ (2010) Abnormal behavior detection using a novel behavior representation. In Apperceiving computing and intelligence analysis (ICACIA), 2010 International Conference on, IEEE, pp 331–336
Ding N, Chen Y, Zhong Z, Xu Y (2010) Energy-based surveillance systems for ATM machines. In: Intelligent control and automation (WCICA), 2010 8th World Congress on, IEEE, pp 2880–2887
Zin TT, Tin P, Toriu T, Hama H (2010) A Markov random walk model for loitering people detection. In: Intelligent information hiding and multimedia signal processing (IIH-MSP), 2010 6th international conference on, IEEE, pp 680–683
Bouttefroy PLM, Beghdadi A, Bouzerdoum A, Phung SL (2010) Markov random fields for abnormal behavior detection on highways. In: Visual information processing (EUVIP), 2010 2nd Europea Workshop on, IEEE, pp 149–154
Husni M, Suryana N (2010) Crowd event detection in computer vision. In: Proceedings of 2nd international conference on signal processing system, Jul. 5–7, 2010, pp V1444–V1447
Cui L, Li K, Chen J, Li Z (2011) Abnormal event detection in traffic video surveillance based on local features. In: Image and signal processing (CISP), 2011 4th International Congress on, IEEE, Vol 1, pp 362–366
Lizhong L, Zhiguo L, Yubin Z (2012) Research on detection and tracking of moving target in intelligent video surveillance. In: Computer science and electronics engineering (ICCSEE), 2012 International Conference on, IEEE, Vol 3, pp 477–481
Wang C, Wu X, Li N, Chen YL (2012) Abnormal detection based on gait analysis. In: Intelligent control and automation (WCICA), 2012 10th World Congress on, IEEE, pp 4859–4864
Lee JJ, Kim GJ, Kim MH (2012) Trajectory extraction for abnormal behavior detection in public area. In: Emerging technologies for a smarter world (CEWIT), 2012 9th international conference and expo on, IEEE, pp 1–5
Revathi AR, Kumar D (2012) A survey of activity recognition and understanding the behavior. In: Video surveillance, ar**v:1207.6774
Gowsikhaa D, Abirami S, Baskaran R (2014) Automated human behavior analysis from surveillance videos: a survey. Artif Intell Rev Spring 42(4):747–765
Senst T, Eiselein V, Badii A, Einig M, Keller I, Sikora T (2013) A decentralized privacy-sensitive video surveillance framework. In: Digital signal processing (DSP), 2013 18th international conference on, IEEE, pp 1–6
Alqaysi HH, Sasi S (2013) Detection of abnormal behavior in dynamic crowded gatherings. In: 2013 IEEE applied imagery pattern recognition workshop (AIPR), IEEE, pp 1–6
Walha A, Wali A, Alimi AM (2013) A system of abnormal behavior detection in aerial surveillance. In: Information assurance and security (IAS), 2013 9th International Conference on, IEEE, pp. 102–107
Cristani M, Raghavendra R, Del Bue A, Murino V (2013) Human behavior analysis in video surveillance: a social signal processing perspective. Neurocomputing 100:86–97
Zhang Y, Dong L, Li S, Li J (2014) Abnormal crowd behavior detection using interest points. In: 2014 IEEE, international symposium on broadband multimedia systems and broadcasting, IEEE, pp 1–4
Wang J, Xu Z (2014) Texture-based homogeneity analysis for crowd scene modelling and abnormality detection. In: Automation and computing (ICAC), 2014 20th international conference on, IEEE, pp 182–187
Chen C, Shao Y (2014) Anomalous crowd behavior detection and localization in video surveillance. In: Control Science and Systems Engineering (CCSSE), 2014 IEEE International Conference on, IEEE, pp 190–194
Delgado B, Tahboub K, Delp EJ (2014) Automatic detection of abnormal human events on train platforms. In: NAECON 2014-IEEE national aerospace and electronics conference, IEEE, pp 169–173
Divya J. Automatic video based surveillance system for abnormal behavior detection
Chen C, Shao Yu, Bi X (2015) Detection of anomalous crowd behavior based on acceleration feature. IEEE Sens J 15(12):7252–7261
Momin BF, Fanase VM (2015) Detection and localization of abnormal activities in video surveillance system. In: Communications and signal processing (ICCSP), 2015 International Conference on IEEE, 2015 Apr 2, pp 0277–0280
Yavari M, Li M, Li S, Qi M (2015) Parallelizing abnormal event detection in crowded scenes with GPU. In: Fuzzy systems and knowledge discovery (FSKD), 12th international conference on, IEEE, pp 274–277
Tiwari C, Hanmandlu M, Vasikarla S (2015) Suspicious face detection based on eye and other facial features movement monitoring. In: 2015 IEEE applied imagery pattern recognition workshop (AIPR), IEEE, pp 1–8
Ahmed M, Mahmood AN, Islam MR (2016) A survey of anomaly detection techniques in financial domain. Fut Gen Comput Syst 55:278–288
Zhang Y, Lu H, Zhang L, Ruan X (2016) Combining motion and appearance cues for anomaly detection. Pattern Recognit 51:443–452
Farzad A, Asli RN (2015) Recognition and classification of human behavior in Intelligent surveillance systems using Hidden Markov model. Int J Image Graph Signal Process 7(12):31
Dawn DD, Shaikh SH (2016) A comprehensive survey of human action recognition with spatio-temporal interest point (STIP) detector. Vis Comput 32(3):289–306
Al-Nawashi M, Obaida MA-H, Mohamad S (2016) A novel framework for intelligent surveillance system based on abnormal human activity detection in academic environments. In: Neural Computing and Applications, Springer, pp 1–8
Kapoor S, Pateriya A (2016) A study on various methods used in automatic video surveillance. In: 6th IEEE international conference on cloud system and big data engineering (confluence), pp 413–417
Cosar S, Donatiello G, Bogorny V, Garate C, Alvares LO, Bremond F (2016) Towards abnormal trajectory and event detection in video surveillance. IEEE Trans Circ Syst Video Process 1(99):1–14. https://doi.org/10.1109/tcsvt.2016.2589859
Subetha T, Chitrakala S (2016) A survey on human activity recognition from videos. In: Information communication and embedded systems (ICICES), 2016 International Conference on, IEEE, pp 1–7
Ma C, Li W, Gravina R, Fortino G (2016) Activity recognition and monitoring for smart wheelchair users. In: Computer supported cooperative work in design (CSCWD), 2016 IEEE 20th international conference on, IEEE, pp 664–669
Nar R, Singal A, Kumar P (2016) Abnormal activity detection for bank ATM surveillance. In: Advances in computing, communications and informatics (ICACCI), 2016 international conference on, IEEE, pp 2042–2046
Khadse MV, Nijampurkar PP, Pardeshi YD, Kale NS (2016) An effective object detection video surveillance and alert system. In: International journal of computer applications pp 0975–8887
Singh D, Krishna Mohan C (2017) Graph formulation of video activities for abnormal activity recognition. Pattern Recognit 65(2017):265–272
Wu X, Ou Y, Qian H, Xu Y (2005) A detection system for human abnormal behavior. In: 2005 IEEE/RSJ international conference on intelligent robots and systems, pp 1204–1208
Sewaiwar P, Kamal KV (2015) Comparative study of various decision tree classification algorithm using WEKA. In: International Journal of Emerging Research in Management &Technology 4 (2015): 2278-9359
Christodoulou L, Kasparis T, Marques O (2011) Advanced statistical and adaptive threshold techniques for moving object detection and segmentation. In: 2011 17th International conference on digital signal processing (DSP), pp 1–6
Karpathy A, Toderici G, Shetty S, Leung T, Sukthankar R, Fei-Fei L (2014) Large-scale video classification with convolutional neural networks. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition, pp 1725–1732
Simonyan K, Zisserman A (2014) Two-stream convolutional networks for action recognition in videos. In: Advances in Neural information processing systems, pp 568–576
Mahasseni B, Todorovic S (2016) Regularizing long short term memory with 3D human-skeleton sequences for action recognition. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition, pp 3054–3062
**n M, Zhang H, Wang H, Sun M, Yuan D (2016) ARCH: adaptive recurrent-convolutional hybrid networks for long-term action recognition. Neurocomputing 178:87–102
Murthy OVR, Goecke R (2015) Ordered trajectories for human action recognition with large number of classes. Image Vis Comput 42:22–34
Wang P, Cao Y, Shen C, Liu L, Shen HT (2015) Temporal pyramid pooling based convolutional neural networks for action recognition
Ullah A, Ahmad J, Muhammad K, Sajjad M, Baik SW (2018) Action recognition in video sequences using deep bi-directional LSTM with CNN features. IEEE Access 6:1155–1166
Foroughi H, Aski BS, Pourreza H (2008) Intelligent video surveillance for monitoring fall detection of elderly in home environments. 2008 11th international conference on computer and information technology. IEEE
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Verma, K.K., Singh, B.M. & Dixit, A. A review of supervised and unsupervised machine learning techniques for suspicious behavior recognition in intelligent surveillance system. Int. j. inf. tecnol. 14, 397–410 (2022). https://doi.org/10.1007/s41870-019-00364-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s41870-019-00364-0