SpringerLink
Log in

Video event detection, classification and retrieval using ensemble feature selection

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

In recent times, video analysis gains more popularity among the researchers, due to its social impact and widespread applications. In this research article, a novel algorithm is proposed to detect the multiple events in the videos. At first, histogram equalization technique is used for contrast enhancement and smoothing of the videos, which are collected from Columbia consumer video, and UCF101 databases. The histogram equalization delivers a better quality of frames without loss of any local information like edges, patches and points. Further, feature extraction is accomplished using gradient local ternary pattern, histogram of oriented gradients, and Tamura features to extract the feature vectors from the enhanced frames that speed up the training process. The extracted feature vectors are high dimension in nature, so an ensemble feature selection algorithm is proposed to select optimal features from the extracted features. The selected optimal feature sub-sets are fed to multi support vector machine classifier to classify the multiple events. Finally, the relevant events are retrieved utilizing Euclidean distance measure and the simulation result showed that the proposed algorithm achieved better performance in multi-event recognition. The proposed algorithm almost showed a maximum of 14.21 % improvement in video event classification by means of accuracy compared to the existing algorithms like two-stage neural network, multi-stream deep learning system, improved two-stream model, Concept wise Power Law Normalization (CPN) with Convolutional Neural Network (CNN), Motion based Shot Boundary Detection (MSBD) algorithm, and motion features with Recurrent Neural Network (RNN).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data Availability

The datasets generated during and/or analysed during the current study are available in the [CCV and UCF101 databases] repository, https://www.ee.columbia.edu/ln/dvmm/CCV/, https://www.crcv.ucf.edu/data/UCF101.php.

References

  1. Jhuo, I.H., Ye, G., Gao, S., Liu, D., Jiang, Y.G., Lee, D.T., Chang, S.F.: Discovering joint audio-visual codewords for video event detection. Mach. Vis. Appl. 25, 33–47 (2014). doi:https://doi.org/10.1007/s00138-013-0567-0

    Article  Google Scholar 

  2. Yu, J., Lei, A., Hu, Y.: Soccer video event detection based on deep learning. In International Conference on Multimedia Modeling 377–389: (2019)

  3. Lim, M.K., Tang, S., Chan, C.S.: iSurveillance: Intelligent framework for multiple events detection in surveillance videos. Expert. Syst. Appl. 41, 4704–4715 (2014). doi:https://doi.org/10.1016/j.eswa.2014.02.003

    Article  Google Scholar 

  4. Xu, J., Denman, S., Reddy, V., Fookes, C., Sridharan, S.: Real-time video event detection in crowded scenes using MPEG derived features: A multiple instance learning approach. Pattern Recogn. Lett. 44, 113–125 (2014). doi:https://doi.org/10.1016/j.patrec.2013.11.019

    Article  Google Scholar 

  5. Liu, A.A., Shao, Z., Wong, Y., Li, J., Su, Y.T., Kankanhalli, M.: LSTM-based multi-label video event detection. Multimedia Tools and Applications. 78, 677–695 (2019). doi:https://doi.org/10.1007/s11042-017-5532-x

    Article  Google Scholar 

  6. Geng, Y., Du, J., Liang, M.: Abnormal event detection in tourism video based on salient spatio-temporal features and sparse combination learning. World Wide Web 22, 689–715 (2019)

    Article  Google Scholar 

  7. Balasundaram, A., Chellappan, C.: An intelligent video analytics model for abnormal event detection in online surveillance video. Journal of Real-Time Image Processing (2018). https://doi.org/10.1007/s11554-018-0840-6

    Article  Google Scholar 

  8. Lee, S.C., Nevatia, R.: Hierarchical abnormal event detection by real time and semi-real time multi-tasking video surveillance system. Mach. Vis. Appl. 25, 133–143 (2014). doi:https://doi.org/10.1007/s00138-013-0516-y

    Article  Google Scholar 

  9. Ullah, H., Islam, I.U., Ullah, M., Afaq, M., Khan, S.D., Iqbal, J.: Multi-feature-based crowd video modeling for visual event detection. Multimedia Systems (2020). https://doi.org/10.1007/s00530-020-00652-x

    Article  Google Scholar 

  10. Jiang, F., Yuan, J., Tsaftaris, S.A., Katsaggelos, A.K.: Anomalous video event detection using spatiotemporal context. Comput. Vision Image Understanding. 115, 323–333 (2011). doi:https://doi.org/10.1016/j.cviu.2010.10.008

    Article  Google Scholar 

  11. Zhou, F., Wang, L., Li, Z., Zuo, W., Tan, H.: Unsupervised Learning Approach for Abnormal Event Detection in Surveillance Video by Hybrid Autoencoder. Neural Processing Letters (2019). https://doi.org/10.1007/s11063-019-10113-w

    Article  Google Scholar 

  12. Mazloom, M., Li, X., Snoek, C.G.: Tagbook: A semantic video representation without supervision for event detection. IEEE Trans. Multimedia. 18, 1378–1388 (2016). doi:https://doi.org/10.1109/tmm.2016.2559947

    Article  Google Scholar 

  13. Li, P., Xu, X.: Recurrent Compressed Convolutional Networks for Short Video Event Detection. IEEE Access. 8, 114162–114171 (2020). doi:https://doi.org/10.1109/ACCESS.2020.3003939

    Article  Google Scholar 

  14. Wang, F., Sun, Z., Jiang, Y.G., Ngo, C.W.: Video event detection using motion relativity and feature selection. IEEE Trans. Multimedia. 16, 1303–1315 (2014). doi:https://doi.org/10.1109/tmm.2014.2315780

    Article  Google Scholar 

  15. Lee, S., Kim, H.G., Ro, Y.M.: BMAN: Bidirectional Multi-Scale Aggregation Networks for Abnormal Event Detection. IEEE Trans. Image Process. 29, 2395–2408 (2019). doi:https://doi.org/10.1109/TIP.2019.2948286

    Article  Google Scholar 

  16. Lu, Y., An, S.: Research on sports video detection technology motion 3D reconstruction based on hidden Markov model. Cluster Computing 23(3), 1899–1909 (2020)

    Article  Google Scholar 

  17. Yan, R.: Researches on hybrid algorithm for moving target detection and tracking in sports video. Cluster Computing 22(2), 3543–3552 (2019)

    Article  Google Scholar 

  18. Iqbal, B., Iqbal, W., Khan, N., Mahmood, A., Erradi, A.: Canny edge detection and Hough transform for high resolution video streams using Hadoop and Spark. Cluster Computing 23(1), 397–408 (2020)

    Article  Google Scholar 

  19. Wan, S., Xu, X., Wang, T., Gu, Z.: An intelligent video analysis method for abnormal event detection in intelligent transportation systems. IEEE Transactions on Intelligent Transportation Systems (2020)

  20. Lohithashva, B.H., Aradhya, V.M., Guru, D.S.: Violent video event detection based on integrated LBP and GLCM texture features. Revue d’Intelligence Artificielle 34(2), 179–187 (2020)

    Article  Google Scholar 

  21. Georgescu, M.I., Ionescu, R., Khan, F.S., Popescu, M., Shah, M.: 2021. A Background-Agnostic Framework with Adversarial Training for Abnormal Event Detection in Video. IEEE Transactions on Pattern Analysis & Machine Intelligence, (01), pp.1–1

  22. Liu, W., Kang, G., Huang, P.Y., Chang, X., Qian, Y., Liang, J., Gui, L., Wen, J., Chen, P.: 2020. Argus: Efficient activity detection system for extended video analysis. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (pp. 126–133)

  23. Wang, T., Qiao, M., Zhu, A., Shan, G., Snoussi, H.: Abnormal event detection via the analysis of multi-frame optical flow information. Frontiers of Computer Science 14(2), 304–313 (2020)

    Article  Google Scholar 

  24. Jiang, Y.G., Ye, G., Chang, S.F., Ellis, D., Loui, A.C.: Consumer video understanding: A benchmark database and an evaluation of human and machine performance. In Proceedings of the 1st ACM International Conference on Multimedia Retrieval 1–8: (2011)

  25. Soomro, K., Zamir, A.R., Shah, M.: UCF101: A dataset of 101 human actions classes from videos in the wild. (2012) ar**v preprint ar**v:1212.0402

  26. Abdullah-Al-Wadud, M., Kabir, M.H., Dewan, M.A.A., Chae, O.: A dynamic histogram equalization for image contrast enhancement. IEEE Trans. Consum. Electron. 53, 593–600 (2007). doi:https://doi.org/10.1109/tce.2007.381734

    Article  Google Scholar 

  27. Holder, R.P., Tapamo, J.R.: Improved gradient local ternary patterns for facial expression recognition. EURASIP Journal on Image and Video Processing: 2017, 42 (2017) doi:https://doi.org/10.1186/s13640-017-0190-5

  28. Nazir, M., Jan, Z., Sajjad, M.: Facial expression recognition using histogram of oriented gradients based transformed features. Cluster Computing 21, 539–554 (2018)

    Article  Google Scholar 

  29. Humeau-Heurtier, A.: Texture feature extraction methods: A survey. IEEE Access. 7, 8975–9000 (2019). doi:https://doi.org/10.1109/access.2018.2890743

    Article  Google Scholar 

  30. Choi, S., Jiang, Z.: Cardiac sound murmurs classification with autoregressive spectral analysis and multi-support vector machine technique. Comput. Biol. Med. 40, 8–20 (2010). doi:https://doi.org/10.1016/j.compbiomed.2009.10.003

    Article  Google Scholar 

  31. Wang, X., Ji, Q.: Hierarchical context modeling for video event recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39, 1770–1782 (2016). doi:https://doi.org/10.1109/tpami.2016.2616308

    Article  Google Scholar 

  32. **ao, Z., Jiang, J., Ming, Z.: High-Level Video Event Modeling, Recognition, and Reasoning via Petri Net. IEEE Access. 7, 129376–129386 (2019). doi:https://doi.org/10.1109/ACCESS.2019.2936493

    Article  Google Scholar 

  33. Caruccio, L., Polese, G., Tortora, G., Iannone, D.: EDCAR: A knowledge representation framework to enhance automatic video surveillance. Expert. Syst. Appl. 131, 190–207 (2019). doi:https://doi.org/10.1016/j.eswa.2019.04.031

    Article  Google Scholar 

  34. Wang, P., Liu, L., Shen, C., Shen, H.T.: Order-aware convolutional pooling for video based action recognition. Pattern Recognit. 91, 357–365 (2019). doi:https://doi.org/10.1016/j.patcog.2019.03.002

    Article  Google Scholar 

  35. Fan, Y., Wen, G., Li, D., Qiu, S., Levine, M.D.: Early event detection based on dynamic images of surveillance videos. J. Visual Commun. Image Represent. 51, 70–75 (2018). doi:https://doi.org/10.1016/j.jvcir.2018.01.002

    Article  Google Scholar 

  36. Soltanian, M., Ghaemmaghami, S.: Hierarchical Concept Score Postprocessing and Concept-Wise Normalization in CNN-Based Video Event Recognition. IEEE Trans. Multimedia. 21, 157–172 (2018). doi:https://doi.org/10.1109/tmm.2018.2844101

    Article  Google Scholar 

  37. Zhang, L., **ang, X.: Video event classification based on two-stage neural network. Multimedia Tools and Applications (2020). https://doi.org/10.1007/s11042-019-08457-5

    Article  Google Scholar 

  38. Li, Q., Qiu, Z., Yao, T., Mei, T., Rui, Y., Luo, J.: Learning hierarchical video representation for action recognition. International Journal of Multimedia Information Retrieval. 6, 85–98 (2017). doi:https://doi.org/10.1007/s13735-016-0117-4

    Article  Google Scholar 

  39. Kanagaraj, K., Priya, G.L.: Curvelet transform based feature extraction and selection for multimedia event classification. Journal of King Saud University-Computer and Information Sciences (2018) doi:https://doi.org/10.1016/j.jksuci.2018.11.006

    Article  Google Scholar 

  40. Zhao, Z., **ang, R., Su, F.: Complex event detection via attention-based video representation and classification. Multimedia Tools and Applications. 77, 3209–3227 (2018). doi:https://doi.org/10.1007/s11042-017-5058-2

    Article  Google Scholar 

  41. Zhao, Y., Man, K.L., Smith, J., Siddique, K., Guan, S.U.: Improved two-stream model for human action recognition. EURASIP Journal on Image and Video Processing 2020(1), 1–9 (2020)

    Article  Google Scholar 

  42. Jaouedi, N., Boujnah, N., Bouhlel, M.S.: A new hybrid deep learning model for human action recognition. Journal of King Saud University-Computer and Information Sciences 32(4), 447–453 (2020)

    Article  Google Scholar 

  43. Abualigah, L.M., Khader, A.T., Hanandeh, E.S.: A new feature selection method to improve the document clustering using particle swarm optimization algorithm. Journal of Computational Science 25, 456–466 (2018)

    Article  Google Scholar 

  44. Abualigah, L.M., Khader, A.T.: Unsupervised text feature selection technique based on hybrid particle swarm optimization algorithm with genetic operators for the text clustering. The Journal of Supercomputing 73(11), 4773–4795 (2017)

    Article  Google Scholar 

  45. Liu, Y., Mu, Y., Chen, K., Li, Y., Guo, J.: Daily activity feature selection in smart homes based on pearson correlation coefficient. Neural Process. Lett. (2020). https://doi.org/10.1007/s11063-019-10185-8

    Article  Google Scholar 

  46. Song, Q., Jiang, H., Liu, J.: Feature selection based on FDA and F-score for multi-class classification. Expert. Syst. Appl. 81, 22–27 (2017). doi:https://doi.org/10.1016/j.eswa.2017.02.049

    Article  Google Scholar 

  47. Yin, S., Jiang, Z.: A Variance–Mean Based Feature Selection in Text Classification. In 2009 First International Workshop on Education Technology and Computer Science 3, 519–522: (2009) doi:https://doi.org/10.1109/etcs.2009.646

  48. Doshi, M.: Correlation based feature selection (CFS) technique to predict student Perfromance. International Journal of Computer Networks & Communications 6, 197 (2014)

    Article  Google Scholar 

  49. Malkauthekar, M.D.: Analysis of Euclidean distance and Manhattan distance measure in Face recognition. Third International Conference on Computational Intelligence and Information Technology (2013). https://doi.org/10.1007/978-3-642-25734-6

    Article  Google Scholar 

  50. Abualigah, L.M., Khader, A.T., Hanandeh, E.S.: A combination of objective functions and hybrid krill herd algorithm for text document clustering analysis, 73, pp. 111–125. Engineering Applications of Artificial Intelligence (2018)

  51. Abualigah, L.M., Khader, A.T., Hanandeh, E.S., Gandomi, A.H.: A novel hybridization strategy for krill herd algorithm applied to clustering techniques. Appl. Soft Comput. 60, 423–435 (2017)

    Article  Google Scholar 

Download references

Funding

This study was not funded by any organization.

Author information

Authors and Affiliations

  1. Department of ECE, CMR Institute of Technology, Bangalore, India

    Susmitha Alamuru

  2. CMR Institute of Technology, Bangalore, India

    Sanjay Jain

Authors
  1. Susmitha Alamuru
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sanjay Jain
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Susmitha Alamuru.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alamuru, S., Jain, S. Video event detection, classification and retrieval using ensemble feature selection. Cluster Comput 24, 2995–3010 (2021). https://doi.org/10.1007/s10586-021-03308-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-021-03308-1

Keywords

Search

Navigation