Abstract
Keyframes extraction, a fundamental problem in video processing and analysis, has remained a challenge to date. In this paper, we introduce a novel method to effectively extract keyframes of a video. It consists of four steps. At first, we generate initial clips for the classified frames, based on consistent content within a clip. Using empirical evidence, we design an adaptive window length for the frame difference processing which outputs the initial keyframes then. We further remove the frames with meaningless information (e.g., black screen) in initial clips and initial keyframes. To achieve satisfactory keyframes, we finally map the current keyframes to the space of current clips and optimize the keyframes based on similarity. Extensive experiments show that our method outperforms to state-of-the-art keyframe extraction techniques with an average of \(96.84\%\) on precision and \(81.55\%\) on \(F_1\).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Open source Python library built to empower developers to build applications and systems with self-contained computer vision capabilities. https://github.com/OlafenwaMoses/frameAI
Asghar, M.N., Hussain, F., Manton, R.: Video indexing: a survey. Int. J. Comput. Inf. Technol. 3(01), 1–22 (2014)
Guan, G., Wang, Z., Lu, S., Deng, J.D., Feng, D.D.: Keypoint-based keyframe selection. IEEE Trans. Circuits Syst. Video Technol. 23(4), 729–734 (2013). https://doi.org/10.1109/TCSVT.2012.2214871
Jiang, L., Shen, G., Zhang, G.: An image retrieval algorithm based on HSV color segment histograms. Mech. Electr. Eng. Mag. 26(11), 54–57 (2009)
Kuanar, S.K., Panda, R., Chowdhury, A.S.: Video key frame extraction through dynamic Delaunay clustering with a structural constraint. J. Vis. Commun. Image Represent. 24(7), 1212–1227 (2013)
Kulhare, S., Sah, S., Pillai, S., Ptucha, R.: Key frame extraction for salient activity recognition. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp. 835–840. IEEE (2016)
Li, X., Zhao, B., Lu, X.: Key frame extraction in the summary space. IEEE Trans. Cybern. 48(6), 1923–1934 (2017)
Liu, H., Li, T.: Key frame extraction based on improved frame blocks features and second extraction. In: 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), pp. 1950–1955. IEEE (2015)
Liu, H., Meng, W., Liu, Z.: Key frame extraction of online video based on optimized frame difference. In: 2012 9th International Conference on Fuzzy Systems and Knowledge Discovery, pp. 1238–1242. IEEE (2012)
Luo, Y., Zhou, H., Tan, Q., Chen, X., Yun, M.: Key frame extraction of surveillance video based on moving object detection and image similarity. Pattern Recogn. Image Anal. 28(2), 225–231 (2018). https://doi.org/10.1134/S1054661818020190
Mehmood, I., Sajjad, M., Rho, S., Baik, S.W.: Divide-and-conquer based summarization framework for extracting affective video content. Neurocomputing 174, 393–403 (2016)
Asha Paul, M.K., Kavitha, J., Jansi Rani, P.A.: Key-frame extraction techniques: a review. Recent Pat. Comput. Sci. 11(1), 3–16 (2018). https://doi.org/10.2174/2213275911666180719111118
Singla, N.: Motion detection based on frame difference method. Int. J. Inf. Comput. Technol. 4(15), 1559–1565 (2014)
Swain, M.J., Ballard, D.H.: Indexing via color histograms. In: Sood, A.K., Wechsler, H. (eds.) Active Perception and Robot Vision. NATO ASI Series, vol. 83, pp. 261–273. Springer, Heidelberg (1992). https://doi.org/10.1007/978-3-642-77225-2_13
Tang, H., Zhou, J.: Method for extracting the key frame of various types video based on machine learning. Ind. Control Comput. 3, 94–95 (2014)
Wang, S., Han, Y., Yadong, W.U., Zhang, S.: Video key frame extraction method based on image dominant color. J. Comput. Appl. 33(9), 2631–2635 (2013)
**a, G., Sun, H., Niu, X., Zhang, G., Feng, L.: Keyframe extraction for human motion capture data based on joint Kernel sparse representation. IEEE Trans. Ind. Electron. 64(2), 1589–1599 (2016)
Yang, S., Lin, X.: Key frame extraction using unsupervised clustering based on a statistical model. Tsinghua Sci. Technol. 10(2), 169–173 (2005)
Yong, S.P., Deng, J.D., Purvis, M.K.: Wildlife video key-frame extraction based on novelty detection in semantic context. Multimed. Tools Appl. 62(2), 359–376 (2013). https://doi.org/10.1007/s11042-011-0902-2
Acknowledgement
This work was partially funded by Key Laboratory of Agricultural Internet of Things, Ministry of Agriculture and Rural Affairs, Yangling, Shaanxi 712100, China \((2018AIOT-09)\). National Natural Science Foundation of China (61702433), Key Research and Development Program of Shaanxi Province \((2018NY-127)\).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Pei, Y., Huang, Z., Yu, W., Wang, M., Lu, X. (2020). A Cascaded Approach for Keyframes Extraction from Videos. In: Tian, F., et al. Computer Animation and Social Agents. CASA 2020. Communications in Computer and Information Science, vol 1300. Springer, Cham. https://doi.org/10.1007/978-3-030-63426-1_8
Download citation
DOI: https://doi.org/10.1007/978-3-030-63426-1_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63425-4
Online ISBN: 978-3-030-63426-1
eBook Packages: Computer ScienceComputer Science (R0)