SpringerLink
Log in

Facial Micro-expression Modelling-Based Student Learning Rate Evaluation Using VGG–CNN Transfer Learning Model

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

Micro-facial expressions hold the potential to identify emotional states of students during their participation in online learning tasks. Through understanding the semi-nute facial cues, e-learning platforms can discern and decode a range of emotions experienced by students, including happiness, sadness, frustration, confusion, and more. This information can be used to personalize the learning experience for each student, provide early intervention and support, and improve engagement and motivation. This paper presents a VGG16 convolution neural network (CNN)-based transfer learning model for facial micro-expression recognition. The proposed model identifies the relation between micro-expressions and emotions in the student faces in online classes to assess the learning rate. Emotion modelling is used in the context of online education, and it entails making use of technology and data analysis to identify the emotional indicators that students are exhibiting and then responding appropriately. The pretrained VGG model allows for the creation of much deeper features than previous architectures while mitigating the vanishing gradient problem with the help of the transfer learning approach. The proposed model can learn more complex representations of the input data, which is important in recognizing subtle emotions, thereby accurately assessing the learning rate of the students. The proposed model obtained an accuracy of 97.3%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data Availability

The dataset generated and analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Liu M, Yu D. Towards intelligent E-learning systems. Educ Inform Technolo. 2022;28(7):7845–76.

    Google Scholar 

  2. Sobaih AEE, Palla IA, Baquee A. Social media use in e-learning amid COVID 19 pandemic: Indian students’ perspective. Int J Environ Res Public Health. 2022;19(9):5380.

    Article  Google Scholar 

  3. Banerjee P. Corporate E-learning initiatives: a new paradigm for employee development. NHRD Netw J. 2022;15(3):345–52.

    Article  Google Scholar 

  4. Du Y, Crespo RG, Martínez OS. Human emotion recognition for enhanced performance evaluation in e-learning. Progress Artif Intell. 2022;12(2):199–211.

    Article  Google Scholar 

  5. Mejbri N, Essalmi F, Jemni M, Alyoubi BA. Trends in the use of affective computing in e-learning environments. Educ Inform Technol. 2022;27:3867–89.

    Article  Google Scholar 

  6. Garcia-Garcia JM, Penichet VMR, Lozano MD, Fernando A. Using emotion recognition technologies to teach children with autism spectrum disorder how to identify and express emotions. Univers Access Inf Soc. 2022;21(4):809–25.

    Article  Google Scholar 

  7. Kumar A, Kumar A. Analysis of machine learning algorithms for facial expression recognition. In: advanced network technologies and intelligent computing: first International Conference, ANTIC 2021, Varanasi, India, December 17–18, 2021, Proceedings, pp. 730–750. Cham: Springer International Publishing; 2022.

  8. Mazhar T, Malik MA, Nadeem MA, Mohsan SAH, Haq I, Karim FK, Mostafa SM. Movie reviews classification through facial image recognition and emotion detection using machine learning methods. Symmetry. 2022;14(12):2607.

    Article  Google Scholar 

  9. Hassouneh A, Mutawa AM, Murugappan M. Development of a real-time emotion recognition system using facial expressions and EEG based on machine learning and deep neural network methods. Inform Med Unlocke. 2020;20:100372.

    Article  Google Scholar 

  10. Tonguç G, Ozkara BO. Automatic recognition of student emotions from facial expressions during a lecture. Comput Educ. 2020;148:103797.

    Article  Google Scholar 

  11. Schoneveld L, Othmani A, Abdelkawy H. Leveraging recent advances in deep learning for audio-visual emotion recognition. Pattern Recognit Lett. 2021;146:1–7.

    Article  Google Scholar 

  12. Zatarain Cabada R, Rangel HR, Estrada MLB, Lopez HMC. Hyperparameter optimization in CNN for learning-centered emotion recognition for intelligent tutoring systems. Soft Comput. 2020;24(10):7593–602.

    Article  Google Scholar 

  13. Mehendale N. Facial emotion recognition using convolutional neural networks (FERC). SN Applied Sci. 2020;2(3):446.

    Article  Google Scholar 

  14. Nezami OM, Dras M, Hamey L, Richards D, Wan S, Paris C. Automatic recognition of student engagement using deep learning and facial expression. In: Joint European Conference on machine learning and knowledge discovery in databases, 2020; pp. 273–289. Springer, Cham.

  15. Ashwin TS, Guddeti RMR. Automatic detection of students’ affective states in classroom environment using hybrid convolutional neural networks. Educ Inf Technol. 2020;25(2):1387–415.

    Article  Google Scholar 

  16. Ninaus M, Greipl S, Kiili K, Lindstedt A, Huber S, Klein E, Karnath H-O, Moeller K. Increased emotional engagement in game-based learning–A machine learning approach on facial emotion detection data. Comput Educ. 2019;142: 103641.

    Article  Google Scholar 

  17. Meléndez JC, Satorres E, Reyes-Olmedo M, Delhom I, Real E, Lora Y. Emotion recognition changes in a confinement situation due to COVID-19. J Environ Psychol. 2020;72:101518.

    Article  Google Scholar 

  18. Estrada MLB, Cabada RZ, Bustillos RO, Graff M. Opinion mining and emotion recognition applied to learning environments. Expert Syst Appl. 2020;150:113265.

    Article  Google Scholar 

  19. Ouherrou N, Elhammoumi O, Benmarrakchi F, El Kafi J. Comparative study on emotions analysis from facial expressions in children with and without learning disabilities in virtual learning environment. Educ Inf Technol. 2019;24(2):1777–92.

    Article  Google Scholar 

  20. Imani M, Montazer GA. A survey of emotion recognition methods with emphasis on E-Learning environments. J Netw Comput Appl. 2019;147:102423.

    Article  Google Scholar 

  21. Jarraya SK, Masmoudi M, Hammami M. A comparative study of Autistic Children Emotion recognition based on Spatio-Temporal and Deep analysis of facial expressions features during a Meltdown Crisis. Multimed Tools Appl. 2021;80:83–125.

    Article  Google Scholar 

  22. Cheng S, Zhou G. Facial expression recognition method based on improved VGG convolutional neural network. Int J Pattern Recognit Artif Intell. 2020;34(07):2056003.

    Article  Google Scholar 

  23. Hu L, Ge Q. Automatic facial expression recognition based on MobileNetV2 in Real-time. J Phys Conf Ser. 2020;1549(2):022136.

    Article  Google Scholar 

  24. Mane S, Shah G. Facial recognition, expression recognition, and gender identification. In: Data management, analytics and innovation: Proceedings of ICDMAI 2018; Volume 1, pp. 275–290. Springer Singapore, 2019.

Download references

Acknowledgements

The authors acknowledged the Lovely Professional University, Phagwara, Punjab, India and Neil Gogte Institute of Technology, Hyderabad, India for supporting the research work by providing the facilities.

Funding

No funding received for this research.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Lovely Professional University, Phagwara, Punjab, India

    Maragoni Mahendar, Arun Malik & Isha Batra

  2. Neil Gogte Institute of Technology, Hyderabad, Telangana, India

    Maragoni Mahendar, Arun Malik & Isha Batra

Authors
  1. Maragoni Mahendar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arun Malik
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Isha Batra
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

This research article was made possible by the collaboration and contributions of all authors.

Corresponding author

Correspondence to Arun Malik.

Ethics declarations

Conflict of Interest

No conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the topical collection “Advances in Computational Approaches for Image Processing, Wireless Networks, Cloud Applications and Network Security” guest edited by P. Raviraj, Maode Ma and Roopashree H R.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mahendar, M., Malik, A. & Batra, I. Facial Micro-expression Modelling-Based Student Learning Rate Evaluation Using VGG–CNN Transfer Learning Model. SN COMPUT. SCI. 5, 204 (2024). https://doi.org/10.1007/s42979-023-02519-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-023-02519-0

Keywords

Search

Navigation