SpringerLink
Log in

Implementation of an adaptive E-learning platform with facial emotion recognition

  • Technical Paper
  • Published:
Microsystem Technologies Aims and scope Submit manuscript

Abstract

The advancement in technology provides various opportunities for students to encourage academic development and provides ease of access to education using E-learning systems. The most challenging and demanding task during learning is to be aware of and support the emotional side of students. Humans can easily recognize the emotions of a person, but it’s been a significantly challenging task to do with a computer as the human face consists of some specific characteristics. With the recent advancements in the computer and image processing fields, it has become possible and easy to detect and classify emotions present in images. This paper focuses on a step-by-step process of design and implementation of an adaptive E-learning system. In general, the most common facial expressions classified from a human face are happy, sad, fearful, surprised, anger, disgust, and neutral. The fusion of different emotions is used to propose a method that will be combining a few emotions to represent a single learner entity. The present research is mainly focused on the identification of features, the fusion of basic features to categorize the learner's mood by taking the input as a video stream of participants. The experimentation being conducted on facial expression to classify a person’s category of listening based on facial expression. The categories of learners considered in this paper are active learner, passive learner, non-learner, and evaluative learner. This is implemented with CNN (Convolutional Neural Network) for facial emotion recognition. The results assured that; the accuracy of learner categorization is exhibiting 97.40%with the maximum number of 200 epochs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (Germany)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

All data generated in this study are available in this article and the online supplementary material.

References

  • Agrawal A, Mittal N (2019) Using CNN for facial expression recognition: a study of the effects of kernel size and number of filters on accuracy. Vis Comput. https://doi.org/10.1007/s00371-019-01630-9

    Article  Google Scholar 

  • Almeida J, Vilaça L, Teixeira IN, Viana P (2021) Emotion identification in movies through facial expression recognition. Appl Sci 11(15):6827

    Article  Google Scholar 

  • Ayvaz U, Gürüler H, Devrim MO (2017) Use of facial emotion recognition in e-learning systems. Inf Technol Learn Tools 60(4):95–104. https://doi.org/10.33407/itlt.v60i4.1743

    Article  Google Scholar 

  • Breuer R, Kimmel R (2017) A deep learning perspective on the origin of facial expressions. ar**vPreprint ar**v170501842

  • Deepika NM, Bala MM, Kumar R (2021) Design and implementation of intelligent virtual laboratory using RASA framework. Mater Today: Proc. https://doi.org/10.1016/j.matpr.2021.01.226

    Article  Google Scholar 

  • Dubey M, Singh L (2016) Automatic emotion recognition using facial expression: a review. Int Res J Eng Technol 3

  • Ekman P, Friesen WV (1971) Constants across cultures in the face and emotion. J Pers Soc Psychol 17(2):124

    Article  Google Scholar 

  • El Hammoumi O, Benmarrakchi F, Ouherrou N, El Kafi J, El Hore A (2018) Emotion recognition in e-learning systems,” In: 2018 6th international conference on multimedia computing and systems, pp 1–6. https://doi.org/10.1109/ICMCS.2018.8525872

  • Fan Y, Lu X, Li D, Liu Y (2016) Video-based emotion recognition using CNN-RNN and C3D hybrid networks. In: Proceedings of the 18th ACM international conference on multimodal interaction, pp 445–450

  • Feidakis M, Daradoumis T, Caballe S (2011) Endowing e-learning systems with emotion awareness. In: Third international conference on intelligent networking and collaborative systems, pp 68–75

  • Harley JM, Lajoie SP, Frasson C, Hall NC (2015) An integrated emotion-aware framework for intelligent tutoring systems. In: Artificial intelligence in education, pp 616–619

  • Jain C, Sawant K, Rehman M, Kumar R (2018) Emotion detection and characterization using facial features. In: 2018 3rd international conference and workshops on recent advances and innovations in engineering, pp 1–6. https://doi.org/10.1109/ICRAIE.2018.8710406

  • Jain DK, Zhang Z, Huang K (2020) Multi angle optimal pattern-based deep learning for automatic facial expression recognition. Pattern Recognit Lett 139:157–165

    Article  Google Scholar 

  • Kanade T, Cohn JF, Tian Y (2000) Comprehensive database for facial expression analysis. In: Proceedings fourth IEEE international conference on automatic face and gesture recognition, pp 46–53

  • Khalfallah J, Slama JBH (2015) Facial expression recognition for intelligent tutoring systems in remote laboratories platform. Procedia Comput Sci 73:274–281. https://doi.org/10.1016/j.procs.2015.12.030

    Article  Google Scholar 

  • Krithika LB (2016) Student emotion recognition system (SERS) for e-learning improvement based on learner concentration metric. Procedia Comput Sci 85:767–776. https://doi.org/10.1016/j.procs.2016.05.264

    Article  Google Scholar 

  • Li K, ** Y, Akram MW, Han R, Chen J (2020) Facial expression recognition with convolutional neural networks via a new face crop** and rotation strategy. Vis Comput 36:391–404

    Article  Google Scholar 

  • Mega C, Ronconi L, De Beni R (2014) What makes a good student? How emotions, self-regulated learning, and motivation contribute to academic achievement. J Educ Psychol 106(1):121

    Article  Google Scholar 

  • Megahed M, Mohammed A (2020) Modeling adaptive E-Learning environment using facial expressions and fuzzy logic. Expert Syst Appl 157:113460. https://doi.org/10.1016/j.eswa.2020.113460

    Article  Google Scholar 

  • Mollahosseini A, Chan D, Mahoor MH (2016) Going deeper in facial expression recognition using deep neural networks. In: 2016 IEEE Winter conference on applications of computer vision (WACV). IEEE, pp 1–10

  • Myneni MB, Srividya Y, Dandamudi A (2017) Correlated cluster-based imputation for treatment of missing values. In: Satapathy S, Prasad V, Rani B, Udgata S, Raju K (eds) Proceedings of the first international conference on computational intelligence and informatics. Advances in intelligent systems and computing, vol 507. Springer, Singapore. https://doi.org/10.1007/978-981-10-2471-9_17

  • Oguine OC et al (2022) Hybrid facial expression recognition (FER2013) model for real-time emotion classification and prediction. Ar**v abs/2206.09509

  • Padmaja B, Myneni MB, Krishna Rao Patro E (2020) A comparison on visual prediction models for MAMO (multi activity-multi object) recognition using deep learning. J Big Data. https://doi.org/10.1186/s40537-020-00296-8

    Article  Google Scholar 

  • Pekrun R, Goetz T, Titz W, Perry RP (2002) Academic emotions in students’ self-regulated learning and achievement: a program of qualitative and quantitative research. Educ Psychol 37(2):91–105

    Article  Google Scholar 

  • Phan-Xuan H, Le-Tien T, Nguyen-Tan S (2019) Fpga platform applied for facial expression recognition system using convolutional neural networks. Procedia Comput Sci 151:651–658. https://doi.org/10.1016/j.procs.2019.04.087

    Article  Google Scholar 

  • Rajendra Prasad K, Kamatam GR, Myneni MB, Ramanjaneya Reddy N (2022) A novel data visualization method for the effective assessment of cluster tendency through the dark blocks image pattern analysis. Microprocess Microsyst 93:104625. https://doi.org/10.1016/j.micpro.2022.104625

    Article  Google Scholar 

  • Shan K, Guo J, You W, Lu D, Bie R (2017) Automatic facial expression recognition based on a deep convolutional-neural-network. In: 2017 IEEE 15th international conference on software engineering research, management and applications (SERA), pp 123–128

  • Siregar B, Wirtjes JS, Nababan EB, Fahmi (2021) Development of a person’s facial expression recognition system using a convolutional neural network. J Phys: Conf Ser

  • Truong KP, van Leeuwen DA, Neerincx MA (2007) Unobtrusive multimodal emotion detection in adaptive interfaces: speech and facial expressions. In: Foundations of augmented cognition, pp 354–363. https://doi.org/10.1007/978-3-540-73216-7 40

  • Wu CH (2016) New technology for develo** facial expression recognition in e-learning. In: 2016 Portland international conference on management of engineering and technology, pp 1719–1722

  • Ye F (2022) Emotion recognition of online education learners by convolutional neural networks. In: Computational intelligence and neuroscience, Hindawi

Download references

Acknowledgements

The computational facility developed in the college as part of the DST's FIST program is gratefully acknowledged by the authors (SR/FST/College-2017/28(c)) laboratory. As a result, they can better complete the task. The IARE management is acknowledged by the writers for their assistance and encouraging words.

Author information

Authors and Affiliations

  1. Institute of Aeronautical Engineering, Hyderabad, India

    Myneni Madhu Bala, Siva Abhishek Sirivella, Siddharth Ambati & Krishna Vamshi Potharaju Venkata Sai

  2. PVP Siddhartha Institute of Technology, Vijayawada, Andhra Pradesh, India

    Haritha Akkineni

Authors
  1. Myneni Madhu Bala
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haritha Akkineni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Siva Abhishek Sirivella
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Siddharth Ambati
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Krishna Vamshi Potharaju Venkata Sai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Myneni Madhu Bala.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bala, M.M., Akkineni, H., Sirivella, S.A. et al. Implementation of an adaptive E-learning platform with facial emotion recognition. Microsyst Technol 29, 609–619 (2023). https://doi.org/10.1007/s00542-023-05420-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00542-023-05420-1

Search

Navigation