SpringerLink
Log in

Evaluating the performance of the cognitive workload model with subjective endorsement in addition to EEG

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

The aptitude-oriented exercises from almost all domains impose cognitive load on their operators. Evaluating such load poses several challenges owing to many factors like measurement mode and complexity, nature of the load, overloading conditions, etc. Nevertheless, the physiological measurement of a specific genre of cognitive load and subjective measurement have not been reported along with each other. In this study, the electroencephalography (EEG)-driven machine learning (Support Vector Machine (SVM)) model is sought along with the support of NASA’s Task Load Index (NASA-TLX) rating scale for a novel purpose in workload exploration of operators. The Cognitive Load Theory (CLT) was used as the foundation to design the intrinsic stimulus (Spot the Difference task), as most workloads operators are exposed to are notably intrinsic. The SVM-based three-level classification accuracy ranged from 85.4 to 97.4% (p < 0.05), and the NASA-TLX-based three-level classification accuracy ranged from 88.33 to 97.33%. The t-test results show that the neurometric indices contributing to the classification significantly differed (p < 0.05) for every level. The NASA-TLX scale was utilised for validation in its basic form after the validity (Pearson correlation coefficients 0.338 to 0.805 (p < 0.05)) and reliability (Cronbach’s α = 0.753) test. This modeling is beneficial to phase out particular-level cognitive exercises from the curriculum during under or overload workload (critical) conditions.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Balfe N, Sharples S, Wilson JR (2015) Impact of automation: measurement of performance, workload and behaviour in a complex control environment. Appl Ergon 47:52–64. https://doi.org/10.1016/j.apergo.2014.08.002

    Article  PubMed  Google Scholar 

  2. Ismail LE, Karwowski W (2020) Applications of EEG indices for the quantification of human cognitive performance: a systematic review and bibliometric analysis. PLoS ONE 15:e0242857

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Dimitrakopoulos GN, Kakkos I, Dai Z et al (2017) Task-independent mental workload classification based upon common multiband EEG cortical connectivity. IEEE Trans Neural Syst Rehabil Eng 25:1940–1949. https://doi.org/10.1109/TNSRE.2017.2701002

    Article  PubMed  Google Scholar 

  4. Zhou Y, Huang S, Xu Z et al (2021) Cognitive workload recognition using EEG signals and machine learning: a review. IEEE Trans Cogn Dev Syst 14(3):799–818. https://doi.org/10.1109/TCDS.2021.3090217

  5. Zhu G, Zong F, Zhang H et al (2021) Cognitive load during multitasking can be accurately assessed based on single channel electroencephalography using graph methods. IEEE Access 9:33102–33109

    Article  Google Scholar 

  6. Na K (2021) The effects of cognitive load on query reformulation: mental demand, temporal demand and frustration. Aslib J Inf Manag 73(3):436–453. https://doi.org/10.1108/AJIM-07-2020-0206

  7. Orru G, Longo L (2018) The evolution of cognitive load theory and the measurement of its intrinsic, extraneous and Germane loads: a review. In: International symposium on human mental workload: models and applications. Springer, pp 23–48. https://doi.org/10.1007/978-3-030-14273-5_3

  8. Hart SG (2006) NASA-task load index (NASA-TLX); 20 years later. In: Proceedings of the human factors and ergonomics society annual meeting. Sage Publications Sage CA, Los Angeles, pp 904–908

  9. Cooper GE (1969) The use of pilot rating in the evaluation of aircraft handling qualities. NASA TND-5153

  10. Yin Z, Zhang J (2017) Cross-session classification of mental workload levels using EEG and an adaptive deep learning model. Biomed Signal Process Control 33:30–47

    Article  Google Scholar 

  11. Parent M, Peysakhovich V, Mandrick K et al (2019) The diagnosticity of psychophysiological signatures: can we disentangle mental workload from acute stress with ECG and fNIRS? Int J Psychophysiol 146:139–147. https://doi.org/10.1016/j.ijpsycho.2019.09.005

    Article  PubMed  Google Scholar 

  12. Katyal A, Singla R (2020) Towards enhanced information transfer rate: a comparative study based on classification techniques. Comput Methods Biomech Biomed Eng Imaging Vis 8(4):446–457. https://doi.org/10.1080/21681163.2020.1727775

  13. Lin C-T, Wang Y, Chen S-F et al (2023) Design and verification of a wearable wireless 64-channel high-resolution EEG acquisition system with wi-fi transmission. Med Biol Eng Comput. https://doi.org/10.1007/s11517-023-02879-y

    Article  PubMed  Google Scholar 

  14. Khosla A, Khandnor P, Chand T (2020) A comparative analysis of signal processing and classification methods for different applications based on EEG signals. Biocybern Biomed Eng 40:649–690. https://doi.org/10.1016/j.bbe.2020.02.002

    Article  Google Scholar 

  15. Zakeri Z, Arif A, Omurtag A et al (2023) Multimodal assessment of cognitive workload using neural, subjective and behavioural measures in smart factory settings. Sensors 23:8926

    Article  PubMed  PubMed Central  Google Scholar 

  16. Mastropietro A, Pirovano I, Marciano A et al (2023) Reliability of mental workload index assessed by EEG with different electrode configurations and signal pre-processing pipelines. Sensors 23:1367

    Article  PubMed  PubMed Central  Google Scholar 

  17. Raufi B, Longo L (2022) An evaluation of the EEG alpha-to-theta and theta-to-alpha band ratios as indexes of mental workload. ar**v Prepr ar**v220212937

  18. Zhao Y, Dai G, Borghini G et al (2021) Label-based alignment multi-source domain adaptation for cross-subject EEG fatigue mental state evaluation. Front Hum Neurosci 15. https://doi.org/10.3389/fnhum.2021.706270

  19. Brunzini A, Peruzzini M, Grandi F et al (2021) A preliminary experimental study on the workers’ workload assessment to design industrial products and processes. Appl Sci 11:12066

    Article  CAS  Google Scholar 

  20. Andreessen LM, Gerjets P, Meurers D, Zander TO (2020) Toward neuroadaptive support technologies for improving digital reading: a passive BCI-based assessment of mental workload imposed by text difficulty and presentation speed during reading. User Model User-adapt Interact 31:75–104

  21. Anand V, Ahmed Z, Sreeja SR (2019) An automated approach for task evaluation using EEG signals. ar**v Prepr ar**v191102966

  22. Plechawska-Wójcik M, Tokovarov M, Kaczorowska M, Zapała D (2019) A three-class classification of cognitive workload based on EEG spectral data. Appl Sci 9(24):5340. https://doi.org/10.3390/app9245340

  23. Gogna Y, Tiwari S, Singla R (2023) Towards a versatile mental workload modeling using neurometric indices. Biomed Eng Tech 68(3):297–316. https://doi.org/10.1515/bmt-2022-0479

  24. Grissmann S, Spuler M, Faller J et al (2017) Context sensitivity of EEG-based workload classification under different affective valence. IEEE Trans Affect Comput 11(2):327–334. https://doi.org/10.1109/TAFFC.2017.2775616

  25. Bashivan P, Yeasin M, Bidelman GM (2014) Modulation of brain connectivity by memory load in a working memory network. In: 2014 IEEE symposium on computational intelligence, cognitive algorithms, mind, and brain (CCMB). IEEE, pp 127–133. https://doi.org/10.1109/CCMB.2014.7020705

  26. Nicolas-Alonso LF, Gomez-Gil J (2012) Brain computer interfaces, a review. Sensors 12:1211–1279

    Article  PubMed  PubMed Central  Google Scholar 

  27. Mathôt S, Schreij D, Theeuwes J (2012) OpenSesame: an open-source, graphical experiment builder for the social sciences. Behav Res Methods 44:314–324

    Article  PubMed  Google Scholar 

  28. Hart SG, Staveland LE (1988) Development of NASA-TLX (task load index): results of empirical and theoretical research. In: Hancock PA, Meshkati N (eds) Human mental workload. North-Holland, 52:139–183. https://doi.org/10.1016/S0166-4115(08)62386-9

  29. Hart SG (1986) NASA task load index (TLX). https://ntrs.nasa.gov/citations/20000021487

  30. Ille N, Berg P, Scherg M (2002) Artifact correction of the ongoing EEG using spatial filters based on artifact and brain signal topographies. J Clin Neurophysiol 19:113–124

    Article  PubMed  Google Scholar 

  31. Rathi N, Singla R, Tiwari S (2022) A comparative study of classification methods for designing a pictorial P300-based authentication system. Med Biol Eng Comput 60:2899–2916. https://doi.org/10.1007/s11517-022-02626-9

    Article  PubMed  Google Scholar 

  32. Liu P-K, Beh W, Shih C-Y et al (2019) Entropy and complexity assisted EEG-based mental workload assessment system. In: 2019 IEEE biomedical circuits and systems conference (BioCAS), pp 1–4. https://doi.org/10.1109/BIOCAS.2019.8919019

  33. Morton J, Vanneste P, Larmuseau C et al (2019) Identifying predictive EEG features for cognitive overload detection in assembly workers in Industry 4.0. In: 3rd international symposium on human mental workload: models and applications (HWORKLOAD 2019), p 1

  34. Wu C, Liu Y, Guo X et al (2022) Enhancing the feasibility of cognitive load recognition in remote learning using physiological measures and an adaptive feature recalibration convolutional neural network. Med Biol Eng Comput 60:3447–3460. https://doi.org/10.1007/s11517-022-02670-5

    Article  PubMed  PubMed Central  Google Scholar 

  35. Zarjam P, Epps J, Lovell NH (2015) Beyond subjective self-rating: EEG signal classification of cognitive workload. IEEE Trans Auton Ment Dev 7:301–310

    Article  Google Scholar 

  36. Zammouri A, Chraa-Mesbahi S, Moussa AA et al (2017) Brain waves-based index for workload estimation and mental effort engagement recognition. J Phys Conf Ser 904(1):012008. https://doi.org/10.1088/1742-6596/904/1/012008

  37. Sinha A, Chatterjee D, Saha SK, Basu A (2015) Validation of stimulus for EEG signal based cognitive load analysis. In: 2015 fifth national conference on computer vision, pattern recognition, image processing and graphics (NCVPRIPG). IEEE, pp 1–4. https://doi.org/10.1109/NCVPRIPG.2015.7490067

  38. Gogna Y, Singla R, Tiwari S (2020) Analyzing attention deviation during collaterally proceeding cognitive tasks. In: International congress on information and communication technology, vol 1. Springer, pp 490–497

  39. Gogna Y, Singla R, Tiwari S (2019) Steady state detection during a cognitive task. In: 2019 IEEE 5th international conference for convergence in technology (I2CT). IEEE, pp 1–4. https://doi.org/10.1109/I2CT45611.2019.9033870

  40. Sleigh JW, Donovan J (1999) Comparison of bispectral index, 95% spectral edge frequency and approximate entropy of the EEG, with changes in heart rate variability during induction of general anaesthesia. Br J Anaesth 82:666–671

    Article  CAS  PubMed  Google Scholar 

  41. Acharya UR, Fujita H, Sudarshan VK et al (2015) Application of entropies for automated diagnosis of epilepsy using EEG signals: a review. Knowl-Based Syst 88:85–96

    Article  Google Scholar 

  42. Wang Q, Sourina O (2013) Real-time mental arithmetic task recognition from EEG signals. IEEE Trans Neural Syst Rehabil Eng 21:225–232

    Article  PubMed  Google Scholar 

  43. Rathi N, Singla R, Tiwari S (2021) A novel approach for designing authentication system using a picture based P300 speller. Cogn Neurodyn 15:805–824. https://doi.org/10.1007/s11571-021-09664-3

  44. So WKY, Wong SWH, Mak JN, Chan RHM (2017) An evaluation of mental workload with frontal EEG. PLoS ONE 12:e0174949

    Article  PubMed  PubMed Central  Google Scholar 

  45. Aghajani H, Garbey M, Omurtag A (2017) Measuring mental workload with EEG+ fNIRS. Front Hum Neurosci 11:359

    Article  PubMed  PubMed Central  Google Scholar 

  46. Rathi N, Singla R, Tiwari S (2020) Authentication framework for security application developed using a pictorial P300 speller. Brain-Comput Interfaces 7:70–89. https://doi.org/10.1080/2326263X.2020.1860520

    Article  Google Scholar 

  47. Ali M, Son D-H, Kang S-H, Nam S-R (2017) An accurate CT saturation classification using a deep learning approach based on unsupervised feature extraction and supervised fine-tuning strategy. Energies 10:1830. https://doi.org/10.3390/en10111830

    Article  Google Scholar 

  48. Raufi B, Longo L (2022) An Evaluation of the EEG alpha-to-theta and theta-to-alpha band ratios as indexes of mental workload. Front Neuroinformatics 16:44

  49. Al-Shargie F, Tang TB, Badruddin N, Kiguchi M (2018) Towards multilevel mental stress assessment using SVM with ECOC: an EEG approach. Med Biol Eng Comput 56:125–136

  50. Mun S, Whang M, Park S, Park MC (2017) Effects of mental workload on involuntary attention: A somatosensory ERP study. Neuropsychologia 106:7–20

  51. Al-Shargie FM, Tang TB, Badruddin N, Kiguchi M (2016) Mental stress quantification using EEG signals. In: International conference for innovation in biomedical engineering and life sciences: ICIBEL2015, 6-8 December 2015, Putrajaya, Malaysia 1. Springer, Singapore, pp 15–19

Download references

Acknowledgements

The authors thank the subjects for patiently attending this experiment, followed by the feedback process.

Author information

Authors and Affiliations

  1. ICE Department, Dr. B. R. Ambedkar NIT Jalandhar, GT Road Bye-Pass, Jalandhar, Punjab, 144008, India

    Yamini Gogna, Sheela Tiwari & Rajesh Singla

Authors
  1. Yamini Gogna
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sheela Tiwari
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rajesh Singla
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yamini Gogna.

Ethics declarations

Ethics approval and consent to participate

Experiments on human subjects were conducted with the prior consent of the subject and the ethical approval of the Ethics Committee of the Institute.

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gogna, Y., Tiwari, S. & Singla, R. Evaluating the performance of the cognitive workload model with subjective endorsement in addition to EEG. Med Biol Eng Comput 62, 2019–2036 (2024). https://doi.org/10.1007/s11517-024-03049-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-024-03049-4

Keywords

Search

Navigation