SpringerLink
Log in

A Survey on the Application of Virtual Reality in Event-Related Potential Research

  • Conference paper
  • First Online:
Machine Learning and Knowledge Extraction (CD-MAKE 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13480))

  • 1206 Accesses

Abstract

Virtual reality (VR) is getting traction in many contexts, allowing users to have a real-life experience in a virtual world. However, its application in the field of Neuroscience, and above all probing newer activity with the analysis of electroencephalographic (EEG) event-related potentials (ERP) is underexplored. This article reviews the state-of-the-art applications of virtual reality in ERP research, analysing current ways to integrate Head-Mounted Displays (HMD) with EEG head-sets for deploying ecologically valid experiments. It also identifies which ERP components are appropriate in VR settings, along with their paradigms, the technical configurations of the experiments conducted, and the reliability of the findings. Finally, the article synthesises this survey, providing recommendations to practitioners and scholars.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
EUR 29.95
Price includes VAT (Germany)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
EUR 74.89
Price includes VAT (Germany)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
EUR 96.29
Price includes VAT (Germany)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://www.brainproducts.com/.

  2. 2.

    https://pressrelease.brainproducts.com/active-electrodes-walkthrough/.

  3. 3.

    https://pressrelease.brainproducts.com/easycap-cap-overview/.

  4. 4.

    https://www.gtec.at/product/ggammasys/.

References

  1. Ahn, M.H., Park, J.H., Jeon, H., Lee, H.J., Kim, H.J., Hong, S.K.: Temporal dynamics of visually induced motion perception and neural evidence of alterations in the motion perception process in an immersive virtual reality environment. Front. Neurosci. 14 (2020)

    Google Scholar 

  2. Aksoy, M., Ufodiama, C.E., Bateson, A.D., Martin, S., Asghar, A.U.R.: A comparative experimental study of visual brain event-related potentials to a working memory task: virtual reality head-mounted display versus a desktop computer screen. Exp. Brain Res. 239(10), 3007–3022, 104107 (2021). https://doi.org/10.1007/s00221-021-06158-w

  3. Arake, M., et al.: Measuring task-related brain activity with event-related potentials in dynamic task scenario with immersive virtual reality environment. Front. Behav. Neurosci. 16, 11 (2022)

    Google Scholar 

  4. Baumgartner, T., Valko, L., Esslen, M., Jäncke, L.: Neural correlate of spatial presence in an arousing and noninteractive virtual reality: an EEG and psychophysiology study. CyberPsychol. Behav. 9(1), 30–45 (2006)

    Article  Google Scholar 

  5. Burns, C.G., Fairclough, S.H.: Use of auditory event-related potentials to measure immersion during a computer game. Int. J. Hum Comput Stud. 73, 107–114 (2015)

    Article  Google Scholar 

  6. Cattan, G., Andreev, A., Visinoni, E.: Recommendations for integrating a p300-based brain-computer interface in virtual reality environments for gaming: an update. Computers 9(4), 92 (2020)

    Article  Google Scholar 

  7. Dey, A., Chatburn, A., Billinghurst, M.: Exploration of an EEG-based cognitively adaptive training system in virtual reality. In: 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 220–226. IEEE (2019)

    Google Scholar 

  8. Du, J., Ke, Y., Kong, L., Wang, T., He, F., Ming, D.: 3D stimulus presentation of ERP-speller in virtual reality. In: 2019 9th International IEEE/EMBS Conference on Neural Engineering (NER), pp. 167–170. IEEE (2019)

    Google Scholar 

  9. Erdogdu, E., Kurt, E., Duru, A.D., Uslu, A., Başar-Eroğlu, C., Demiralp, T.: Measurement of cognitive dynamics during video watching through event-related potentials (ERPS) and oscillations (EROS). Cogn. Neurodyn. 13(6), 503–512 (2019)

    Article  Google Scholar 

  10. Garduno Luna, C.D.: Feasibility of virtual and augmented reality devices as psychology research tools: a pilot study. Ph.D. thesis, UC Santa Barbara (2020)

    Google Scholar 

  11. Gehrke, L., et al.: Detecting visuo-haptic mismatches in virtual reality using the prediction error negativity of event-related brain potentials. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–11 (2019)

    Google Scholar 

  12. Grassini, S., Laumann, K., Thorp, S., Topranin, V.d.M.: Using electrophysiological measures to evaluate the sense of presence in immersive virtual environments: an event-related potential study. Brain Behav. 11(8), e2269 (2021)

    Google Scholar 

  13. Hajcak, G., Klawohn, J., Meyer, A.: The utility of event-related potentials in clinical psychology. Annu. Rev. Clin. Psychol. 15, 71–95 (2019)

    Article  Google Scholar 

  14. Harjunen, V.J., Ahmed, I., Jacucci, G., Ravaja, N., Spapé, M.M.: Manipulating bodily presence affects cross-modal spatial attention: a virtual-reality-based ERP study. Front. Hum. Neurosci. 11, 79 (2017)

    Article  Google Scholar 

  15. Herweg, A., Gutzeit, J., Kleih, S., Kübler, A.: Wheelchair control by elderly participants in a virtual environment with a brain-computer interface (BCI) and tactile stimulation. Biol. Psychol. 121, 117–124 (2016)

    Google Scholar 

  16. Hou, G., Dong, H., Yang, Y.: Develo** a virtual reality game user experience test method based on EEG signals. In: 2017 5th International Conference on Enterprise Systems (ES), pp. 227–231. IEEE (2017)

    Google Scholar 

  17. Hubbard, R., Sipolins, A., Zhou, L.: Enhancing learning through virtual reality and neurofeedback: a first step. In: Proceedings of the Seventh International Learning Analytics & Knowledge Conference, pp. 398–403 (2017)

    Google Scholar 

  18. Hyun, K.Y., Lee, G.H.: Analysis of change of event related potential in escape test using virtual reality technology. Biomed. Sci. Lett. 25(2), 139–148 (2019)

    Article  Google Scholar 

  19. Käthner, I., Kübler, A., Halder, S.: Rapid p300 brain-computer interface communication with a head-mounted display. Front. Neurosci. 9, 207 (2015)

    Article  Google Scholar 

  20. Kim, S., Lee, S., Kang, H., Kim, S., Ahn, M.: P300 brain-computer interface-based drone control in virtual and augmented reality. Sensors 21(17), 5765 (2021)

    Article  Google Scholar 

  21. Kirasirova, L., Zakharov, A., Morozova, M., Kaplan, A.Y., Pyatin, V.: ERP correlates of emotional face processing in virtual reality. Opera Med. Physiol. 8(3), 12–19 (2021)

    Google Scholar 

  22. Kober, S.E., Neuper, C.: Using auditory event-related EEG potentials to assess presence in virtual reality. Int. J. Hum Comput Stud. 70(9), 577–587 (2012)

    Article  Google Scholar 

  23. Li, G., Zhou, S., Kong, Z., Guo, M.: Closed-loop attention restoration theory for virtual reality-based attentional engagement enhancement. Sensors 20(8), 2208 (2020)

    Google Scholar 

  24. Liang, S., Choi, K.S., Qin, J., Pang, W.M., Wang, Q., Heng, P.A.: Improving the discrimination of hand motor imagery via virtual reality based visual guidance. Comput. Methods Programs Biomed. 132, 63–74 (2016)

    Article  Google Scholar 

  25. Lin, C.T., Chung, I.F., Ko, L.W., Chen, Y.C., Liang, S.F., Duann, J.R.: EEG-based assessment of driver cognitive responses in a dynamic virtual-reality driving environment. IEEE Trans. Biomed. Eng. 54(7), 1349–1352 (2007)

    Article  Google Scholar 

  26. Ogawa, R., Kageyama, K., Nakatani, Y., Ono, Y., Murakami, S.: Event-related potentials-based evaluation of attention allocation while watching virtual reality. Adv. Biomed. Eng. 11, 1–9 (2022)

    Article  Google Scholar 

  27. Pavone, E.F., Tieri, G., Rizza, G., Tidoni, E., Grisoni, L., Aglioti, S.M.: Embodying others in immersive virtual reality: electro-cortical signatures of monitoring the errors in the actions of an avatar seen from a first-person perspective. J. Neurosci. 36(2), 268–279 (2016)

    Article  Google Scholar 

  28. Peeters, D.: Bilingual switching between languages and listeners: insights from immersive virtual reality. Cognition 195, 104107 (2020)

    Google Scholar 

  29. Peterson, S.M., Furuichi, E., Ferris, D.P.: Effects of virtual reality high heights exposure during beam-walking on physiological stress and cognitive loading. PLoS ONE 13(7) (2018)

    Google Scholar 

  30. Petras, K., Ten Oever, S., Jansma, B.M.: The effect of distance on moral engagement: Event related potentials and alpha power are sensitive to perspective in a virtual shooting task. Front. Psychol. 6, 2008 (2016)

    Google Scholar 

  31. Pezzetta, R., Nicolardi, V., Tidoni, E., Aglioti, S.M.: Error, rather than its probability, elicits specific electrocortical signatures: a combined EEG-immersive virtual reality study of action observation. J. Neurophysiol. 120(3), 1107–1118 (2018)

    Google Scholar 

  32. Picton, T.W., et al.: Guidelines for using human event-related potentials to study cognition: recording standards and publication criteria. Psychophysiology 37(2), 127–152 (2000)

    Google Scholar 

  33. Schubring, D., Kraus, M., Stolz, C., Weiler, N., Keim, D.A., Schupp, H.: Virtual reality potentiates emotion and task effects of alpha/beta brain oscillations. Brain Sci. 10(8), 537 (2020)

    Google Scholar 

  34. Si-Mohammed, H., et al.: Detecting system errors in virtual reality using EEG through error-related potentials. In: 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 653–661. IEEE (2020)

    Google Scholar 

  35. Simões, M.., Amaral, C.., Carvalho, Paulo, Castelo-Branco, Miguel: Specific EEG/ERP responses to dynamic facial expressions in virtual reality environments. In: Zhang, Yuan-Ting. (ed.) The International Conference on Health Informatics. IP, vol. 42, pp. 331–334. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-03005-0_84

  36. Singh, A.K., Chen, H.T., Cheng, Y.F., King, J.T., Ko, L.W., Gramann, K., Lin, C.T.: Visual appearance modulates prediction error in virtual reality. IEEE Access 6, 24617–24624 (2018)

    Google Scholar 

  37. Sun, R., Wu, Y.J., Cai, Q.: The effect of a virtual reality learning environment on learners’ spatial ability. Virtual Reality 23(4), 385–398 (2019)

    Google Scholar 

  38. Tauscher, J.P., Schottky, F.W., Grogorick, S., Bittner, P.M., Mustafa, M., Magnor, M.: Immersive EEG: evaluating electroencephalography in virtual reality. In: 2019 Conference on Virtual Reality and 3D User Interfaces, pp. 1794–1800. IEEE (2019)

    Google Scholar 

  39. de Tommaso, M., et al.: Pearls and pitfalls in brain functional analysis by event-related potentials: a narrative review by the Italian psychophysiology and cognitive neuroscience society on methodological limits and clinical reliability-Part I. Neurol. Sci. 41, 3503–3515 (2020)

    Google Scholar 

  40. Török, Á., et al.: Comparison between wireless and wired EEG recordings in a virtual reality lab: Case report. In: 2014 5th Conference on Cognitive Infocommunications (CogInfoCom), pp. 599–603. IEEE (2014)

    Google Scholar 

  41. Tosoni, A., Altomare, E.C., Brunetti, M., Croce, P., Zappasodi, F., Committeri, G.: Sensory-motor modulations of EEG event-related potentials reflect walking-related macro-affordances. Brain Sci. 11(11), 1506 (2021)

    Google Scholar 

  42. Vass, L.K., et al.: Oscillations go the distance: low-frequency human hippocampal oscillations code spatial distance in the absence of sensory cues during teleportation. Neuron 89(6), 1180–1186 (2016)

    Google Scholar 

  43. Wu, J., Zhou, Q., Li, J., Kong, X., **ao, Y.: Inhibition-related n2 and p3: Indicators of visually induced motion sickness (vims). Int. J. Ind. Ergon. 78, 102981 (2020)

    Google Scholar 

  44. Yokota, Y., Naruse, Y.: Temporal fluctuation of mood in gaming task modulates feedback negativity: Eeg study with virtual reality. Front. Hum. Neurosci. 15, 246 (2021)

    Article  Google Scholar 

Download references

Acknowledgement

This work was conducted with the financial support of the Science Foundation Ireland Centre for Research Training in Digitally-Enhanced Reality (D-real) under Grant No. 18/CRT/6224.

Author information

Authors and Affiliations

  1. The Artificial Intelligence and Cognitive Load Research Lab, The Applied Intelligence Research Center, School of Computer Science, Technological University Dublin, Dublin, Ireland

    Vladimir Marochko & Luca Longo

  2. School of Medicine, Institute of Neuroscience, Trinity College Dublin, The University of Dublin, Dublin, Ireland

    Richard Reilly

  3. School of Computer Science and Statistics, Trinity College, The University of Dublin, Dublin, Ireland

    Rachel McDonnell

Authors
  1. Vladimir Marochko
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Richard Reilly
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rachel McDonnell
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Luca Longo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vladimir Marochko .

Editor information

Editors and Affiliations

  1. University of Natural Resources and Life Sciences Vienna, Vienna, Austria

    Andreas Holzinger

  2. St. Pölten University of Applied Sciences, St. Pölten, Austria

    Peter Kieseberg

  3. TU Wien, Vienna, Austria

    A Min Tjoa

  4. SBA Research, Vienna, Austria

    Edgar Weippl

Rights and permissions

Reprints and permissions

Copyright information

© 2022 IFIP International Federation for Information Processing

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Marochko, V., Reilly, R., McDonnell, R., Longo, L. (2022). A Survey on the Application of Virtual Reality in Event-Related Potential Research. In: Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds) Machine Learning and Knowledge Extraction. CD-MAKE 2022. Lecture Notes in Computer Science, vol 13480. Springer, Cham. https://doi.org/10.1007/978-3-031-14463-9_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-14463-9_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-14462-2

  • Online ISBN: 978-3-031-14463-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation