SpringerLink
Log in

Emotion Prediction in Real-Life Scenarios: On the Way to the BIRAFFE3 Dataset

  • Conference paper
  • First Online:
Artificial Intelligence for Neuroscience and Emotional Systems (IWINAC 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14674))

  • 203 Accesses

Abstract

Despite over 20 years of research in affective computing, emotion prediction models that would be useful in real-life out-of-the-lab scenarios such as health care or intelligent assistants have still not been developed. The identification of the fundamental problems behind this concern led to the initiation of the BIRAFFE series of experiments, whose main goal is to develop a set of techniques, tools and good practices to introduce personalized context-based emotion processing modules in intelligent systems/assistants. The aim of this work is to present the work-in-progress concept of the third experiment in the BIRAFFE series and discuss the results of the pilot study. After all conclusions have been drawn up, actual study will be carried out, and then the collected data will be processed and made available under the creative commons license as BIRAFFE3 dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
EUR 29.95
Price includes VAT (Germany)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
EUR 128.39
Price includes VAT (Germany)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
EUR 87.73
Price includes VAT (Germany)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bhatt, P., et al.: Machine learning for cognitive behavioral analysis: datasets, methods, paradigms, and research directions. Brain Inform. 10(1), 18 (2023). https://doi.org/10.1186/s40708-023-00196-6

    Article  Google Scholar 

  2. Bradley, M.M., Lang, P.J.: The international affective digitized sounds (2nd edition; iads-2): affective ratings of sounds and instruction manual. technical report B-3. Technical report, University of Florida, Gainsville, FL (2007)

    Google Scholar 

  3. Costa, P., McCrae, R.: Revised NEO Personality Inventory (NEO-PI-R) and NEO Five Factor Inventory (NEO-FFI). Professional manual. Psychological Assessment Resources, Odessa, FL (1992)

    Google Scholar 

  4. Dan-Glauser, E.S., Scherer, K.R.: The geneva affective picture database (GAPED): a new 730-picture database focusing on valence and normative significance. Behav. Res. Methods 43(2), 468–477 (2011). https://doi.org/10.3758/s13428-011-0064-1

    Article  Google Scholar 

  5. van Dooren, M., de Vries, J.J.G., Janssen, J.H.: Emotional sweating across the body: comparing 16 different skin conductance measurement locations. Physiol. Behav. 106(2), 298–304 (2012)

    Article  Google Scholar 

  6. Dzedzickis, A., Kaklauskas, A., Bucinskas, V.: Human emotion recognition: review of sensors and methods. Sensors 20(3), 592 (2020). https://doi.org/10.3390/s20030592

    Article  Google Scholar 

  7. Fanourakis, M., Chanel, G.: AMuCS: affective multimodal counter-strike video game dataset (2024). https://doi.org/10.36227/techrxiv.170630398.84528625/v1

  8. Hasnul, M.A., Aziz, N.A.B.A., Alelyani, S., Mohana, M., Aziz, A.A.: Electrocardiogram-based emotion recognition systems and their applications in healthcare - a review. Sensors 21(15), 5015 (2021). https://doi.org/10.3390/s21155015

    Article  Google Scholar 

  9. IJsselsteijn, W.A., de Kort, Y.A.W., Poels, K.: The Game Experience Questionnaire. Technische Universiteit Eindhoven (2013)

    Google Scholar 

  10. Katsis, C.D., Katertsidis, N.S., Ganiatsas, G., Fotiadis, D.I.: Toward emotion recognition in car-racing drivers: a biosignal processing approach. IEEE Trans. Syst. Man Cybern. Part A 38(3), 502–512 (2008). https://doi.org/10.1109/TSMCA.2008.918624

    Article  Google Scholar 

  11. Khare, S.K., Blanes-Vidal, V., Nadimi, E.S., Acharya, U.R.: Emotion recognition and artificial intelligence: a systematic review (2014–2023) and research recommendations. Inf. Fusion 102, 102019 (2024). https://doi.org/10.1016/J.INFFUS.2023.102019

    Article  Google Scholar 

  12. Kutt, K., Bobek, S., Nalepa, G.J.: BIRAFFE: bio-reactions and faces for emotion-based personalization. Zenodohttps://doi.org/10.5281/zenodo.3442143 (2020)

  13. Kutt, K., Drążyk, D., Bobek, S., Nalepa, G.J.: Personality-based affective adaptation methods for intelligent systems. Sensors 21(1), 163 (2021). https://doi.org/10.3390/s21010163

    Article  Google Scholar 

  14. Kutt, K., et al.: BIRAFFE: bio-reactions and faces for emotion-based personalization. In: AfCAI 2019. CEUR Workshop Proceedings, vol. 2609. CEUR-WS.org (2020)

    Google Scholar 

  15. Kutt, K., Drążyk, D., Żuchowska, L., Szelążek, M., Bobek, S., Nalepa, G.J.: BIRAFFE2, a multimodal dataset for emotion-based personalization in rich affective game environments. Sci. Data 9, 274 (2022). https://doi.org/10.1038/s41597-022-01402-6

    Article  Google Scholar 

  16. Kutt, K., Ściga, Ł., Nalepa, G.J.: Emotion-based dynamic difficulty adjustment in video games. In: DSAA 2023, pp. 1–5. IEEE (2023). https://doi.org/10.1109/DSAA60987.2023.10302578

  17. Kutt, K., Sobczyk, P., Nalepa, G.J.: Evaluation of selected APIs for emotion recognition from facial expressions. In: Ferrández Vicente, J.M., Álvarez-Sánchez, J.R., de la Paz López, F., Adeli, H. (eds.) IWINAC 2022. LNCS, vol. 13259, pp. 65–74. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-06527-9_7

    Chapter  Google Scholar 

  18. Lang, P.J., Bradley, M.M., Cuthbert, B.N.: International affective picture system (IAPs): affective ratings of pictures and instruction manual. technical report B-3. Technical report, The Center for Research in Psychophysiology, University of Florida, Gainsville, FL (2008)

    Google Scholar 

  19. Lara-Cabrera, R., Camacho, D.: A taxonomy and state of the art revision on affective games. Futur. Gener. Comput. Syst. 92, 516–525 (2019)

    Article  Google Scholar 

  20. Michałowski, J.M., Droździel, D., Matuszewski, J., Koziejowski, W., Jednoróg, K., Marchewka, A.: The set of fear inducing pictures (SFIP): development and validation in fearful and nonfearful individuals. Behav. Res. Methods 49(4), 1407–1419 (2017). https://doi.org/10.3758/s13428-016-0797-y

    Article  Google Scholar 

  21. Milkowski, P., Saganowski, S., Gruza, M., Kazienko, P., Piasecki, M., Kocon, J.: Multitask personalized recognition of emotions evoked by textual content. In: PerCom 2022 Workshops, pp. 347–352. IEEE (2022). https://doi.org/10.1109/PerComWorkshops53856.2022.9767502

  22. Nalepa, G.J., Kutt, K., Giżycka, B., Jemioło, P., Bobek, S.: Analysis and use of the emotional context with wearable devices for games and intelligent assistants. Sensors 19(11), 2509 (2019). https://doi.org/10.3390/s19112509

    Article  Google Scholar 

  23. Park, C.Y., et al.: K-EmoCon, a multimodal sensor dataset for continuous emotion recognition in naturalistic conversations. Sci. Data 7(1), 293 (2020). https://doi.org/10.1038/s41597-020-00630-y

    Article  Google Scholar 

  24. Peirce, J., et al.: Psychopy2: experiments in behavior made easy. Behav. Res. Methods 51(1), 195–203 (2019). https://doi.org/10.3758/s13428-018-01193-y

    Article  Google Scholar 

  25. Phan, L.V., Rauthmann, J.F.: Personality computing: New frontiers in personality assessment. Soc. Pers. Psychol. Compass 15(7) (2021). https://doi.org/10.1111/spc3.12624

  26. Prokop, M., Pilar, L., Tichá, I.: Impact of think-aloud on eye-tracking: a comparison of concurrent and retrospective think-aloud for research on decision-making in the game environment. Sensors 20(10), 2750 (2020). https://doi.org/10.3390/s20102750

    Article  Google Scholar 

  27. Saganowski, S., Perz, B., Polak, A.G., Kazienko, P.: Emotion recognition for everyday life using physiological signals from wearables: a systematic literature review. IEEE Trans. Affect. Comput. 12(1), 1–21 (2021). https://doi.org/10.1109/TAFFC.2022.3176135

    Article  Google Scholar 

  28. Zawadzki, B., Strelau, J., Szczepaniak, P., Śliwińska, M.: Inwentarz osobowości NEO-FFI Costy i McCrae. Adaptacja polska. Pracownia Testów Psychologicznych, Warszawa (1998)

    Google Scholar 

  29. Zhao, S., Gholaminejad, A., Ding, G., Gao, Y., Han, J., Keutzer, K.: Personalized emotion recognition by personality-aware high-order learning of physiological signals. ACM Trans. Multim. Comput. Commun. Appl. 15(1s), 14:1–14:18 (2019). https://doi.org/10.1145/3233184

Download references

Acknowledgments

The research for this publication has been supported by a grant from the Priority Research Area DigiWorld under the Strategic Programme Excellence Initiative at Jagiellonian University. The research has been supported by a grant from the Faculty of Physics, Astronomy and Applied Computer Science under the Strategic Programme Excellence Initiative at Jagiellonian University.

Author information

Authors and Affiliations

  1. Jagiellonian Human-Centered AI Lab, Mark Kac Center for Complex Systems Research, Institute of Applied Computer Science, Faculty of Physics, Astronomy and Applied Computer Science, Jagiellonian University, Kraków, Poland

    Krzysztof Kutt & Grzegorz J. Nalepa

Authors
  1. Krzysztof Kutt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Grzegorz J. Nalepa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Krzysztof Kutt .

Editor information

Editors and Affiliations

  1. Universidad Politécnica de Cartagena, Cartagena, Spain

    José Manuel Ferrández Vicente

  2. Polytechnic University of Valencia, Valencia, Spain

    Mikel Val Calvo

  3. Ohio State University, Columbus, OH, USA

    Hojjat Adeli

Ethics declarations

Disclosure of Interests

The authors have no competing interests to declare that are relevant to the content of this article.

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kutt, K., Nalepa, G.J. (2024). Emotion Prediction in Real-Life Scenarios: On the Way to the BIRAFFE3 Dataset. In: Ferrández Vicente, J.M., Val Calvo, M., Adeli, H. (eds) Artificial Intelligence for Neuroscience and Emotional Systems. IWINAC 2024. Lecture Notes in Computer Science, vol 14674. Springer, Cham. https://doi.org/10.1007/978-3-031-61140-7_44

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-61140-7_44

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-61139-1

  • Online ISBN: 978-3-031-61140-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation