SpringerLink
Log in

Classification and Generation of Virtual Dancer Social Behaviors Based on Deep Learning in a Simple Virtual Environment Paradigm

  • Conference paper
  • First Online:
Biologically Inspired Cognitive Architectures 2021 (BICA 2021)

Part of the book series: Studies in Computational Intelligence ((SCI,volume 1032))

Included in the following conference series:

  • 654 Accesses

Abstract

This work examines one possibility of using a deep neural network to control a virtual dance partner behavior. A neural-network-based system is designed and used for classification, evaluation, prediction, and generation of socially emotional behavior of a virtual actor. The network is trained using deep learning on the data generated with an algorithm implementing the eBICA cognitive architecture. Results show that, in the selected virtual dance paradigm, (1) the functionality of the cognitive model can be efficiently transferred to the neural network using deep learning, allowing the network to generate socially emotional behavior of a dance partner similar to a human participant behavior or the behavior generated algorithmically based on eBICA, and (2) the trained neural network can correctly identify the character types of virtual dance partners based on their behavior. When considered together with related studies, our findings lead to more general implications extending beyond the selected paradigm.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
EUR 29.95
Price includes VAT (Germany)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
EUR 181.89
Price includes VAT (Germany)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
EUR 235.39
Price includes VAT (Germany)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info
Hardcover Book
EUR 235.39
Price includes VAT (Germany)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Trafton, J.G., Hiatt, L.M., Brumback, B., McCurry, J.M.: Using cognitive models to train big data models with small data. In: An, B., Yorke-Smith, N., El Fallah Seghrouchni, A., Sukthankar, G. (eds.). Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2020), pp. 1413–1421. International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC (2020). ISBN 978-1-4503-7518-4

    Google Scholar 

  2. Karabelnikova, Y., Samsonovich, A.V.: Virtual partner dance as a paradigm for empirical study of cognitive models of emotional intelligence. Procedia Comput. Sci. 190, 414–433 (2021). https://doi.org/10.1016/j.procs.2021.06.05

    Article  Google Scholar 

  3. Samsonovich, A.V.: Emotional biologically inspired cognitive architecture. Biol. Inspired Cogn. Archit. 6, 109–125 (2013)

    Google Scholar 

  4. Samsonovich, A.V.: Socially emotional brain-inspired cognitive architecture framework for artificial intelligence. Cogn. Syst. Res. 60, 57–76 (2020). https://doi.org/10.1016/j.cogsys.2019.12.002

    Article  Google Scholar 

  5. Marsella, S., Gratch, J., Petta, P.: Computational models of emotion. In: Scherer, K.R., Bänziger, T., Roesch, E. (eds.) A Blueprint for Affective Computing: A Sourcebook and Manual. Oxford University Press, Oxford (2010)

    Google Scholar 

  6. Goleman, D.: Emotional Intelligence. Bantam Books, New York (1995)

    Google Scholar 

  7. Goodfellow, I., Bengio, Y., Courville, A.: Deep learning. MIT Press, Cambridge (2016)

    MATH  Google Scholar 

  8. Khaikin, S.: Neural Networks: A Complete Course, 2nd edn. Williams Publishing House, Moscow (2006). 1104 p.

    Google Scholar 

  9. Scholle, F.: Deep Learning in Python. Publishing House “Peter”, St. Petersburg (2018). (in Russian)

    Google Scholar 

  10. Isakov, S.: Recurrent neural networks: types, training, examples and applications (Electronic resource). https://neurohive.io/ru/osnovy-data-science/rekurrentnye-nejronnye-seti. (in Russian)

  11. Gafarov, A.F.: G12 Artificial Neural Networks and Applications. Kazan Publishing House, Kazan (2018). (in Russian). 121 p.

    Google Scholar 

  12. Tsaregorodtsev, V.G.: Optimization of preprocessed data. Neurocomput. Dev. Appl. 7, 3–8 (2003)

    Google Scholar 

  13. Chollet, F.: Deep Learning with Python, 2nd edn. Manning Pub. Co., Shelter Island, New York (2021). ISBN-13: 978-1617296864

    Google Scholar 

  14. Nikolenko, S., Kadurin, A., Arkhangelskaya, E.: Deep learning. Piter, St. Petersburg (2018). (in Russian)

    Google Scholar 

  15. Osgood, C.E., Suci, G., Tannenbaum, P.: The Measurement of Meaning. University of Illinois Press, Urbana (1957)

    Google Scholar 

  16. Russell, J.A.: A circumplex model of affect. J. Pers. Soc. Psychol. 39(6), 1161–1178 (1980)

    Article  Google Scholar 

  17. Plutchik, R.: A psychoevolutionary theory of emotions. Soc. Sci. Inf. 21, 529–553 (1982)

    Article  Google Scholar 

  18. Lövheim, H.: A new three-dimensional model for emotions and monoamine neurotransmitters. Med. Hypotheses 78(2), 341–348 (2012)

    Article  Google Scholar 

  19. Harris, D., Harris, S.: Digital design and computer architecture, 2nd edn. Morgan Kaufmann, San Francisco (2012). ISBN 978-0-12-394424-5

    Google Scholar 

  20. Samsonovich, A.V.: A virtual actor behavior model based on emotional biologically inspired cognitive architecture. In: Goertzel, B., Iklé, M., Potapov, A. (eds.) AGI 2021. LNCS, vol. 13154, pp. 221–227. Springer, Cham (2022)

    Google Scholar 

  21. Samsonovich, A., Dodonov, A., Klychkov, M., Budanitsky, A., Grishin, I., Anisimova, A.: A virtual clown behavior model based on emotional biologically inspired cognitive architecture. In: Kryzhanovsky, B., Dunin-Barkowski, W., Redko, V., Tiumentsev, Y., Klimov, V.V. (eds.) NEUROINFORMATICS 2021. SCI, vol. 1008, pp. 99–108. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-91581-0_14

    Chapter  Google Scholar 

  22. Berman, A., James, V.: Kinetic imaginations: exploring the possibilities of combining AI and dance. In: Proceedings of the 24th International Joint Conference on Artificial Intelligence, pp. 2431–2437 (2015)

    Google Scholar 

  23. Deng, L.Q., Leung, H., Gu, N.J., Yang, Y.: Real-time mocap dance recognition for an interactive dancing game. Comput. Animation Virtual Worlds 22(2–3), 229–237 (2011). https://doi.org/10.1002/cav.397

    Article  Google Scholar 

  24. Ho, E.S.L., Chan, J.C.P., Komura, T., Leung, H.: Interactive partner control in close interactions for real-time applications. ACM Trans. Multimed. Comput. Commun. Appl. 9(3), 19 (2013). https://doi.org/10.1145/2487268.2487274

    Article  Google Scholar 

  25. Holldampf, J., Peer, A., Buss, M.: Virtual partner for a haptic interaction task. In: Ritter, H., Sagerer, G., Dillmann, R., Buss, M. (eds.) Human Centered Robot Systems. COSMOS, vol. 6, pp. 183–191. Springer, Heidelberg (2009). ISBN: 978-3-642-10402-2. https://doi.org/10.1007/978-3-642-10403-9_19

  26. Kirakosian, S., Maravelakis, E., Mania, K., and IEEE: Immersive simulation and training of person-to-3D character dance in real-time, pp. 170–173 (2019). ISBN: 978-1-7281-4540-2

    Google Scholar 

  27. Mousas, C.: Performance-driven dance motion control of a virtual partner character, pp. 57–64. IEEE (2018). ISBN: 978-1-5386-3365-6

    Google Scholar 

  28. Senecal, S., Nijdam, N.A., Aristidou, A., Magnenat-Thalmann, N.: Salsa dance learning evaluation and motion analysis in gamified virtual reality environment. Multimedia Tools Appl. 79(33–34), 24621–24643 (2020). https://doi.org/10.1007/s11042-020-09192-y

    Article  Google Scholar 

  29. Tamborini, R., et al.: The effect of behavioral synchrony with black or white virtual agents on outgroup trust. Comput. Hum. Behav. 83, 176–183 (2018). https://doi.org/10.1016/j.chb.2018.01.037

    Article  Google Scholar 

  30. Tsampounaris, G., El Raheb, K., Katifori, V., Ioannidis, Y.: Exploring visualizations in real-time motion capture for dance education. Association for Computing Machinery (2016). https://doi.org/10.1145/3003733.3003811

  31. Yokoyama, R., Sugiura, M., Yamamoto, Y., Nejad, K.K., Kawashima, R.: Neural bases of the adaptive mechanisms associated with reciprocal partner choice. Neuroimage 145, 74–81 (2017). https://doi.org/10.1016/j.neuroimage.2016.09.052

    Article  Google Scholar 

  32. Samsonovich, A.V., Ascoli, G.A.: Augmenting weak semantic cognitive maps with an “abstractness” dimension. Comput. Intell. Neurosci. 2013, 308176 (2013). https://doi.org/10.1155/2013/308176

    Article  Google Scholar 

  33. Samsonovich, A.V., Ascoli, G.A.: The conscious self: ontology, epistemology and the mirror quest. Cortex 41(5), 621–636 (2005). https://doi.org/10.1016/S0010-9452(08)70280-6

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the Ministry of Science and Higher Education of the Russian Federation, state assignment project No. 0723-2020-0036.

Author information

Authors and Affiliations

  1. National Research Nuclear University MEPhI, Kashirskoe Hwy 31, Moscow, 115409, Russia

    Andrey I. Kuzmin, Denis A. Semyonov & Alexei V. Samsonovich

Authors
  1. Andrey I. Kuzmin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Denis A. Semyonov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexei V. Samsonovich
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexei V. Samsonovich .

Editor information

Editors and Affiliations

  1. MEPhI, National Research Nuclear University, Moscow, Russia

    Valentin V. Klimov

  2. Boston Consulting Group, Seattle, WA, USA

    David J. Kelley

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kuzmin, A.I., Semyonov, D.A., Samsonovich, A.V. (2022). Classification and Generation of Virtual Dancer Social Behaviors Based on Deep Learning in a Simple Virtual Environment Paradigm. In: Klimov, V.V., Kelley, D.J. (eds) Biologically Inspired Cognitive Architectures 2021. BICA 2021. Studies in Computational Intelligence, vol 1032. Springer, Cham. https://doi.org/10.1007/978-3-030-96993-6_23

Download citation

Publish with us

Policies and ethics

Search

Navigation