SpringerLink
Log in

Using Theory of Mind to Assess Users’ Sense of Agency in Social Chatbots

  • Conference paper
  • First Online:
Chatbot Research and Design (CONVERSATIONS 2019)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11970))

Included in the following conference series:

Abstract

The technological advancements in the field of chatbot research is booming. Despite this, it is still difficult to assess which social characteristics a chatbot needs to have for the user to interact with it as if it had a mind of its own. Review studies have highlighted that the main cause is the low number of research papers dedicated to this question, and the lack of a consistent protocol within the papers that do address it. In the current paper, we suggest the use of a Theory of Mind task to measure the implicit social behaviour users exhibit towards a text-based chatbot. We present preliminary findings suggesting that participants adapt towards this basic chatbot significantly more than when they conduct the task alone (p < .017). This task is quick to administer and does not require a second chatbot for comparison, making it an efficient universal task. With it, a database could be built with scores of all existing chatbots, allowing fast and efficient meta-analyses to discover which characteristics make the chatbot appear more ‘human’.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Weizenbaum, J.: ELIZA—a computer program for the study of natural language communication between man and machine. Commun. ACM 26, 23–28 (1983). https://doi.org/10.1145/357980.357991

    Article  Google Scholar 

  2. Wallace, R.: The Elements of AIML Style (2003)

    Google Scholar 

  3. Comendador, B.E.V., Francisco, B.M.B., Medenilla, J.S., Nacion, S.M.T., Serac, T.B.E.: Pharmabot: a pediatric generic medicine consultant chatbot. J. Autom. Control Eng. 3, 137–140 (2015). https://doi.org/10.12720/joace.3.2.137-140

    Article  Google Scholar 

  4. Ciechanowski, L., Przegalinska, A., Magnuski, M., Gloor, P.: In the shades of the uncanny valley: an experimental study of human – chatbot interaction. Future Gener. Comput. Syst. 92, 539–548 (2019). https://doi.org/10.1016/j.future.2018.01.055

    Article  Google Scholar 

  5. Neururer, M., Schlögl, S., Brinkschulte, L., Groth, A.: Perceptions on authenticity in chat bots. Multimodal Technol. Interact. 2, 60 (2018). https://doi.org/10.3390/mti2030060

    Article  Google Scholar 

  6. Radziwill, N., Benton, M.: Evaluating Quality of Chatbots and Intelligent Conversational Agents (2016)

    Google Scholar 

  7. Chaves, A.P., Gerosa, M.: How should my chatbot interact? 1–44 (2018)

    Google Scholar 

  8. Morrissey, K., Kirakowski, J.: ‘Realness’ in chatbots: establishing quantifiable criteria. In: Kurosu, M. (ed.) HCI 2013. LNCS, vol. 8007, pp. 87–96. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39330-3_10

    Chapter  Google Scholar 

  9. Frith, C.D., Frith, U.: Mechanisms of social cognition. Ann. Rev. Psychol. 63, 287–313 (2012). https://doi.org/10.1146/annurev-psych-120710-100449

    Article  Google Scholar 

  10. Premack, D., Woodruff, G.: Does the chimpanzee have a theory of mind? Behav. Brain Sci. 4, 515–526 (1978)

    Article  Google Scholar 

  11. Vanlangendonck, F., et al.: An early influence of common ground during speech planning. 3798 (2016). https://doi.org/10.1080/23273798.2016.1148747

    Article  Google Scholar 

  12. Vanlangendonck, F., Willems, R.M., Hagoort, P.: Taking common ground into account: specifying the role of the mentalizing network in communicative language production. PLoS ONE 13, e0202943 (2018). https://doi.org/10.1371/journal.pone.0202943

    Article  Google Scholar 

  13. Van Der Kallen, T., Communicatiewetenschap, M., Nijmegen, R.U.: Chatbots: kunnen menselijk sprekende computers beter tegen een foutje? [Chatbots: Are human-like talking computers more resilient to mistakes?]

    Google Scholar 

  14. Bates, D., Maechler, M., Bolker, B.: lme4: linear mixed-effects models using S4 classes (2011). R package version 0.999375-42 (2012)

    Google Scholar 

  15. R Core Development Team: R: a language and environment for statistical computing, Vienna, Austria (2011)

    Google Scholar 

  16. Barr, D.J., Levy, R., Scheepers, C., Tily, H.J.: Random effects structure for confirmatory hypothesis testing: keep it maximal. J. Mem. Lang. 68, 255–278 (2013). https://doi.org/10.1016/j.jml.2012.11.001

    Article  Google Scholar 

  17. Keysar, B., Barr, D.J., Balin, J.A., Brauner, J.S.: Taking perspective in conversation: the role of mutual knowledge in comprehension. Psychol. Sci. 11, 32–38 (2000)

    Article  Google Scholar 

  18. Yoon, S.O., Koh, S., Brown-Schmidt, S.: Influence of perspective and goals on reference production in conversation. Psychon. Bull. Rev. 19, 699–707 (2012). https://doi.org/10.3758/s13423-012-0262-6

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Radboud Universiteit Nijmegen, Nijmegen, The Netherlands

    Evelien Heyselaar & Tibor Bosse

Authors
  1. Evelien Heyselaar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tibor Bosse
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Evelien Heyselaar .

Editor information

Editors and Affiliations

  1. SINTEF, Oslo, Norway

    Asbjørn Følstad

  2. University of Amsterdam, Amsterdam, The Netherlands

    Theo Araujo

  3. CERTH-ITI, Thessaloniki, Greece

    Symeon Papadopoulos

  4. University of Leicester, Leicester, UK

    Effie Lai-Chong Law

  5. University of Agder, Grimstad, Norway

    Ole-Christoffer Granmo

  6. University of Edinburgh, Edinburgh, UK

    Ewa Luger

  7. University of Oslo, Oslo, Norway

    Petter Bae Brandtzaeg

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Heyselaar, E., Bosse, T. (2020). Using Theory of Mind to Assess Users’ Sense of Agency in Social Chatbots. In: Følstad, A., et al. Chatbot Research and Design. CONVERSATIONS 2019. Lecture Notes in Computer Science(), vol 11970. Springer, Cham. https://doi.org/10.1007/978-3-030-39540-7_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-39540-7_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-39539-1

  • Online ISBN: 978-3-030-39540-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation