SpringerLink
Log in

Assessing the Social Agency of Pedagogical Agents in Adaptive Training Systems

  • Conference paper
  • First Online:
Adaptive Instructional Systems (HCII 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13332))

Included in the following conference series:

  • 719 Accesses

Abstract

Pedagogical agents (PAs) could be used to mimic a one-on-one human tutoring experience in adaptive training systems. Social agency theory (SAT) is one perspective that describes how humans learn from PAs. However, there are no measures to test the components of SAT and their effects on learning. Therefore, we discuss the development of a subjective measure, the Social Agency Theory Questionnaire (SATQ), to assess components of the social agency theory framework: social cues, the cooperation principle, and deep cognitive processing. Next, we present a study that investigates the effectiveness of a PA instructor who provides error-sensitive human-voiced feedback to learners in an adaptive training system (PA-Present group), compared against an instructionally equivalent text-based instructor system (PA-Absent group). We hypothesize that the PA-Present group will exhibit higher learning outcomes than the PA-Absent group. Additionally, we hypothesize that participants in the PA-Present group will rate the “instructor” higher on all three subscales of the SATQ than participants in the PA-Absent group. Data collection for this project is currently underway. Future analyses will examine between-group differences on performance and learning outcomes, and their association with subjective ratings on SATQ subscales. In addition, we will present psychometric analyses of the SATQ and proposed revisions to the scale.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Landsberg, C.R., Astwood Jr., R.S., Van Buskirk, W.L., Townsend, L.N., Steinhauser, N.B., Mercado, A.D.: Review of adaptive training system techniques. Mil. Psychol. 24(2), 96–113 (2012)

    Article  Google Scholar 

  2. Landsberg, C.R., Mercado, A., Van Buskirk, W.L., Lineberry, M., Steinhauser, N.: Evaluation of an adaptive training system for submarine periscope operations. In: Proceedings of the Human Factors and Ergonomics Society 56th Annual Meeting, pp. 2422–2426. SAGE Publications, Los Angeles (2012)

    Google Scholar 

  3. Marraffino, M.D., Johnson, C.I., Whitmer, D.E., Steinhauser, N.B., Clement, A.: Advise when ready for game plan: adaptive training for JTACs. In: Proceedings of the Interservice/Industry, Training, Simulation, and Education Conference (I/ITSEC), Orlando (2019)

    Google Scholar 

  4. Wickens, C.D., Hutchins, S., Carolan, T., Cumming, J.: Effectiveness of part-task training and increasing-difficulty training strategies: a meta-analysis approach. Hum. Factors 55(2), 461–470 (2013). https://doi.org/10.1177/0018720812451994

    Article  Google Scholar 

  5. Bloom, B.S.: The 2 sigma problem: the search for methods of group instruction as effective as one-to-one tutoring. Educ. Res. 13, 4–16 (1984)

    Article  Google Scholar 

  6. VanLehn, K.: The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educ. Psychol. 46(4), 197–221 (2011)

    Article  Google Scholar 

  7. Buff, W.L., Campbell, G.E.: What to do or what not to do?: identifying the content of effective feedback. In: Proceedings of the 46th Annual Meeting of the Human Factors and Ergonomics Society, pp. 2074–2078. HFES, Santa Monica (2002)

    Google Scholar 

  8. Landsberg, C.R., Bailey, S., Van Buskirk, W.L., Gonzalez-Holland, E., Johnson, C.I.: Designing effective feedback in adaptive training systems. In: Interservice/Industry, Training, Simulation, and Education Conference (I/ITSEC), Orlando (2016)

    Google Scholar 

  9. Veletsianos, G., Russell, G.S.: Pedagogical agents. In: Spector, J., Merrill, M., Elen, J., Bishop, M. (eds.) Handbook of Research on Educational Communications and Technology, pp. 759–769. Springer, New York (2014). https://doi.org/10.1007/978-1-4616-3185-5_61

    Chapter  Google Scholar 

  10. Martha, A.S.D., Santoso, H.B.: The design and impact of the pedagogical agent: a systematic literature review. J. Educat. Online 16(1) (2019)

    Google Scholar 

  11. Backlund, P., Engstrom, H., Hammar, C., Johannesson, M., Lebram, M.: SIDH–a game based firefighter training simulation. In: 2007 11th International Conference Information Visualization (IV 2007), pp. 899–907. IEEE (2007)

    Google Scholar 

  12. Halverson, R., Blakesley, C., Figueiredo-Brown, R.: Video game design as a model for professional learning. In: Learning to Play: Exploring the Future of Education with Video Games, pp. 9–28 (2011)

    Google Scholar 

  13. Muntean, C.H., Andrews, J., Muntean, G.M.: Final frontier: an educational game on solar system concepts acquisition for primary schools. In: 2017 IEEE 17th International Conference on Advanced Learning Technologies (ICALT), pp. 335–337. IEEE (2017)

    Google Scholar 

  14. Lankoski, P., Björk, S.: Gameplay design patterns for believable non-player characters. In: Akira, B. (ed.) Situated Play: Proceedings of the 2007 Digital Games Research Association Conference, pp. 416–423. The University of Tokyo, Tokyo (2007)

    Google Scholar 

  15. Riedl, M., Lane, H.C., Hill, R., Swartout, W.: Automated story direction and intelligent tutoring: towards a unifying architecture. U.S. Army Research, Development, and Engineering Command (2006). https://apps.dtic.mil/dtic/tr/fulltext/u2/a459187.pdf

  16. Warpefelt, H., Johansson, M., Verhagen, H.: Analyzing the believability of game character behavior using the game agent matrix. In: DiGRA Conference, pp. 1–11 (2013)

    Google Scholar 

  17. Mitrovic, A., Suraweera, P.: Evaluating an animated pedagogical agent. In: Gauthier, G., Frasson, C., VanLehn, K. (eds.) ITS 2000. LNCS, vol. 1839, pp. 72–82. Springer, Berlin (2000). https://doi.org/10.1007/3-540-45108-0_11

    Chapter  Google Scholar 

  18. Moreno, R., Mayer, R.E., Spires, H.A., Lester, J.C.: The case for social agency in computer based teaching: do students learn more deeply when they interact with animated pedagogical agents? Cogn. Instr. 19(2), 177–213 (2001)

    Article  Google Scholar 

  19. Zakharov, K., Mitrovic, A., Johnston, L.: Towards emotionally-intelligent pedagogical agents. In: Woolf, B.P., Aïmeur, E., Nkambou, R., Lajoie, S. (eds.) ITS 2008. LNCS, vol. 5091, pp. 19–28. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-69132-7_7

    Chapter  Google Scholar 

  20. Adams, E.: Fundamentals of Game Design, 3rd edn. Pearson Education, Peachpit (2014)

    Google Scholar 

  21. Schroeder, N.L., Adesope, O.O., Gilbert, R.B.: How effective are pedagogical agents for learning? A meta-analytic review. J. Educ. Comput. Res. 49(1), 1–39 (2013)

    Article  Google Scholar 

  22. Atkinson, R.K.: Optimizing learning from examples using animated pedagogical agents. J. Educ. Psychol. 94(2), 416–427 (2002)

    Article  Google Scholar 

  23. Domagk, S.: Do pedagogical agents facilitate learner motivation and learning outcomes? J. Media Psychol. 22(2), 82–95 (2010)

    Article  Google Scholar 

  24. Wang, N., Johnson, W.L., Mayer, R.E., Rizzo, P., Shaw, E., Collins, H.: The politeness effect: pedagogical agents and learning outcomes. Int. J. Hum Comput Stud. 66(2), 98–112 (2008)

    Article  Google Scholar 

  25. Kim, Y., Baylor, A.L.: Research-based design of pedagogical agent roles: a review, progress, and recommendations. Int. J. Artif. Intell. Educ. 26(1), 160–169 (2016)

    Article  Google Scholar 

  26. Baylor, A.L., Kim, Y.: Simulating instructional roles through pedagogical agents. Int. J. Artif. Intell. Educ. 15(1), 95 (2005)

    Google Scholar 

  27. Veletsianos, G.: Contextually relevant pedagogical agents: visual appearance, stereotypes, and first impressions and their impact on learning. Comput. Educ. 55(2), 576–585 (2010)

    Article  Google Scholar 

  28. Mayer, R.E., Sobko, K., Mautone, P.D.: Social cues in multimedia learning: role of speaker’s voice. J. Educ. Psychol. 95(2), 419–425 (2003). https://doi.org/10.1037/0022-0663.95.2.419

    Article  Google Scholar 

  29. Mayer, R.E., DaPra, C.S.: An embodiment effect in computer-based learning with animated pedagogical agents. J. Exp. Psychol. Appl. 18(3), 239–252 (2012). https://doi.org/10.1037/a0028616

    Article  Google Scholar 

  30. Grice, H.P.: Logic and conversation. In: Cole, P., Morgan, J. (eds.) Syntax and Semantics, vol. 3, pp. 41–58. Academic Press (1975). https://doi.org/10.1163/9789004368811_003

  31. Lin, L., Atkinson, R.K., Christopherson, R.M., Joseph, S.S., Harrison, C.J.: Animated agents and learning: does the type of verbal feedback they provide matter? Comput. Educ. 67, 239–249 (2013). https://doi.org/10.1016/j.compedu.2013.04.017

    Article  Google Scholar 

  32. Lester, J.C., Converse, S.A., Kahler, S.E., Barlow, S.T., Stone, B.A., Bhogal, R.S.: The persona effect: affective impact of animated pedagogical agents. In: Pemberton, S. (ed.) Proceedings of CHI 1997: Human Factors in Computing Systems, pp. 259–266. ACM Press (1997). https://doi.org/10.1145/258549.258797

  33. Arner, T., McCarthy, K.S., McNamara, D.S.: iSTART stairstepper – using comprehension strategy to game the test. Computers 10(4), 48 (2021). https://doi.org/10.3390/computers10040048

    Article  Google Scholar 

  34. Lester, J.C., Stone, B.A.: Increasing believability in animated pedagogical agents. In: Proceedings of the First International Conference on Autonomous Agents, pp. 16–21. ACM Press (1997)

    Google Scholar 

  35. van Mulken, S., Andre, E., Müller, J.: The persona effect: how substantial is it? In: Johnson, H., Nigay, L., Roast, C. (eds.) People and Computers XIII, pp. 53–66. Springer, London (1998). https://doi.org/10.1007/978-1-4471-3605-7_4

    Chapter  Google Scholar 

  36. Walker, J.H., Sproull, L., Subramani, R.: Using a human face in an interface. In: Adelson, B., Dumais, S., Olson, J. (eds.) Proceedings of CHI 1994: Human Factors in Computing Systems, pp. 85–91. ACM Press (1994). https://doi.org/10.1145/191666.191708

  37. Schroeder, N.L., Gotch, C.M.: Persisting issues in pedagogical agent research. J. Educ. Comput. Res. 53(2), 183–204 (2015). https://doi.org/10.1177/0735633115597625

    Article  Google Scholar 

  38. Louwerse, M.M., Graesser, A.C., Lu, S., Mitchell, H.H.: Social cues in animated conversational agents. Appl. Cogn. Psychol. 19(6), 693–704 (2005)

    Article  Google Scholar 

  39. Atkinson, R.K., Mayer, R.E., Merrill, M.M.: Fostering social agency in multimedia learning: examining the impact of an animated agent’s voice. Contemp. Educ. Psychol. 30(1), 117–139 (2005)

    Article  Google Scholar 

  40. Moon, J., Ryu, J.: The effects of social and cognitive cues on learning comprehension, eye-gaze pattern, and cognitive load in video instruction. J. Comput. High. Educ. 33(1), 39–63 (2020). https://doi.org/10.1007/s12528-020-09255-x

    Article  Google Scholar 

  41. Baylor, A., Ryu, J.: The API (agent persona instrument) for assessing pedagogical agent persona. In: EdMedia+ Innovate Learning, pp. 448–451. Association for the Advancement of Computing in Education (AACE) (2003)

    Google Scholar 

  42. Schroeder, N.L., Romine, W.L., Craig, S.D.: Measuring pedagogical agent persona and the influence of agent persona on learning. Comput. Educ. 109, 176–186 (2017)

    Article  Google Scholar 

  43. Johnson, W.L., Rickel, J.W., Lester, J.C.: Animated pedagogical agents: face-to-face interaction in interactive learning environments. Int. J. Artif. Intell. Educ. 11(1), 47–78 (2000)

    Google Scholar 

  44. Martin, A.J., Jackson, S.A.: Brief approaches to assessing task absorption and enhanced subjective experience: examining ‘short’ and ‘core’ flow in diverse performance domains. Motiv. Emot. 32(3), 141–157 (2008)

    Article  Google Scholar 

  45. Brockmeyer, J.H., Fox, C.M., Curtiss, K.A., McBroom, E., Burkhart, K.M., Pidruzny, J.N.: The development of the game engagement questionnaire: a measure of engagement in video game playing. J. Exp. Soc. Psychol. 45(4), 624–634 (2009)

    Article  Google Scholar 

  46. Schroeder, N.L., Yang, F., Banerjee, T., Romine, W.L., Craig, S.D.: The influence of learners’ perceptions of virtual humans on learning transfer. Comput. Educ. 126, 170–182 (2018)

    Article  Google Scholar 

  47. Nass, C., Steuer, J., Tauber, E.R.: Computers are social actors. In: Adelson, B., Dumais, S., Olson, J. (eds.) CHI 1994: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, vol. 12, pp. 72–78. Association for Computing Machinery, New York (1994)

    Google Scholar 

Download references

Acknowledgments

We gratefully acknowledge Dr. Jason Hochreiter, Mr. Rob Veira, and Ms. Rebecca Pharmer for their assistance with testbed development. This work was funded under the Naval Innovative Science and Engineering program established by the National Defense Authorization Act, Section 219. Due to difficulties with data collection brought on by the COVID-19 pandemic, data collection for this study is ongoing at time of writing. Data relevant to this project are to be presented virtually at the 2022 HCII conference. Interested readers are encouraged to contact the authors for recent developments and analyses with the SATQ. Presentation of this material does not constitute or imply its endorsement, recommendation, or favoring by the U.S. Navy or the Department of Defense (DoD). The views expressed herein are those of the authors and do not necessarily reflect the official position of the Department of Defense or its components. NAWCTSD Public Release 22-ORL016 Distribution Statement A – Approved for public release; distribution is unlimited.

Author information

Authors and Affiliations

  1. Naval Air Warfare Center Training Systems Division, Orlando, FL, 32826, USA

    Bradford L. Schroeder & Wendi L. Van Buskirk

  2. StraCon Services Group, Fort Worth, TX, 76109, USA

    Nicholas W. Fraulini

  3. Morehead State University, Morehead, KY, 40351, USA

    Reganne M. Miller

Authors
  1. Bradford L. Schroeder
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nicholas W. Fraulini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wendi L. Van Buskirk
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Reganne M. Miller
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bradford L. Schroeder .

Editor information

Editors and Affiliations

  1. Soar Technology, Inc., Orlando, FL, USA

    Robert A. Sottilare

  2. Fraunhofer FKIE, Wachtberg, Germany

    Jessica Schwarz

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Schroeder, B.L., Fraulini, N.W., Van Buskirk, W.L., Miller, R.M. (2022). Assessing the Social Agency of Pedagogical Agents in Adaptive Training Systems. In: Sottilare, R.A., Schwarz, J. (eds) Adaptive Instructional Systems. HCII 2022. Lecture Notes in Computer Science, vol 13332. Springer, Cham. https://doi.org/10.1007/978-3-031-05887-5_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-05887-5_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-05886-8

  • Online ISBN: 978-3-031-05887-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation