Abstract
This study aimed to investigate the effectiveness of using AI-assisted game-based learning on science learning outcomes, intrinsic motivation, cognitive load, and learning behavior. A total of 202 seventh graders were recruited and randomly assigned to the following three groups: (1) Game only (N = 70), (2) GameGPT (N = 63), and (3) GameGPT_examples (N = 69). The experimental groups received game-based learning with the assistance of ChatGPT with or without examples, while the control group received only game-based learning. The results showed that students in the GameGPT_examples group significantly outperformed those in the Game only group. Students in the GameGPT and GameGPT_examples groups reported significantly higher perceived competence than those in the Game only group. Furthermore, students in the Game only group reported a greater mental burden than those in the GameGPT_examples and GameGPT groups. The findings from learning behavioral analytics and interviews suggest that AI-assisted game-based learning can enhance students’ intrinsic motivation, reduce cognitive load, and promote effective learning behavior in science learning. This study has important implications for the design and implementation of AI in game-based learning environments that aim to improve students’ learning outcomes and motivation.
Similar content being viewed by others
Data availability
The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.
References
Abdullah, M., Madain, A., & Jararweh, Y. (2022). ChatGPT: Fundamentals, applications and social impacts. Ninth International Conference on Social Networks Analysis, Management and Security (SNAMS).
Alam, A. (2021). Should robots replace teachers? Mobilisation of AI and learning analytics in education. International Conference on Advances in Computing, Communication, and Control (ICAC3).
Aleven, V., Roll, I., McLaren, B. M., & Koedinger, K. R. (2016). Help helps, but only so much: Research on help seeking with intelligent tutoring systems. International Journal of Artificial Intelligence in Education, 26(1), 205–223. https://doi.org/10.1007/s40593-015-0089-1
Alonso-Fernández, C., Martínez-Ortiz, I., Caballero, R., Freire, M., & Fernández-Manjón, B. (2020). Predicting students’ knowledge after playing a serious game based on learning analytics data: A case study. Journal of Computer Assisted Learning, 36(3), 350–358.
Bakeman, R., & Gottman, J. M. (1997). Observing interaction: An introduction to sequential analysis (2nd ed.). Cambridge University Press.
Berends, I. E., & van Lieshout, E. C. D. M. (2009). The effect of illustrations in arithmetic problem-solving: Effects of increased cognitive load. Learning and Instruction, 19, 345–353.
Block, R. A., Handcock, P. A., & Zakay, D. (2010). How cognitive load affects duration judgments: A meta-analytic review. Acta Psychologica, 134, 330–343.
Cai, Z., Mao, P., Wang, D., He, J., Chen, X., & Fan, X. (2022). Effects of scaffolding in digital game-based learning on student’s achievement: A three-level meta-analysis. Educational Psychology Review, 34(2), 537–574. https://doi.org/10.1007/s10648-021-09655-0
Chen, C. H., & Law, V. (2016). Scaffolding individual and collaborative game-based learning in learning performance and intrinsic motivation. Computers in Human Behavior, 55, 1201–1212. https://doi.org/10.1016/j.chb.2015.03.010
Chen, X., Zou, D., **. Educational Technology & Society, 24(1), 205–222.
Chu, S. T., Hwang, G. J., Chien, S. Y., & Chang, S. C. (2023). Incorporating teacher intelligence into digital games: An expert system-guided self-regulated learning approach to promoting EFL students’ performance in digital gaming contexts. British Journal of Educational Technology, 54(2), 534–553.
Clark, D. B., Tanner-Smith, E. E., & Killingsworth, S. S. (2016). Digital games, design, and learning: A systematic review and meta-analysis. Review of Educational Research, 86(1), 79–122.
Connolly, T. M., Boyle, E. A., MacArthur, E., Hainey, T., & Boyle, J. M. (2012). A systematic literature review of empirical evidence on computer games and serious games. Computers & Education, 59, 661–686. https://doi.org/10.1016/j.compedu.2012.03.004
Cox, A. M. (2021). Exploring the impact of artificial intelligence and robots on higher education through literature-based design fictions. International Journal of Educational Technology in Higher Education, 18(1), 3. https://doi.org/10.1186/s41239-020-00237-8
Dale, R. (2021). GPT-3: What’s it good for? Natural Language Engineering, 27(1), 113–118.
Deci, E. L., & Ryan, R. M. (2000). The “what” and “why” of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry, 11(4), 227–268. https://doi.org/10.1207/S15327965PLI1104_01
Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). BERT: Pre-training of deep bidirectional transformers for language understanding. ar**v, 1810.04805. https://doi.org/10.48550/ar**v.1810.04805
Dibitonto, M., Leszczynska, K., Tazzi, F., & Medaglia, C. M. (2018). Chatbot in a campus environment: design of LiSA, a virtual assistant to help students in their university life. Human-Computer Interaction. Interaction Technologies: 20th International Conference.
Dupont, W. D., & Plummer, W. D. (1990). Power and sample size calculations: A review and computer program. Controlled Clinical Trials, 11(2), 116–128.
Emerson, A., Cloude, E. B., Azevedo, R., & Lester, J. (2020). Multimodal learning analytics for game-based learning. British Journal of Educational Technology, 51(5), 1505–1526.
Emerson, A., Min, W., Azevedo, R., & Lester, J. (2023). Early prediction of student knowledge in game-based learning with distributed representations of assessment questions. British Journal of Educational Technology, 54(1), 40–57. https://doi.org/10.1111/bjet.13281
Erhel, S., & Jamet, E. (2013). Digital game-based learning: Impact of instructions and feedback on motivation and learning effectiveness. Computers & Education, 67, 156–167.
Følstad, A., & Brandtzæg, P. B. (2017). Chatbots and the new world of HCI. Interactions, 24, 38–42.
Fryer, L. K., Nakao, K., & Thompson, A. (2019). Chatbot learning partners: Connecting learning experiences, interest and competence. Computers in Human Behavior, 93, 279–289. https://doi.org/10.1016/j.chb.2018.12.023
Haleem, A., Javaid, M., & Singh, R. P. (2022). An era of ChatGPT as a significant futuristic support tool: A study on features, abilities, and challenges. BenchCouncil Transactions on Benchmarks, Standards and Evaluations, 2(4). https://doi.org/10.1016/j.tbench.2023.100089
Hamari, J., Shernoff, D. J., Rowe, E., Coller, B., Asbell-Clarke, J., & Edwards, T. (2016). Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning. Computers in Human Behavior, 54, 170–179. https://doi.org/10.1016/j.chb.2015.07.045
Hao, Y., Lee, K. S., Chen, S.-T., & Sim, S. C. (2019). An evaluative study of a mobile application for middle school students struggling with English vocabulary learning. Computers in Human Behavior, 95, 208–216. https://doi.org/10.1016/j.chb.2018.10.013
Hassani, H., & Silva, E. S. (2023). The role of ChatGPT in data science: How AI-assisted conversational interfaces are revolutionizing the field. Big Data and Cognitive Computing, 7(2), 62. https://www.mdpi.com/2504-2289/7/2/62. Accessed 4 Feb 2024
Ipsen, A. (2023). GPT Base, GPT-3.5 Turbo & GPT-4: What's the difference? Pluralsight. Retrieved February 4, 2024, from https://www.pluralsight.com/resources/blog/data/ai-gpt-models-differences. Accessed 4 Feb 2024
Jackson, G. T., & McNamara, D. (2013). Motivation and performance in a game-based intelligent tutoring system. Journal of Educational Psychology, 105(4), 1036–1049.
Jackson, T., Boonthum-Denecke, C., & McNamara, D. (2015). Natural language processing and game-based practice in iSTART. Journal of Interactive Learning Research, 26(2), 189–208.
Jeon, J. (2022). Exploring AI chatbot affordances in the EFL classroom: young learners’ experiences and perspectives. Computer Assisted Language Learning, 1–26. https://doi.org/10.1080/09588221.2021.2021241
Ke, F. (2016). Designing and integrating purposeful learning in game play: A systematic review. Educational Technology Research & Development, 64(2), 219–244. https://doi.org/10.1007/s11423-015-9418-1
Leppink, J., Paas, F., Van der Vleuten, C. P. M., Van Gog, T., & Van Merriënboer, J. J. G. (2013). Development of an instrument for measuring different types of cognitive load. Behavior Research Methods, 45(4), 1058–1072. https://doi.org/10.3758/s13428-013-0334-1
Li, F., **, Y., Liu, W., Rawat, B. P. S., Cai, P., & Yu, H. (2019). Fine-tuning bidirectional encoder representations from transformers (BERT)–based models on large-scale electronic health record notes: An empirical study. JMIR Medical Informatics, 7(3), e14830.
Liao, C.-W., Chen, C.-H., & Shih, S.-J. (2019). The interactivity of video and collaboration for learning achievement, intrinsic motivation, cognitive load, and behavior patterns in a digital game-based learning environment. Computers & Education, 133, 43–55.
Liu, C., Hou, J., Tu, Y.-F., Wang, Y., & Hwang, G.-J. (2021). Incorporating a reflective thinking promoting mechanism into artificial intelligence-supported English writing environments. Interactive Learning Environments, 1–19. https://doi.org/10.1080/10494820.2021.2012812
Lo, C. K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13(4)
Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis (2nd ed.). SAGE Publications.
Neville, D. O., Shelton, B. E., & McInnis, B. (2009). Cybertext redux: Using digital game-based learning to teach L2 vocabulary, reading, and culture. Computer Assisted Language Learning, 22(5), 409–424. https://doi.org/10.1080/09588220903345168
Papastergiou, M. (2009). Digital game-based learning in high school computer science education: Impact on educational effectiveness and student motivation. Computers & Education, 52(1), 1–12. https://doi.org/10.1016/j.compedu.2008.06.004
Peters, H., Kyngdon, A., & Stillwell, D. (2021). Construction and validation of a game-based intelligence assessment in minecraft. Computers in Human Behavior, 119. https://doi.org/10.1016/j.chb.2021.106701
Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding by generative pre-training. Retrieved February 4 from https://www.mikecaptain.com/resources/pdf/GPT-1.pdf
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsu-pervised multitask learners. OpenAI Blog, 1(8), 9.
Rice, J. W. (2007). New media resistance: Barriers to implementation of computer video games in the classroom. Journal of Educational Multimedia and Hypermedia, 16(3), 249–261.
Rowe, E., Almeda, M. V., Asbell-Clarke, J., Scruggs, R., Baker, R., Bardar, E., & Gasca, S. (2021). Assessing implicit computational thinking in Zoombinis puzzle gameplay. Computers in Human Behavior, 120, 106707.
Shree, P. (2020). The journey of Open AI GPT models. Retrieved February 4, 2024, from https://medium.com/walmartglobaltech/the-journey-of-open-ai-gpt-models-32d95b7b7fb2. Accessed 4 Feb 2024
Shute, V. J., & Ke, F. (2012). Games, learning, and assessment. In Assessment in Game-Based Learning: Foundations, Innovations, and Perspectives (pp. 43–58). Springer.
Sottilare, R. A., Shawn Burke, C., Salas, E., Sinatra, A. M., Johnston, J. H., & Gilbert, S. B. (2018). Designing adaptive instruction for teams: A meta-analysis. International Journal of Artificial Intelligence in Education, 28, 225–264.
Tan, D. Y., & Cheah, C. W. (2021). Develo** a gamified AI-enabled online learning application to improve students’ perception of university physics. Computers and Education: Artificial Intelligence, 2,. https://doi.org/10.1016/j.caeai.2021.100032
Taub, M., Mudrick, N. V., Azevedo, R., Millar, G. C., Rowe, J., & Lester, J. (2017). Using multi-channel data with multi-level modeling to assess in-game performance during gameplay with Crystal Island. Computers in Human Behavior, 76, 641–655.
Vlachopoulos, D., & Makri, A. (2017). The effect of games and simulations on higher education: A systematic literature review. International Journal of Educational Technology in Higher Education, 14(1), 22. https://doi.org/10.1186/s41239-017-0062-1
Westera, W. (2015). Games are motivating, aren´t they? Disputing the arguments for digital game-based learning. International Journal of Serious Games, 2(2), 3–17. https://doi.org/10.17083/ijsg.v2i2.58
Wouters, P., van Nimwegen, C., van Oostendorp, H., & van der Spek, E. D. (2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105(2), 249–265. https://doi.org/10.1037/a0031311
Acknowledgements
This research study was supported by the National Science and Technology Council in Taiwan through the contract number MOST 108-2511-H-018-017-MY3. The authors would like to thank Yu-Kai Zhang for the development of the game. The authors also extend gratitude to the reviewers for their valuable feedback and insights, which significantly improved the scientific rigor of the manuscript.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of interests
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Chen, CH., Chang, CL. Effectiveness of AI-assisted game-based learning on science learning outcomes, intrinsic motivation, cognitive load, and learning behavior. Educ Inf Technol (2024). https://doi.org/10.1007/s10639-024-12553-x
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10639-024-12553-x