SpringerLink
Log in

The Effectiveness of AI Generated, On-Demand Assistance Within Online Learning Platforms

  • Conference paper
  • First Online:
Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky (AIED 2024)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 2151))

Included in the following conference series:

  • 129 Accesses

Abstract

Since GPT-4’s release it has shown novel abilities in a variety of domains. This paper explores the use of LLM-generated explanations as on-demand assistance for problems within the ASSISTments platform. In particular, we are studying whether GPT-generated explanations are better than nothing on problems that have no supports and whether GPT-generated explanations are as good as or better than teacher-authored explanations. This study contributes to existing literature since as of yet, there are no studies on the scale of ASSISTments evaluating the effectiveness of GPT support in education. Should GPT explanations prove effective then we plan to continue develo** and evaluating explanations, hints, and other supports with GPT within ASSISTments.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free ship** worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://openai.com/gpt-4.

  2. 2.

    https://ai.meta.com/llama/.

  3. 3.

    https://bard.google.com/.

  4. 4.

    https://assistments.org.

  5. 5.

    https://osf.io/b7p6v/?view_only=03b19d094a9440a0ae5df4177907a1d1.

References

  1. Adams, D.M., et al.: Using erroneous examples to improve mathematics learning with a web-based tutoring system. Comput. Hum. Behav. 36, 401–411 (2014)

    Article  Google Scholar 

  2. Feng, M., Heffernan, N.T.: Informing teachers live about student learning: reporting in the assistment system. Technol. Instr. Cogn. Learn. 3(1/2), 63 (2006)

    Google Scholar 

  3. Gurung, A., et al.: How common are common wrong answers? Crowdsourcing remediation at scale. In: Proceedings of the Tenth ACM Conference on Learning @ Scale, L@S 2023, pp. 70–80. Association for Computing Machinery, New York (2023). https://doi.org/10.1145/3573051.3593390

  4. Heffernan, N.T., Heffernan, C.L.: The ASSISTments ecosystem: building a platform that brings scientists and teachers together for minimally invasive research on human learning and teaching. Int. J. Artif. Intell. Educ. 24(4), 470–497 (2014)

    Article  MathSciNet  Google Scholar 

  5. Imani, S., Du, L., Shrivastava, H.: MathPrompter: mathematical reasoning using large language models (2023)

    Google Scholar 

  6. Khanmigo (2023). https://khanmigo.ai/

  7. Markel, J.M., Opferman, S.G., Landay, J.A., Piech, C.: GPTeach: interactive ta training with GPT-based students. In: Proceedings of the Tenth ACM Conference on Learning @ Scale, L@S 2023, pp. 226–236. Association for Computing Machinery, New York (2023). https://doi.org/10.1145/3573051.3593393

  8. McLaren, B.M., van Gog, T., Ganoe, C., Karabinos, M., Yaron, D.: The efficiency of worked examples compared to erroneous examples, tutored problem solving, and problem solving in computer-based learning environments. Comput. Hum. Behav. 55, 87–99 (2016)

    Article  Google Scholar 

  9. Pardos, Z.A., Bhandari, S.: Learning gain differences between ChatGPT and human tutor generated algebra hints (2023)

    Google Scholar 

  10. Patikorn, T., Heffernan, N.T.: Effectiveness of crowd-sourcing on-demand assistance from teachers in online learning platforms. In: Proceedings of the Seventh ACM Conference on Learning @ Scale, L@S 2020, pp. 115–124. Association for Computing Machinery, New York (2020). https://doi.org/10.1145/3386527.3405912

  11. Prihar, E., et al.: Comparing different approaches to generating mathematics explanations using large language models. In: Wang, N., Rebolledo-Mendez, G., Dimitrova, V., Matsuda, N., Santos, O.C. (eds.) AIED 2023. CCIS, vol. 1831, pp. 290–295. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-36336-8_45

    Chapter  Google Scholar 

  12. Prihar, E., Patikorn, T., Botelho, A., Sales, A., Heffernan, N.: Toward personalizing students’ education with crowdsourced tutoring. In: Proceedings of the Eighth ACM Conference on Learning @ Scale, L@S 2021, pp. 37–45. Association for Computing Machinery, New York (2021). https://doi.org/10.1145/3430895.3460130

  13. Razzaq, L.M., Heffernan, N.T.: To tutor or not to tutor: that is the question. In: Dimitrova, V. (ed.) AIED, pp. 457–464. IOS Press (2009)

    Google Scholar 

  14. Schnepper, L.C., McCoy, L.P.: Analysis of misconceptions in high school mathematics. Netw. Online J. Teach. Res. 15(1), 625–625 (2013). https://newprairiepress.org/networks/vol15/iss1/7/

  15. VanLehn, K., Siler, S., Murray, C., Yamauchi, T., Baggett, W.B.: Why do only some events cause learning during human tutoring? Cogn. Instr. 21(3), 209–249 (2003). https://doi.org/10.1207/S1532690XCI2103_01

    Article  Google Scholar 

  16. Wei, J., et al.: Chain-of-thought prompting elicits reasoning in large language models. In: Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., Oh, A. (eds.) Advances in Neural Information Processing Systems, vol. 35, pp. 24824–24837. Curran Associates, Inc. (2022). https://proceedings.neurips.cc/paper_files/paper/2022/file/9d5609613524ecf4f15af0f7b31abca4-Paper-Conference.pdf

  17. Whitehill, J., Seltzer, M.: A crowdsourcing approach to collecting tutorial videos–toward personalized learning-at-scale. In: Proceedings of the Fourth ACM Conference on Learning@ Scale, pp. 157–160 (2017)

    Google Scholar 

  18. Williams, J.J., et al.: AXIS: generating explanations at scale with learnersourcing and machine learning. In: Proceedings of the Third ACM Conference on Learning@ Scale, pp. 379–388 (2016)

    Google Scholar 

  19. **ao, C., Xu, S.X., Zhang, K., Wang, Y., **a, L.: Evaluating reading comprehension exercises generated by LLMs: a showcase of ChatGPT in education applications. In: Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), pp. 610–625 (2023)

    Google Scholar 

Download references

Acknowledgements

We would like to thank NSF (2118725, 2118904, 1950683, 1917808, 1931523, 1940236, 1917713, 1903304, 1822830, 1759229, 1724889, 1636782, & 1535428), IES (R305N210049, R305D210031, R305A170137, R305A170243, R305A180401, R305A120125, & R305R220012), GAANN (P200A120238, P200A180088, P200A150306, & P200A150306), EIR (U411B190024 S411B210024, & S411B220024), ONR (N00014-18-1-2768), NIH (via a SBIR R44GM146483), Schmidt Futures, BMGF, CZI, Arnold, Hewlett and a $180,000 anonymous donation. None of the opinions expressed here are those of the funders.

Author information

Authors and Affiliations

  1. Worcester Polytechnic Institute, Worcester, MA, 01609, USA

    Aaron Haim, Eamon Worden & Neil T. Heffernan

Authors
  1. Aaron Haim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eamon Worden
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Neil T. Heffernan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Aaron Haim , Eamon Worden or Neil T. Heffernan .

Editor information

Editors and Affiliations

  1. University of Memphis, Memphis, TN, USA

    Andrew M. Olney

  2. University of Duisburg-Essen, Duisburg, Germany

    Irene-Angelica Chounta

  3. **an University, Guangzhou, China

    Zitao Liu

  4. UNED, Madrid, Spain

    Olga C. Santos

  5. Universidade Federal de Alagoas, Maceio, Brazil

    Ig Ibert Bittencourt

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Haim, A., Worden, E., Heffernan, N.T. (2024). The Effectiveness of AI Generated, On-Demand Assistance Within Online Learning Platforms. In: Olney, A.M., Chounta, IA., Liu, Z., Santos, O.C., Bittencourt, I.I. (eds) Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky. AIED 2024. Communications in Computer and Information Science, vol 2151. Springer, Cham. https://doi.org/10.1007/978-3-031-64312-5_45

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-64312-5_45

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-64311-8

  • Online ISBN: 978-3-031-64312-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation