Abstract
The advancement of knowledge in medicine presents an important challenge when identifying gaps and deciding what content to include in a medical school curriculum and how to establish learning outcomes. Monitoring alignment between lesson objectives, the curriculum and achievement of intended outcomes can be difficult. A system that can automatically evaluate lesson objectives would be highly beneficial. We aim to assess the efficacy of using machine learning techniques to evaluate individual lesson objectives to a graduate entry allopathic medical school curriculum. The school’s curriculum objectives consist of 11 categories and 356 curriculum objectives sentences. We considered the first year courses with a total of 1888 lesson objectives. Using various word embeddings (TF-IDF, word2vec, fastText, BioBERT), we then use cosine similarity to map each lesson objective to the curriculum objectives. Cognitive levels of lesson objectives were compared against the school’s curriculum using Bloom’s Taxonomy verbs. After implementation, 319 lesson objectives from each approach were randomly sampled (sample size, 95% CL, 5% CI) to examine match with curriculum objectives and curriculum categories. BioBERT performed best with 46.71% and 80.56% match between lesson objectives and curriculum objectives, and lesson objectives and categories, respectively. Further validation by a domain expert shows 80% match (without order). Visualisation of the Bloom’s Taxonomy cognitive levels of lesson objectives and school’s curriculum objectives showed a good match. Machine learning can be used to evaluate lesson and curriculum and automatically map** lesson objectives to the medical school curriculum and analysing cognitive levels of lesson objectives.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Biggs, J.: Constructive alignment in university teaching. HERDSA Rev. High. Educ. 1, 5–22 (2014)
Chan, K.S., Zary, N.: Applications and challenges of implementing artificial intelligence in medical education: integrative review. JMIR Med. Educ. 5(1), e13930 (2019)
Das, S., Das Mandal, S.K., Basu, A.: Cognitive complexity analysis of learning-related texts: a case study on school textbooks. In: Vittorini, P., Di Mascio, T., Tarantino, L., Temperini, M., Gennari, R., De la Prieta, F. (eds.) MIS4TEL 2020. AISC, vol. 1241, pp. 74–84. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-52538-5_9
Devlin, J., Chang, M.W., Lee, K., Google, K.T., Language, A.I.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT 2019. pp. 4171–4186 (2019)
Joulin, A., Grave, É., Bojanowski, P., Mikolov, T.: Bag of Tricks for Efficient Text Classification (2017)
Khan, R.A., Spruijt, A., Mahboob, U., van Merrienboer, J.J.: Determining ‘curriculum viability’ through standards and inhibitors of curriculum quality: a sco** review. BMC Med. Educ. 19(1), 336 (2019)
Komenda, M., et al.: Curriculum map** with academic analytics in medical and healthcare education. PLoS ONE 10(12) (2015). https://doi.org/10.1371/journal.pone.0143748
Krathwohl, D.R.: A revision of bloom’s taxonomy: an overview. Theory into Pract. 41(4), 212–218 (2002)
Lee, J., et al.: BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36(4), 1234–1240 (2019)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: Proceedings of Workshop at ICLR (2013)
Mikolov, T., Grave, É., Bojanowski, P., Puhrsch, C., Joulin, A.: Advances in Pre-Training Distributed Word Representations (2018)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed Representations of Words and Phrases and their Compositionality. Adv. Neural Inf. Process. Syst. 26 (2013)
Mikolov, T., Yih, W.t., Zweig, G.: Linguistic Regularities in Continuous Space Word Representations (2013)
Omar, N., et al.: Automated analysis of exam questions according to Bloom’s taxonomy. Proc. Soc. Behav. Sci. 59, 297–303 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Cher, P.H., Lee, J.W.Y., Bello, F. (2022). Machine Learning Techniques to Evaluate Lesson Objectives. In: Rodrigo, M.M., Matsuda, N., Cristea, A.I., Dimitrova, V. (eds) Artificial Intelligence in Education. AIED 2022. Lecture Notes in Computer Science, vol 13355. Springer, Cham. https://doi.org/10.1007/978-3-031-11644-5_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-11644-5_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-11643-8
Online ISBN: 978-3-031-11644-5
eBook Packages: Computer ScienceComputer Science (R0)