Camelot: A Council of Machine Learning Strategies to Enhance Teaching

Abstract

Several educational institutions including schools, colleges, and training institutes use teaching evaluations to improve instruction. In due course, the massive data overwhelms educators, leading to sub-optimal analysis and marginal instructional improvements. This book chapter presents Camelot, a collection of supervised and unsupervised machine learning strategies to facilitate an automatic and effective analysis of both quantitative and qualitative teaching evaluations. Camelot inputs the quantitative evaluations (numerical ratings for various instructor attributes) and provides insights with varying granularity. The coarse-grained insights comprise categorical ratings for instance, ‘exceeds expectations’, ‘at par’, ‘below par’, etc. The fine-grained insights include the identification of crucial instructor attributes for improvement and pertinent strategies. Camelot uses a deep semantics model to analyze the qualitative teaching evaluations (free-form textual responses) and identify keywords that characterize an educator’s teaching methodology. Camelot combines the results from machine learning techniques to dispense effective improvement strategies. Although we describe Camelot using teaching evaluations as a case study, we assert that it can be easily adapted for other areas of education including student learning outcomes and accreditation studies. We envision Camelot to be a useful automation for seasoned educators, new teachers, school leaders, and policymakers.

Appendix A

See Fig. 7.

Fig. 7

An example decision tree generated by the decision tree classifier. In this tree, the gini score at each node depicts the impurity in classification; the closer it is to 0, the better the classification performance. To classify a new feature point, we simply traverse this decision tree and identify the terminal node (leaf). The class at the leaf node is the classification of the new feature point. For instance, consider a feature point with the following feature values: Enthusiasm equal to 4.3, Organization equal to 4.4, Ability to explain difficult concepts equal to 4.1, Organization equal to 3.8, Speaking ability equal to 4.2, Encourages discussion equal to 4.4, and Knowledge equal to 4.5. For this feature point, the final class is ‘Average’ (7th node from the left).