Abstract
Learning analytics (LA) tools promise to improve student learning and retention. However, adoption and use of LA tools in higher education is often uneven. In this case study, part of a larger exploratory research project, we interviewed and observed 32 faculty and advisors at a public research university to understand the technological incentives and barriers related to LA tool adoption and use. Findings indicate that lack of a trustworthy technological infrastructure, misalignment between LA tool capabilities and user needs, and the existence of ethical concerns about the data, visualizations, and algorithms that underlie LA tools created barriers to adoption. Improving tool integration, clarity, and accuracy, soliciting the technological needs and perspectives of LA tool users, and providing data context may encourage inclusion of these tools into teaching and advising practice.
Similar content being viewed by others
References
Aguilar, S., Lonn, S., & Teasley, S. D. (2014, March). Perceptions and use of an early warning system during a higher education transition program. In Proceedings of the fourth international conference on learning analytics and knowledge, ACM (pp. 113–117).
Ali, L., Hatala, M., Gašević, D., & Jovanović, J. (2012). A qualitative evaluation of evolution of a learning analytics tool. Computers and Education, 58(1), 470–489.
Appleby, D. C. (2008). Advising as teaching and learning. Academic advising: A comprehensive handbook, 2, 85–102.
Arnold, K. E., & Pistilli, M. D. (2012, April). Course signals at Purdue: using learning analytics to increase student success. In Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, Association for Computing Machinery (pp. 267–270).
Austin, A. E. (2011). Promoting evidence-based change in undergraduate science education. National Academies National Research Council. Retrieved from: tidemarkinstitute.org.
Balcer, Y., & Lippman, S. A. (1984). Technological expectations and adoption of improved technology. Journal of Economic Theory, 34(2), 292–318.
Brown, M. G. (2016). Blended instructional practice: A review of the empirical literature on instructors’ adoption and use of online tools in face-to-face teaching. The Internet and Higher Education, 31, 1–10.
Creswell, J. W. (2002). Educational research: Planning, conducting, and evaluating quantitative (pp. 146–166). Upper Saddle River, NJ: Prentice Hall.
Crookston, B. B. (1994). A developmental view of academic advising as teaching. NACADA Journal, 14(2), 5–9.
Dahlstrom, E., Brooks, D. C., & Bichsel, J. (2014). The Current Ecosystem of Learning Management Systems in Higher Education: Student, Faculty, and IT Perspectives. Research report. Louisville, CO: ECAR, Sept 2014. http://www.educause.edu/ecar.
Daniel, B. (2015). Big data and analytics in higher education: Opportunities and challenges. British Journal of Educational Technology, 46(5), 904–920.
Davis, F. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13, 319–340.
Dawson, S., Jovanovic, J., Gašević, D., & Pardo, A. (2017, March). From prediction to impact: Evaluation of a learning analytics retention program. In Proceedings of the Seventh International Learning Analytics & Knowledge Conference, ACM (pp. 474–478).
Dawson, S., McWilliam, E., & Tan, J.P.L. (2008). Teaching smarter: How mining ICTdata can inform and improve learning and teaching practice. In Hello! Where are you in the landscape of educational technology? Proceedings ascilite Melbourne 2008. http://www.ascilite.org.au/conferences/melbourne08/procs/dawson.pdf.
Hagen, P. L., & Jordan, P. (2008). Theoretical foundations of academic advising. Academic Advising: A Comprehensive Handbook, 2, 17–35.
Hora. M. T, Bouwma-Gearhart, J., & Park, H. J. (2014). Using Practice-based Research to Frame and Focus Pedagogical Reform: Exploring the Use of Data and Other Information to Guide Instructional Decision-making (WCER Working Paper No. 2014–3). Retrieved from University of Wisconsin–Madison, Wisconsin Center for Education Research website: http://www.wcer.wisc.edu/publications/workingPapers/papers.php.
Kezar, A. J., & Lester, J. (2009). Organizing higher education for collaboration: a guide for campus leaders. San Francisco: Jossey-Bass.
Klein, C., Lester, J., Rangwala, H., & Johri, A. (2019). Learning analytics tools in higher education: Adoption at the intersection of institutional commitment and individual action. The Review of Higher Education, 42(2), 565–593.
Klein, C., Lester, J., Rangwala, H., & Johri, A. (in press). Learning analytics for learning assessment: Complexities in efficacy, implementation, and broad use. In K. Webber, & H. Zheng (Eds.), Analytics and data-informed decision making in higher education: Concepts and real-world applications. Baltimore: Johns Hopkins University Press.
Knight, D. B., Brozina, C., Kinoshita, T., Novoselich, B., Young, G., & Grohs, J. R. (2018). Discipline-focused learning analytics approaches with instead of for users. In J. Lester, C. Klein, A. Johri, & H. Rangwala (Eds.), Learning analytics in higher education: Current innovations, future potential, and practical applications. New York: Routledge.
Knight, D. B., Brozina, C., & Novoselich, B. (2016). An investigation of first-year engineering student and instructor perspectives of learning analytics approaches. Journal of Learning Analytics, 3(3), 215–238.
Lester, J., Klein, C., Rangwala, H., & Johri, A. (2017). Learning analytics in higher education. ASHE Higher Education Report, 43(5), 9–135.
Lincoln, Y. S., & Guba, E. G. (2000). The only generalization is: There is no generalization. Case study method: Key issues, key texts, 17, 27–44.
Lockyer, L., Heathcote, E., & Dawson, S. (2013). Informing pedagogical action: Aligning learning analytics with learning design. American Behavioral Scientist, 57(10), 1439–1459.
Mertens, D. M. (2005). Research and evaluation in education and psychology: Integrating diversity with quantitative, qualitative, and mixed methods. Thousand Oaks: SAGE Publications.
Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for integrating technology in teachers’ knowledge. Teachers College Record, 108(6), 1017–1054.
Norris, D. M., & Baer, L. L. (2013). Building organizational capacity for analytics. Educause Learning Initiative, EDUCAUSE. Retrieved from: https://net.educause.edu/ir/library/pdf/PUB9012.pdf.
Oster, M., Lonn, S., Pistilli, M. D., & Brown, M. G. (2016, April). The learning analytics readiness instrument. In Proceedings of the Sixth International Conference on Learning Analytics & Knowledge,. ACM (pp. 173–182).
Papamitsiou, Z., & Economides, A. A. (2014). Learning analytics and educational data mining in practice: A systematic literature review of empirical evidence. Journal of Educational Technology and Society, 17(4), 49.
Peña-Ayala, A. (2014). Educational data mining: A survey and a data mining-based analysis of recent works. Expert Systems with Applications, 41(4), 1432–1462.
Picciano, A. G. (2012). The evolution of big data and learning analytics in American higher education. Online Learning, 16(3), 9–20.
Privateer, P. M. (1999). Academic technology and the future of higher education: Strategic paths taken and not taken. Journal of Higher Education, 70(1), 60–79.
Rhode, J., Richter, S., Gowen, P., Miller, T., & Wills, C. (2017). Understanding faculty use of the learning management system. Online Learning, 21(3), 68–86.
Rogers, E. (1995). Diffusion of innovations (4th ed.). New York: Free Press.
Saldaña, J. (2015). The coding manual for qualitative researchers. Thousand Oaks: Sage.
Siemens, G. (2011). Learning and academic analytics [website blog post]. Learning and Knowledge Analytics. http://www.learninganalytics.net/?p=131.
Stake, R. (2003). Case studies. In N. K. Denzin & Y. S. Lincoln (Eds.), Strategies of qualitative inquiry (2nd ed., pp. 134–164). Thousand Oaks: Sage.
Stake, R. E. (2005). Qualitative case studies. In N. K. Denzin & Y. S. Lincoln (Eds.), The SAGE handbook of qualitative research (3rd ed., pp. 433–466). Thousand Oaks: Sage.
Straub, E. T. (2009). Understanding technology adoption: Theory and future directions for informal learning. Review of educational research, 79(2), 625–649.
Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park: SAGE Publications.
Svinicki, M. D., Williams, K., Rackley, K., Sanders, A. J., Pine, L., & Stewart, J. (2016). Factors associated with faculty use of student data for instructional improvement. International Journal for the Scholarship of Teaching and Learning, 10(2), n2.
Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27, 425–478.
Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The current landscape of learning analytics in higher education. Computers in Human Behavior, 89, 98–110.
Zellweger Moser, F. (2007a). Faculty adoption of educational technology. EDUCAUSE quarterly, 30(1), 66.
Zellweger Moser, F. (2007b). The strategic management of E-learning support. New York: Waxmann Münster.
Acknowledgements
This research was supported in part by a grant from the National Science Foundation under Grant IIS-1447489.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical standards
The authors have complied with all ethical standards.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Klein, C., Lester, J., Rangwala, H. et al. Technological barriers and incentives to learning analytics adoption in higher education: insights from users. J Comput High Educ 31, 604–625 (2019). https://doi.org/10.1007/s12528-019-09210-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12528-019-09210-5