Abstract
International large-scale assessments, such as PISA and TIMSS, have had a profound effect on educational policy and school accountability. In an open letter to the OECD in 2014, a large number of academics expressed their concerns over the continuous cycle of global testing and how it negatively affects student’s well-being and impoverishes classrooms, as it inevitably involves more and longer batteries of multiple-choice testing, more scripted “vendor”-made lessons, and less autonomy for teachers. Others have pointed to the ways in which international large-scale assessments have opened up education systems and built country’s assessment capacities. This chapter reviews the ways in which international standardized assessments permeate national policy and school-level decision-making, drawing on the sociology of numbers to understand the appeal of standardization and benchmarking. The OECD initiative “PISA for Schools” is described and how it potentially extends the influence of international assessments on the classroom. An alternative project within the OECD (the “Strategic Education Governance” project) is presented and how the approach of case studies and learning seminars allows countries to learn more holistically about how to hold schools accountable and improve education. Openness, situational awareness, and a consideration for the complexity of successful reform are presented as necessary “habits of mind” to successfully improve learning outcomes. The final conclusion reflects on the limitations and potential usage of international large-scale assessments for effective education reform.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Addey, C., & Sellar, S. (2019). Rationales for (non) participation in international large-scale learning assessments. UNESCO Working paper ED-2019/WP/2.
Auld, E., & Morris, P. (2014). Comparative education, the ‘New Paradigm’ and policy borrowing: constructing knowledge for educational reform. Comparative Education, 50(2), 129–155. https://doi.org/10.1080/03050068.2013.826497
Breakspear, S. (2014, November). How does PISA shape education policy making? Why how we measure learning determines what counts in education. In Centre for Strategic Education Seminar Series Paper (Vol. 40).
Burns, T., & Köster, F. (2016). Governing education in a complex world. Educational Research and Innovation. OECD Publishing.
Burns, B., Köster, F., & Fuster, M. (2016). Educational research and innovation education governance in action lessons from case studies: Lessons from case studies. OECD Publishing.
Carvalho, L. M., & Costa, E. (2015). Seeing education with one’s own eyes and through PISA lenses: Considerations of the reception of PISA in European countries. Discourse: Studies in the Cultural Politics of Education, 36(5), 638–646. https://doi.org/10.1080/01596306.2013.871449
Chabbot, C., & Elliott, E. (Eds.). (2003). Understanding others, educating ourselves: Getting more from international comparative studies in education. National Academies Press.
Desrosières, A. (1998). The politics of large numbers: A history of statistical reasoning. Harvard University Press.
Diaz-Bone, R., & Didier, E. (2016). The sociology of quantification – Perspectives on an emerging field in the social sciences. Historical Social Research, 41(2), 7–26. https://doi.org/10.12759/hsr.41.2016.2.7-26
Espeland, W., & Sauder, M. (2007). Rankings and reactivity: How public measures recreate social worlds. American Journal of Sociology, 113(1), 1–40. https://doi.org/10.1086/517897
Espeland, W., & Stevens, M. L. (1998). Commensuration as a social process. Annual Review of Sociology, 24, 313–343.
Espeland, W. N., & Stevens, M. L. (2008). A sociology of quantification. European Journal of Sociology/Archives Européennes de Sociologie, 49(3), 401–436. https://doi.org/10.1017/S0003975609000150
Ferrer, G., & Fiszbein, A. (2015). What has happened with learning assessment systems in Latin America? In Lessons from the last decade of experience. The World Bank.
Frey, B. S., Homberg, F., & Osterloh, M. (2013). Organizational control systems and pay-for-performance in the public service. Organization Studies, 34(7), 949–972. https://doi.org/10.1177/0170840613483655
Gregory, A. J. (2007). Target setting, lean systems and viable systems: a systems perspective on control and performance measurement. Journal of the Operational Research Society, 58(11), 1503–1517. https://doi.org/10.1057/palgrave.jors.2602319
Grek, S. (2009). Governing by numbers: the PISA ‘effect’ in Europe. Journal of Education Policy, 24(1), 23–37. https://doi.org/10.1080/02680930802412669
Gruber, K. H. (2006, May). The German ‘PISA-Shock’: some aspects of the extraordinary impact of the OECD’s PISA study on the German education system. In Cross-national attraction in education accounts from England and Germany. Symposium Books.
Hegarty, S., & Finlay, S. (2020). Publications and dissemination. In Reliability and validity of international large-scale assessment (pp. 221–230). Springer publishers.
Hopfenbeck, T., Lenkeit, J., El Masri, Y., Cantrell, K., Ryan, J., & Baird, J. (2017). Lessons learned from PISA: A systematic review of Peer Reviewed Articles on the Programme for International Student Assessment. Scandinavian Journal of Educational Research. https://doi.org/10.1080/00313831.2016.1258726
Klieme, E. (2020). Policies and practices of assessment: A showcase for the use (and misuse) of international large scale assessments in educational effectiveness research. In International perspectives in educational effectiveness research (pp. 147–181). Springer.
Lewis, S. (2016). Governing schooling through ‘what works’: the OECD’s PISA for Schools. Journal of Education Policy. https://doi.org/10.1080/02680939.2016.1252855
Lewis, S. (2017). Communities of practice and PISA for Schools: Comparative learning or a mode of educational governance? Education Policy Analysis Archives, 25(92). https://doi.org/10.14507/epaa.25.2901
Lockheed, M., Prokic-Bruer, T., & Shadrova, A. (2015). The experience of middle-income countries participating in PISA 2000–2015 (PISA series). The World Bank & OECD Publishing. https://doi.org/10.1787/9789264246195-en
Meyer, H. D., & Zahedi, K. (2014). An Open Letter: To Andreas Schleicher, OECD, Paris. Mitteilungen der Gesellschaft für Didaktik der Mathematik, 40(97), 31–36.
Morgan, C., & Volante, L. (2016). A review of the Organisation for Economic Cooperation and Development’s international education surveys: Governance, human capital discourses, and policy debates. Policy Futures in Education, 14(6), 775–792. https://doi.org/10.1177/1478210316652024
Niyozov, & Hughes. (2019). http://theconversation.com/problems-with-pisa-why-canadians-should-be-skeptical-of-the-global-test-118096
OECD. (2012). How your school compares internationally: OECD test for schools (Based on PISA) Pilot Trial [US Version]. OECD. http://www.oecd.org/pisa/aboutpisa/pisa-based-test-for-schools-assessment.htm
Pizmony-Levy, O., Harvey, J., Schmidt, W. H., Noonan, R., Engel, L., Feuer, M. J., & Chatterji, M. (2014). On the merits of, and myths about, international assessments. Quality Assurance in Education. https://doi.org/10.1108/QAE-07-2014-0035
Rogers, K. H., Luton, R., Biggs, H., Biggs, R., Blignaut, S., Choles, A. G., Palmer, C. G., & Tangwe, P. (2013). Fostering complexity thinking in action research for change in social–ecological systems. Ecology and Society, 18(2), 31. https://doi.org/10.5751/ES-05330-180231
Rutkowski, D. J. (2007). Converging us softly: how intergovernmental organizations promote neoliberal educational policy. Critical Studies in Education, 48(2), 229–247. https://doi.org/10.1080/17508480701494259
Rutkowski, L., & Rutkowski, D. (2016). A call for a more measured approach to reporting and interpreting PISA results. Educational Researcher, 45(4), 252–257. https://doi.org/10.3102/0013189X16649961
Rutkowski, D., Thompson, G., & Rutkowski, L. (2020). Understanding the policy influence of international large-scale assessments in education. In Reliability and validity of international large-scale assessment, 261. https://doi.org/10.1007/978-3-030-53081-5_15
Sauder, M., & Lancaster, R. (2006). Do rankings matter? The effects of US News & World Report rankings on the admissions process of law schools. Law & Society Review, 40(1), 105–134. https://doi.org/10.1111/j.1540-5893.2006.00261.x
Sellar, S., & Lingard, B. (2018). International large-scale assessments, affective worlds and policy impacts in education. International Journal of Qualitative Studies in Education, 31(5), 367–381
Shewbridge, C., & Köster, F. (2019). Strategic education governance; policy toolkit, design. OECD.
Sjøberg, S. (2015). PISA and global educational governance-a critique of the project, its uses and implications. Eurasia Journal of Mathematics, Science & Technology Education, 11(1). https://doi.org/10.12973/eurasia.2015.1310a
Van Petegem, P., & Vanhoof, J. (2004). Feedback over schoolprestatie-indicatoren als strategisch instrument voor schoolontwikkeling? Lessen uit twee Vlaamse cases [Feedback of performance indicators as a strategic instrument for school improvement? Lessons from two Flemish cases]. Pedagogische Studiën, 81(5), 338–353.
Verger, A., & Parcerisa, L. (2017). Accountability and education in the post-2015 scenario: International trends, enactment dynamics and socio-educational effects. Paper commissioned for the 2017/8 Global Education Monitoring Report, Accountability in education: Meeting our commitments. Paris: UNESCO. ED/GEMR/MRT/2017/P1/1/REV.
Verger, A., Fontdevila, C., & Parcerisa, L. (2019). Reforming governance through policy instruments: how and to what extent standards, tests and accountability in education spread worldwide. Discourse: Studies in the Cultural Politics of Education. https://doi.org/10.1080/01596306.2019.1569882
Wagemaker, H. (2020). Introduction to reliability and validity of international large-scale assessment. In Reliability and validity of international large-scale assessment (pp. 1–5). Springer publishers.
Waldow, F. (2009). What PISA did and did not do: Germany after the ‘PISA-shock’. European Educational Research Journal, 8(3), 476–483. https://doi.org/10.2304/eerj.2009.8.3.476
Wiseman, A. (2013). Policy responses to PISA in comparative perspective. In H. D. Meyer & A. Benavot (Eds.), PISA, power, and policy. The emergence of global educational governance. Symposium Books.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Section Editor information
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this entry
Cite this entry
Ehren, M. (2021). International Large-Scale Assessments and Education System Reform. In: Nilsen, T., Stancel-Piątak, A., Gustafsson, JE. (eds) International Handbook of Comparative Large-Scale Studies in Education. Springer International Handbooks of Education. Springer, Cham. https://doi.org/10.1007/978-3-030-38298-8_36-1
Download citation
DOI: https://doi.org/10.1007/978-3-030-38298-8_36-1
Received:
Accepted:
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-38298-8
Online ISBN: 978-3-030-38298-8
eBook Packages: Springer Reference EducationReference Module Humanities and Social SciencesReference Module Education