Abstract
Many local tests were developed before the widespread use of the Common European Framework of Reference for Languages (CEFR), and therefore were not designed to reflect the categorizations of CEFR levels. Institutions wishing to express their local tests in terms of CEFR benchmarks are encouraged to carry out a map** procedure (Council of Europe, Relating language examinations to the common European framework of reference for languages: learning, teaching, assessment (CEFR). Language Policy Division. https://rm.coe.int/1680667a2d, 2009). Although the CoE offers guidelines for such undertakings, the brunt of the challenge remains with the institutions, administrators, and language professionals who must reconcile numerous practical and methodological considerations specific to their local contexts. In this chapter, we report on an exploratory CEFR map** study involving a local ESL placement and proficiency test at a Canadian university. We highlight the ways in which this specific type of testing research is useful to report linkages, but that the real benefit of the exercise is that it encourages documentation and analysis of test constructs and assumptions and facilitates informed reflections about the implications for language programs and local tests, especially those outside Europe, to create or maintain an association to the CEFR.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
As this university is in the French-speaking Canadian province of Quebec, English as an Additional Language (EAL) may more accurately describe the English programs in this institution’s French as a Second Language (FLS) context.
- 2.
See Coombe et al. (2020) for an overview of the multifaceted modes and constructs involved in LAL.
- 3.
See the Francophone Association for Knowledge (Association francophone pour le savoir) or ACFAS (acfas.ca) for more about efforts to maintain and promote French in the dissemination of academic and scientific information.
- 4.
It is important to note that the study began before the publication of the CoE 2018 preliminary version of the updated Companion Volume.
- 5.
Due to financial limitations, the panel included the authors of this study. Although this may raise some concerns, we felt confident in the integrity of the panel judgments, which were not mitigated by inter/intra-rater reliability nor performance rankings as part of the scope of this project.
- 6.
Only responses from test-takers who consented to having their results used for research were considered.
- 7.
A colleague from the university’s French as a Second Language program generously provided this feedback for the panelists.
- 8.
- 9.
This table indicates the ANG test scores of the majority of the samples. When possible, samples at scores lower than 32 and higher than 87 were also evaluated.
- 10.
Inter-rater and intra-rater reliability was outside the scope of this exploratory study.
- 11.
It goes without saying that if we had adopted different methodologies, we might have different results. For example, we could have used the Extended Tucker-Angoff method instead of the Minimally Acceptable Person for standard setting. ROC statistical analysis may also yield more valid results with the Body of Work method (Kingston et al., 2001).
References
Alderson, J. C., Figueras, N., Kuijper, H., Nold, G., Takala, S., & Tardieu, C. (2006). Analysing tests of reading and listening in relation to the common European framework of reference: The experience of the Dutch CEFR construct project. Language Assessment Quarterly, 3(1), 3–30. https://doi.org/10.1207/s15434311laq0301_2
Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 508–600). American Council on Education.
Bournot-Trites, M. G., Friesen, L., Ruest, C., & Zumbo, B. D. (2020). A made-in-Canada second language framework for K-12 education: Another case where no prophet is accepted in their own land. Canadian Journal of Applied Linguistics, 23(2), 141–167. https://journals.lib.unb.ca/index.php/CJAL/article/view/30434
Bruemmer, R. (2018, June 16). Anglophones and francophones have distorted views of each other: Survey. Montreal Gazette.. https://montrealgazette.com/news/local-news/anglophones-and-francophones-have-distorted-views-of-each-other-survey
Coombe, C., Vafadar, H., & Mohebbi, H. (2020). Language assessment literacy: What do we need to learn, unlearn, and relearn? Lang Test Asia, 10(3), 1–16. https://doi.org/10.1186/s40468-020-00101-6
Council of Europe. (2001). Common European framework of reference for languages: Learning, teaching, assessment. Cambridge University Press. https://rm.coe.int/1680459f97
Council of Europe. (2009). Relating language examinations to the common European framework of reference for languages: Learning, teaching, assessment (CEFR). Language Policy Division. https://rm.coe.int/1680667a2d
Council of Europe. (2011). Manual for language test development and examining. Language Policy Division. https://rm.coe.int/manual-for-language-test-development-and-examining-for-use-with-the-ce/1680667a2b
Council of Europe. (2018). Common European framework of reference for languages: Learning, teaching, assessment. Companion volume with new descriptors. Council of Europe. https://rm.coe.int/cefr-companion-volume-with-new-descriptors-2018/1680787989
Council of Europe. (2020). Common European framework of reference for languages: Learning, teaching, assessment. Companion volume. Council of Europe. https://rm.coe.int/common-european-framework-of-reference-for-languages-learning-teaching/16809ea0d4
Dendrinos, B., & Gotsouilia, V. (2015). Setting standards for multilingual frameworks in foreign language education. In B. Spolsky, O. Inbar, & M. Tannenbaum (Eds.), Challenges for language education and policy. Making space for people (pp. 23–50). Routledge
Díez-Belmar, M. B. (2018). Fine-tuning descriptors for CEFR B1 level: Insights from learner corpora. ELT Journal, 17(2), 199–209
Deygers, B., Zeidler, B., Vilcu, D., & Carlsen, C. H. (2018). One framework to unite them all? Use of the CEFR in European university entrance policies. Language Assessment Quarterly, 15(1), 3–15. https://doi.org/10.1080/15434303.2016.1261350
Fulcher, G. (2012). Assessment literacy for the language classroom. Language Assessment Quarterly, 9(2), 113–132. https://doi.org/10.1080/15434303.2011.642041
Fulcher, G. (2020). Operationalizing language assessment literacy. In D. Tsagari (Ed.), Language assessment literacy: From theory to practice (pp. 8–28). Cambridge Scholars Publishing.
Harsch, C., & Hartig, J. (2015). What are we aligning tests to when we report test alignment to the CEFR? Language Assessment Quarterly, 12(4), 333–362. https://doi.org/10.1080/15434303.2015.1092545
Kane, M. (1998). Choosing between examinee-centered and test-centered standard-setting methods. Educational Assessment, 5(3), 129–145. https://doi.org/10.1207/s15326977ea0503_1
Kingston, N. M., Kahl, S. R., Sweeny, K. P., & Bay, L. (2001). Setting performance standards using the body of work method. In G. J. Cizek (Ed.), Setting performance standards: Concepts, methods and perspectives (pp. 219–248). Lawrence Erlbaum.
Lim, G. S., Geranpayeh, A., Khalifa, H., & Buckendahl, C. W. (2013). Standard setting to an international reference framework: Implications for theory and practice. International Journal of Testing, 13(1), 32–49. https://doi.org/10.1080/15305058.2012.678526
Martyniuk, W. (Ed.). (2010). Aligning tests with the CEFR: Reflections on using the Council of Europe’s draft manual. Cambridge University Press.
Milanovic, M., & Weir, C. J. (2010). Series editors’ note. In W. Martyniuk (Ed.), Aligning tests with the CEFR: Reflections on using the Council of Europe’s draft manual (pp. viii–xx). Cambridge University Press.
Nomand-Marconnet, N., & Lo Bianco, J. (2015). The common European framework of reference down under: A survey of its use and non-use in Australian universities. Language Learning in Higher Education, 5(2), 281–307. https://doi.org/10.1515/cercles-2015-0014
North, B. (2014). Putting the common European framework of reference to good use. Language Teaching, 47(2), 228–249. https://doi.org/10.1017/s0261444811000206
North, B. (2020, June). Trolls, unicorns and the CEFR: Precision and professionalism in criticism of the CEFR. CEFR Journal – Research and Practice, 2, 8–24. https://cefrjapan.net/images/PDF/Newsletter/CEFRJournal-2-1_BNorth.pdf
North, B., & Piccardo, E. (2018). Aligning the Canadian language benchmarks (CLB) to the common european framework of reference (CEFR). https://www.language.ca/wp-content/uploads/2019/01/Aligning-the-CLB-and-CEFR.pdf
North, B., & Jaroscz, E. (2013). Implementing the CEFR in teacher-based assessment: Approaches and challenges. In E. Galaczi & C. J. Weir (Eds.), Exploring language frameworks: Proceedings of the ALTE Krakow Conference, July 2011 (Studies in Language Testing Series 36, pp. 118–134). Cambridge University Press.
O’Sullivan, B. (2010). The City & Guilds Communicator examination linking project: A brief overview with reflections on the process. In W. Martyniuk (Ed.), Aligning tests with the CEFR: Reflections on using the Council of Europe’s draft manual (pp. 8–24). Cambridge University Press.
Papageorgiou, S., & Tannenbaum, R. J. (2016). Situating standard setting within argument-based validity. Language Assessment Quarterly, 13(2), 109–123. https://doi.org/10.1080/15434303.2016.1149857
Read, J. (2019). The influence of the common European framework of reference (CEFR) in the Asia-Pacific region. LEARN Journal: Language Education and Acquisition Research Network Journal, 12(1), 12–18. http://files.eric.ed.gov/fulltext/EJ1225686.pdf
Savski, K. (2021). CEFR as language policy: Opportunities and challenges for local agency in a global era. The English Teacher, 50(2), 60–70. https://doi.org/10.52696/aide2513
Shackleton, C. (2018). Develo** CEFR-related language proficiency tests: A focus on the role of piloting. CercleS 8(2), 333–352. https://doi.org/10.1515/cercles-2018-0019
Szabo, T., & Goodier, T. (2018). Collated representative samples of descriptors of language competence developed for young learners. Resource for educators. https://www.coe.int/en/web/common-european-framework-reference-languages/bank-of-supplementary-descriptors
Tannenbaum, R. J., & Baron, P. A. (2011). Map** TOEFL ITP scores onto the common European framework of reference. Educational Testing Service. https://www.ets.org/s/toefl_itp/pdf/map**_toefl_itp_scores_onto_the_common_european_framework_of_reference.pdf
Tannenbaum, R. J., & Wylie, E. C. (2004). Map** test scores onto the common European framework: Setting standards of language proficiency on the test of English as a foreign language (TOEFL), the test of spoken English (TSE), the test of written English (TWE), and the test of English for international communication (TOEIC). Educational Testing Service. http://www.ets.org/Media/Tests/TOEFL/pdf/CEFstudyreport.pdf
Tannenbaum, R. J., & Wylie, E. C. (2008). Linking English-language test scores onto the common European framework of reference: An application of standard-setting methodology. ETS Research Report Series, 2008(1), i–75. https://doi.org/10.1002/j.2333-8504.2008.tb02120.x
Taylor, L. (2004). Issues of test comparability. Research Notes, 15, 2–5. https://www.cambridgeenglish.org/Images/23131-research-notes-15.pdf
Tracy, R. (2017). Language testing in the context of migration. In J.-C. Beacco, H.-J. Krumm, D. Little, & P. Thalgott (Eds.), The linguistic integration of adult migrants/L’intégration linguistique des migrants adulte : Some lessons from research/les enseignements de la recherche (pp. 45–56). De Gruyter. https://doi.org/10.1515/9783110477498-007
Venne, J.-F. (2019, October 5). UQAM: 50 ans de démocratisation de l’éducation universitaire. Le Devoir. https://www.ledevoir.com/societe/education/563910/uqamuqam-50-ans-de-democratisation-de-l-education-universitaire
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Springer, S., Kozlowska, M. (2023). Speaking “CEFR” about Local Tests: What Map** a Placement Test to the CEFR Can and Can’t Do. In: Yan, X., Dimova, S., Ginther, A. (eds) Local Language Testing. Educational Linguistics, vol 61. Springer, Cham. https://doi.org/10.1007/978-3-031-33541-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-33541-9_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-33540-2
Online ISBN: 978-3-031-33541-9
eBook Packages: EducationEducation (R0)