Assessing Second Language Pronunciation

Abstract

Pronunciation assessment (PA) is a resurgent subfield within applied linguistics that traverses the domains of psycholinguistics, second language acquisition (SLA), speech sciences, sociolinguistics, and more recently, computational linguistics. Though the terms ‘pronunciation’ and ‘assessment’ are sometimes defined in different ways by different authors, here we regard pronunciation as the vocal articulation of consonants and vowels (segmentals) combined with aspects of oral speech that extend beyond individual sounds, including stress, rhythm and intonation (suprasegmentals).

Appendices

The Research Questions

• 1. What is the role of pronunciation in the construct of L2 speaking? In theoretical models of communicative competence/communicative language ability? And how does it relate to other language components?

• 2. How can the features that are most important for L2 intelligibility and comprehensibility best be targeted in pronunciation assessment, including in classrooms where L2 learners are from mixed first language (L1) backgrounds?

• 3. How can pronunciation assessment (PA) best be addressed in teacher training to promote both familiarity with key concepts in L2 pronunciation, and assessment literacy?

• 4. How is pronunciation most effectively integrated with other language skills, such as grammar and vocabulary when providing feedback for formative assessment purposes?

• 5. How can PA be used to help improve workplace safety/communications in language for specific purposes environments (e.g., ensuring adequate L2 speaking and listening ability for professionals in aviation, medical, and higher education sectors)?

• 6. What is an appropriate standard for assessing L2 pronunciation in settings where the target language is the majority language of the society? And in settings where there is no established L1 norm and interlocutors’ L2 is used as the shared medium of communication?

• 7. After conducting a needs analysis for an L2 learner in terms of perceiving and/or producing key pronunciation features, which linguistic factors are most appropriate for hel** determine whether remediation is needed?

• 8. How can we optimize the listening prompts that we use to make them more authentic representations of the language that learners are likely to encounter outside of instructed contexts (e.g., language varieties represented)?

• 9. How can sources of variance that may bias listeners’ judgments of L2 pronunciation be minimized to promote fairer assessments (e.g., listener familiarity with L1- accented speech)? How can this best be implemented in both research contexts and operational assessments?

• 10. How can technological innovations such as automatic speech recognition, automated scoring, and dialogue systems be harnessed to target L2 pronunciation features that are important for aural/oral communication, including testing learners’ abilities and providing them with meaningful feedback?

Suggested Resources

Celce-Murcia, M., Brinton, D., & Goodwin, J., with Griner, B. (2010). Teaching pronunciation: A course book and reference guide (2nd ed.). Cambridge, UK: Cambridge University Press.

There is, as yet, no book dedicated to the topic of guiding teachers on how to assess L2 pronunciation, including designing, administering, and scoring pronunciation tests and feedback provision for formative assessment purposes. Celce-Murcia et al.’s (2010) influential volume, now in its second edition, constitutes a wide-ranging pronunciation reference for L2 English teachers. In contrast to other pronunciation books for teachers, which either explicitly avoid covering assessment or have scant coverage of the topic, this book includes a ‘Testing and Evaluation’ chapter. It provides examples of sample items that could be used for diagnostic assessment; overviews formal language tests; discusses formative assessment, including feedback (self, peer, and teacher); and introduces testing pronunciation in the classroom. However, the chapter is largely descriptive, with little citation of relevant research, and focuses primarily on age-old practices, such as discrete-point items prototyped in the early 1960s. Beyond a few paragraphs on eliciting extemporaneous speech samples as part of needs analyses, the chapter bypasses coverage of integrated assessment, an indicator of modern assessment practices. Therefore, while serving as a helpful general resource for teachers, particularly given the dearth of practical resources, the book remains a glaring reminder of the gap between PA research and practice.

Isaacs, T. (2018). Shifting sands in second language pronunciation teaching and assessment research and practice. Language Assessment Quarterly, 15, 273–293.

This state-of-the-art article provides a comprehensive sampling of topics under the PA umbrella as part of a special issue on the L2 speaking construct, with the inclusion of an article on PA attesting to its presumed importance in assessing speaking. The article uncovers trends—historical and recent—in pronunciation research, teaching, and assessment to orient PA within the broader context of L2 speaking assessments. It concludes by arguing for the importance of interdisciplinary research in this area and setting out directions for an ambitious research agenda for the future. Content coverage, reflected in an extensive reference list, includes discussions of accent discrimination and scapegoating, pronunciation teaching and assessment goals and targets, major global constructs, models from speech sciences research with potential applications for applied PA, the role of pronunciation in L2 speaking scales, rater effects, PA in interactional exchanges and lingua franca communication, automated assessment, and the potential for human and machine integrated scoring. The article is densely written and it would be worthwhile to unpack it in a larger work or series of works.

Isaacs, T., & Harding, L. (2017). Pronunciation assessment. Language Teaching, 50, 347–366.

This research timeline delineates evolutions in PA and includes an annotated bibliography of seminal works, including research articles, curricular materials, and language tests. The article motivates the topic with a general introduction before presenting a tabular summary of key developments or milestones in the field, which are placed in chronological order and are tagged with at least one of six themes. The themes depict a PA meronomy consisting of operational assessments and scoring systems, practitioner-oriented guides, theoretical frameworks, and empirical studies. The article is an accessible reference for language practitioners and researchers, highlighting key themes and their interrelationships by tracing the historical developments through the discussion of the featured work.

Isaacs, T., & Trofimovich, P. (Eds.). (2017). Second language pronunciation assessment: Interdisciplinary perspectives. Bristol, UK: Multilingual Matters.

This open access volume is the first edited collection dedicated to the topic of PA. The book features chapters on key issues, including rating scale validation, revisiting the role of the native speaker and standard language, pronunciation and lingua franca communication, researching factors that impact intelligibility and comprehensibility, and investigating factors that may compromise the validity of formal and informal judgments of pronunciation (e.g., topic familiarity, listener attitudes). A novel contribution of this volume is its inclusion of chapters on how examining other language skills, including assessing fluency and assessing writing, can inform the topic of pronunciation assessment. Readers will benefit from the cogent exposition of current trends and future directions, though given the book’s overall progressive approach, it would have benefited from a greater focus on automated pronunciation assessment and potential interchange with mobile- or web-based applications.

Kang, O., & Ginther, A. (Eds.). (2018). Assessment in second language pronunciation. Abingdon, UK: Routledge.

In this edited collection, leading researchers share foundational concepts in PA and explore the intersection between PA and technology. Readers will find numerous merits to this volume, including an updated exposition of accent, intelligibility, and comprehensibility; theoretical justifications for PA; the development of PAs in the context of World Englishes; and factors which affect rating and scoring. Those seeking information on recent technological advancements in PA and the role of pronunciation in automated scoring will benefit from three chapters devoted to technology and PA. One shortcoming, however, is that the book largely neglects perception as it pertains to PA, focusing almost exclusively on oral production. Given that L2 teachers are part of the book’s target audience, it would have been auspicious to include at least one chapter on perception in PA. Overall, however, the book represents one of two edited works featuring PA and remains an essential collation of theory and practice.