SpringerLink
Log in

Remodeling effects of linguistic features on L2 writing quality of two genres

  • Published:
Reading and Writing Aims and scope Submit manuscript

Abstract

This study modeled the effects of essay length and language features on the rated quality of second language (L2) expository and argumentative essays composed by Chinese university students. Latent variables were writing quality captured by essay scores, and lexical sophistication, syntactic complexity and cohesion, each of which was measured by different linguistic features in the essays. Results showed that, (i) essay length, sophisticated words (i.e., words eliciting longer lexical decision times, and used in academic texts), complex nominals and coordinate phrases could account for approximately 45% of the variance in the scores of argumentative essays, and (ii) essay length, sophisticated words (i.e., academic words, words used in restricted context, and words that have fewer orthographic neighbors), complex nominals, coordinate phrases and word type-token ratio could explain approximately 49% of the variance in the scores of expository essays. Such findings indicate that, although differing in genre, raters tended to give higher scores to the essays of two genres that contained more sophisticated words, complex phrases, and distinct types of words.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Attali, Y., & Powers, D.(2008). A developmental writing scale (ETS Research Report No. RR-08–19). Princeton, NJ: Educational Testing Service.

  • Bachman, L. F., & Palmer, A. S. (2010). Language assessment in practice: Develo** language assessments and justifying their use the real world. Oxford University Press.

  • Bae, J., & Min, S. (2018). Genre-based analysis of lexical complexity in L2 college students’ academic writing. Journal of Language Sciences, 58, 289–314.

    Article  Google Scholar 

  • Balota, D., Yap, M. J., Cortese, M. J., Hutchison, K., Kessler, B., Loftis, B., & Treiman, R. (2007). The English lexicon project. Behavior Research Methods, 39, 445–459.

    Article  Google Scholar 

  • Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173–1182.

    Article  Google Scholar 

  • Beers, S. F., & Nagy, W. E. (2009). Syntactic complexity as a predictor of adolescent writing quality: Which measures? Which genre? Reading & Writing, 22, 185–200.

    Article  Google Scholar 

  • Biber, D., Gray, B., & Poonpon, K. (2011). Should we use characteristics of conversation to measure grammatical complexity in L2 writing development? TESOL Quarterly, 45, 5–35.

    Article  Google Scholar 

  • Biber, D., Gray, B., & Staples, S. (2016). Predicting patterns of grammatical complexity across language exam task types and proficiency levels. Applied Linguistics, 37, 639–668.

    Article  Google Scholar 

  • Bloor, M., & Bloor, T. (1986). Language for specific purposes: Practice and theory. In CLCS occasional papers. Dublin: Centre for Language & Communication Studies, Trinity College.

  • Byrne, B. M. (2006). Structural equation modeling with EQS: Basic concepts, applications, and programming (2nd ed.). Lawrence Erlbaum.

  • Chiang, S. (2003). The importance of cohesive conditions to perceptions of writing quality at the early stages of foreign language learning. System, 31, 471–484.

    Article  Google Scholar 

  • Chenoweth, N. A., & Hayes, J. R. (2001). Fluency in writing: Generating text in L1 and L2. Written Communication, 18, 90–98.

    Article  Google Scholar 

  • Chodorow, M., & Brustein, J. (2004). Beyond essay length: Evaluating e-rater®’s performance on TOEFL essays (TOEFL Research Report No. 73). Princeton, NJ: Educational Testing Service.

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum.

    Google Scholar 

  • Connor, U. (1990). Linguistic/rhetorical measures for international persuasive student writing. Research in the Teaching of English, 24, 67–87.

    Google Scholar 

  • Connor, U., Davis, K., & de Rycker, T. (1995). Correctness and clarity in applying for overseas jobs: A cross-cultural analysis of U.S. and Flemish applications. Text, 15, 457–475.

    Google Scholar 

  • Coxhead, A. (2000). A new Academic Word List. TESOL Quarterly, 34, 213–238.

    Article  Google Scholar 

  • Crossley, S. A. (2020). Linguistic features in writing quality and development: An overview. Journal of Writing Research, 11, 415–443.

    Article  Google Scholar 

  • Crossley, S. A., Kyle, K., & McNamara, D. S. (2015). To aggregate or not? Linguistic features in automatic essay scoring and feedback systems. Journal of Writing Assessment, 8(1), 1–19.

  • Crossley, S. A., Kyle, K., & McNamara, D. S. (2016a). The development and use of cohesive devices in L2 writing and their relations to judgments of essay quality. Journal of Second Language Writing, 32, 1–16.

    Article  Google Scholar 

  • Crossley, S. A., Kyle, K., & McNamara, D. S. (2016b). The tool for the automatic analysis of text cohesion (TAACO): Automatic assessment of local, global, and text cohesion. Behavior Research Methods, 48, 1227–1237.

    Article  Google Scholar 

  • Crossley, S. A., & McNamara, D. S. (2010). Cohesion, coherence, and expert evaluations of writing proficiency. In S. Ohlsson & R. Catrambone (eds.): Proceedings of the 32nd Annual Conference of the Cognitive Science Society. Cognitive Science Society, pp. 984–989.

  • Crossley, S. A., & McNamara, D. S. (2011). Text coherence and judgments of essay quality: Models of quality and coherence. In L. Carlson, C. Hoelscher, and T. F. Shipley (eds.): Proceedings of the 29th Annual Conference of the Cognitive Science Society. Cognitive Science Society, pp. 1236–1241.

  • Crossley, S. A., & McNamara, D. S. (2012). Predicting second language writing proficiency: The roles of cohesion and linguistic sophistication. Journal of Research in Reading, 35, 115–135.

    Article  Google Scholar 

  • Crossley, S. A., & McNamara, D. S. (2014). Does writing development equal writing quality? A computational investigation of syntactic complexity in L2 learners. Journal of Second Language Writing, 26, 66–79.

    Article  Google Scholar 

  • Cumming, A., Kantor, R., Baba, K., Erdosy, U., Eouanzoui, K., & James, M. (2005). Differences in written discourse in independent and integrated prototype tasks for next generation TOEFL. Assessing Writing, 10, 5–43.

    Article  Google Scholar 

  • Davies, M. (2009). The 385+ million word Corpus of Contemporary American English (1990–2008+): Design, architecture, and linguistic insights. International Journal of Corpus Linguistics, 14, 159–190.

    Article  Google Scholar 

  • East, M. (2009). Evaluating the reliability of a detailed analytic scoring rubric for foreign language writing. Assessing Writing, 14, 88–115.

    Article  Google Scholar 

  • Ferris, D. (1994). Lexical and syntactic features in ESL writing by students at different levels of L2 proficiency. TESOL Quarterly, 28(2), 414–420.

  • Flowerdew, J., & Peacock, M. (Eds.) (2001). Research perspectives on english for academic purposes. Cambridge: Cambridge University Press.

  • Garner, J., Crossley, S., & Kyle, K. (2019). N-gram measures and L2 writing proficiency. System, 80, 176–187.

    Article  Google Scholar 

  • Grabe, W., & Kaplan, R. B. (1996). Theory and practice of writing: An applied linguistic perspective. Longman.

  • Grant, L., & Ginther, A. (2000). Using computer-tagged linguistic features to describe L2 writing differences. Journal of Second Language Writing, 9, 123–145.

    Article  Google Scholar 

  • Guo, L., Crossley, S. A., & McNamara, D. S. (2013). Predicting human judgments of essay quality in both integrated and independent second language writing samples: A comparison study. Assessing Writing, 18, 218–238.

    Article  Google Scholar 

  • Halliday, M. A. K., & Hasan, R. (1985). Language, context and text: Aspects of language in social-semiotic perspective. Deakin University Press.

  • Jung, Y., Crossley, S. A., & McNamara, D. S. (2015). Linguistic features in MELAB writing performances (Working Paper No. 2015–05). http://www.cambridgemichigan.org/wp-content/uploads/2015/04/CWP-2015-05.pdf.

  • Kim, M., & Crossley, S. A. (2018). Modeling second language writing quality: A structural equation investigation of lexical, syntactic, and cohesive features in source-based and independent writing. Assessing Writing, 37, 39–56.

    Article  Google Scholar 

  • Kim, M., Crossley, S. A., & Kyle, K. (2018). Lexical sophistication as a multidimensional phenomenon: Relations to second language lexical proficiency, development, and writing quality. Modern Language Journal, 102, 120–141.

    Article  Google Scholar 

  • Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). Guilford.

  • Knapp, P., & Watkins, M. (2005). Genre, text. University of New South Wales Press Ltd.

    Google Scholar 

  • Knoch, U., Rouhshad, A., & Storch, N. (2014). Does the writing of undergraduate ESL students develop after one year of study in an English-medium university? Assessing Writing, 21, 1–17.

    Article  Google Scholar 

  • Kuperman, V., Stadthagen-Gonzalez, H., & Brysbaert, M. (2012). Age-of-acquisition ratings for 30,000 English words. Behavior Research Methods, 44, 978–990.

    Article  Google Scholar 

  • Kyle, K. (2016). Measuring syntactic development in L2 writing: Fine grained indices of syntactic complexity and usage-based indices of syntactic sophistication. Georgia State University Dissertation. http://scholarworks.gsu.edu/alesl_diss/35.

  • Kyle, K., & Crossley, S. A. (2015). Automatically assessing lexical sophistication: Indices, tools, findings, and application. TESOL Quarterly, 49, 757–786.

    Article  Google Scholar 

  • Kyle, K., & Crossley, S. A. (2016). The relationship between lexical sophistication and independent and source-based writing. Journal of Second Language Writing, 34, 12–24.

    Article  Google Scholar 

  • Kyle, K., & Crossley, S. A. (2018). Measuring syntactic complexity in L2 writing using fine-grained clausal and phrasal indices. Modern Language Journal, 102, 333–349.

    Article  Google Scholar 

  • Li, H. (2015). Relationship between measures of syntactic complexity and judgments of EFL writing quality. In Proceedings of 2015 Youth Academic Forum on Linguistics, Literature, Translation and Culture. American Scholars Press, pp. 216–222.

  • Li, Q., & Kong, W. (2011). Review of research on ESL/EFL writing rating. Foreign Language Testing and Teaching, 1, 18–27.

    Google Scholar 

  • Lu, X. (2010). Automatic analysis of syntactic complexity in second language writing. International Journal of Corpus Linguistics, 15(4), 474–496.

  • Lu, X. (2011). A corpus-based evaluation of syntactic complexity measures as indices of college-level ESL writers’ language development. TESOL Quarterly, 45, 36–62.

    Article  Google Scholar 

  • Lu, X. (2012). The relationship of lexical richness to the quality of ESL learners’ oral narratives. Modern Language Journal, 96, 190–208.

    Article  Google Scholar 

  • Lu, X. (2017). Automated measurement of syntactic complexity in corpus-based L2 writing research and implications for writing assessment. Language Testing, 34, 493–511.

    Article  Google Scholar 

  • Lu, X., & Ai, H. (2015). Syntactic complexity in college-level English writing: Differences among writers with diverse L1 backgrounds. Journal of Second Language Writing, 29, 16–27.

    Article  Google Scholar 

  • MacArthur, C. A., Jennings, A., & Philippakos, Z. A. (2019). Which linguistic features predict quality of argumentative writing for college basic writers, and how do those features change with instruction? Reading and Writing, 32, 1553–1574.

    Article  Google Scholar 

  • McCarthy, P. M., & Jarvis, S. (2010). MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment. Behavior Research Methods, 42, 381–392.

    Article  Google Scholar 

  • McNamara, D. S., Crossley, S. A., & McCarthy, P. M. (2010). Linguistic features of writing quality. Written Communication, 27, 57–86.

    Article  Google Scholar 

  • McNamara, D. S., Crossley, S. A., & Roscoe, R. (2013). Natural language processing in an intelligent writing strategy tutoring system. Behavior Research Methods, 45(2), 499–515.

  • Ortega, L. (2003). Syntactic complexity measures and their relationship to L2 proficiency: A research synthesis of college-level L2 writing. Applied Linguistics, 24, 492–518.

    Article  Google Scholar 

  • Ortega, L. (2015). Syntactic complexity in L2 writing: Progress and expansion. Journal of Second Language Writing, 29, 82–94.

    Article  Google Scholar 

  • Perin, D., & Lauterbach, M. (2016). Assessing text-based writing of low-skilled college students. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-016-0122-z.

  • Ravid, D. D. (2005). Emergence of linguistic complexity in later language development: Evidence from expository text construction. In D. D. Ravid & H.

  • Read, J. (2000). Assessing vocabulary. Cambridge University Press.

    Book  Google Scholar 

  • Römer, U., & Schulze, R. (Eds.). (2009). Exploring the lexis-grammar interface. John Benjamins.

    Google Scholar 

  • Schoonen, R., Van Gelderen, A., Stoel, R. D., Hulstijn, J., & De Glopper, K. (2011). Modeling the development of L1 and EFL writing proficiency of secondary school students. Language Learning, 61, 31–79.

    Article  Google Scholar 

  • Shi, D., Lee, T., & Maydeu-Olivares, A. (2019). Understanding the model size effect on SEM fit indices. Educational and Psychological Measurement, 79, 310–334. https://doi.org/10.1177/0013164418783530

    Article  Google Scholar 

  • Shi, L. (2001). Native- and nonnative-speaking EFL teachers’ evaluation of Chinese students’ English writing. Language Testing, 18, 303–325.

    Google Scholar 

  • Simpson-Vlach, R., & Ellis, N. C. (2010). An academic formulas list: New methods in phraseology research. Applied Linguistics, 31, 487–512.

    Article  Google Scholar 

  • Stevens, R. J., Lu, X., Baker, D. P., Ray, M. N., Eckert, S. A., & Gamson, D. A. (2015). Assessing the cognitive demands of elementary school reading curricula: An analysis of reading text and comprehension tasks from 1910 to 2000. American Educational Research Journal, 52, 582–617.

    Article  Google Scholar 

  • Tabachnick, B. G., & Fidell, L. S. (2001). Using multivariate statistics (4th ed.). Allyn & Bacon.

    Google Scholar 

  • Taguchi, N., Crawford, W., & Wetzel, D. Z. (2013). What linguistic features are indicative of writing quality? A case of argumentative essays in a college composition program. TESOL Quarterly, 47(2), 420–430.

  • Tseng, W. T., Dörnyei, Z., & Schmitt, N. (2006). A new approach to assessing strategic learning: The case of self-regulation in vocabulary acquisition. Applied Linguistics, 27, 78–102.

    Article  Google Scholar 

  • Vögelin, C., Jansen, T., Keller, S. D., Machts, N., & Möller, J. (2019). The influence of lexical features on teacher judgments of ESL argumentative essays. Assessing Writing, 39, 50–63.

    Article  Google Scholar 

  • Way, P., Joiner, E. G., & Seaman, M. (2000). Writing in the secondary foreign language classroom: The effects of prompts and tasks on novice learners of French. Modern Language Journal, 84, 171–184.

    Article  Google Scholar 

  • Weigle, S. C. (2002). Assessing writing. Cambridge University Press.

    Book  Google Scholar 

  • Wittgenstein, L. (1953). Philosophical investigations. Blackwell.

    Google Scholar 

  • Wolfe-Quintero, K., Inagaki, S., & Kim, H.Y. (1998). Second language development in writing: Measures of fluency, accuracy, and complexity. University of Hawaii Press.

  • Wu, M. (2010). Structural equation model: Application of AMOS. Chongqing University Press.

  • Yan, J. L. (2012). Analysis on feedback of new rating scale for TEM8 writing. Foreign Language Testing and Teaching, 4, 6–13.

    Google Scholar 

  • Yang, W., Lu, X., & Weigle, S. C. (2015). Different topics, different discourse: Relationships among writing topic, measures of syntactic complexity, and judgments of writing quality. Journal of Second Language Writing, 28, 53–67.

    Article  Google Scholar 

  • Yoon, H., & Polio, C. (2017). The linguistic development of students of English as a second language in two written genres. TESOL Quarterly, 51, 275–301.

    Article  Google Scholar 

  • Zhang, X., & Li, W. (2021). Effects of n-grams on the rated L2 writing quality of expository essays: A conceptual replication and extension. System, 97, Article 102437.

  • Zhang, X., Lu, X., & Li, W. (2022). Beyond differences: Assessing effects of shared linguistic features on L2 writing quality of two genres. Applied Linguistics, 43(1), 168–195.

Download references

Funding

This study was supported by the National Social Science Foundation of China (GRANT NUMBER: 20BYY084).

Author information

Authors and Affiliations

  1. School of Foreign Studies, **an Jiaotong University, No.28, West **anning Road, **’an, 710049, Shaanxi, People’s Republic of China

    **aopeng Zhang

  2. School of Foreign Studies, North Minzu University, No. 204, Wenchang Avenue, **xia District, Yinchuan, 750021, People’s Republic of China

    Wenwen Li

Authors
  1. **aopeng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenwen Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to **aopeng Zhang.

Ethics declarations

Conflict of interest

Non-financial interests that are directly or indirectly related to the work submitted for publication.

Ethics approval

Due to that fact that our study is corpus-based with no human participants involved, so no study-specific approval by the appropriate ethics is needed.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 55 kb)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, X., Li, W. Remodeling effects of linguistic features on L2 writing quality of two genres. Read Writ 37, 1209–1234 (2024). https://doi.org/10.1007/s11145-023-10438-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11145-023-10438-y

Keywords

Search

Navigation