Abstract
The purpose of this chapter is to describe the use of Rasch measurement theory to examine the impact of cognitive complexity and linguistic cohesion on science items. This chapter describes a linear logistic Rasch model (LLRM), and conducts a case study to illustrate how to apply the LLRM. In order to provide continuity with previous research on the Rasch model, this chapter describes the connections between methods used to explore item covariates including the Many Facet Rasch Model, hierarchical regression models, and LLRM. The case study uses high school biology assessment data to illustrate the connections between these three approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Azen, R., & Walker, C. M. (2011). Categorical data analysis for the behavioral and social sciences. Routledge.
Bloom, B. S., et al. (1956). Taxonomy of educational objectives, handbook 1: Cognitive domain. Longman.
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48.
Bond, T. G., Yan, Z., & Heene, M. (2021). Applying the Rasch model: Fundamental measurement in the human sciences. Routledge.
Boone, W. J., & Staver, J. R. (2020). Advances in Rasch analyses in the human sciences (p. 13). Springer.
Bulut, O., Gorgun, G., & Yildirim-Erbasli, S. N. (2021). Estimating explanatory extensions of dichotomous and Polytomous Rasch models: The eirm package in R. Psych, 3(3), 308–321.
De Boeck, P., Bakker, M., Zwitser, R., Nivard, M., Hofman, A., Tuerlinckx, F., & Partchev, I. (2011). The estimation of item response models with the lmer function from the lme4 package in R. Journal of Statistical Software, 39, 1–28.
De Boeck, P., & Wilson, M. (2004). Explanatory item response models: A generalized linear and nonlinear approach. Springer.
Embretson, S. E. (1997). Multicomponent latent trait models. In W. van der Linden & R. Hambleton (Eds.), Handbook of modern item response theory (pp. 305–322). Springer.
Embretson, S. E. (Ed.). (2010). Measuring psychological constructs: Advances in model-based approaches. American Psychological Association.
Embretson, S. E., & McCollam, K. M. S. (2000). A multicomponent Rasch model for measuring covert processes: Application to life span ability changes. In M. Wilson & G. Engelhard (Eds.), Objective measurement: Theory into practice. Ablex Publishing Company.
Engelhard, G., & Wang, J. (2021). Rasch models for solving measurement problems: Invariant measurement in the social sciences. Sage.
Engelhard, G., & Wind, S. A. (2022). A history of Rasch measurement theory. In B. Clauser & M. Bunch (Eds.), History of educational measurement in the United States (pp. 346–360). Routledge.
Faraway, J. J. (2016). Extending the linear model with R: Generalized linear, mixed effects and nonparametric regression models. Chapman and Hall/CRC.
Fischer, G. H. (1973). The linear logistic test model as an instrument in educational research. Acta Psychologica, 37(6), 359–374.
Fischer, G. H. (1997). Unidimensional linear logistic Rasch models. In W. van der Linden & R. Hambleton (Eds.), Handbook of modern item response theory (pp. 225–243). Springer.
Flesch, R. (1948). A new readability yardstick. Journal of Applied Psychology, 32, 221–233.
Galvin, E., Simmie, M. G., & O’Grady, A. (2015). Identification of misconceptions in the teaching of biology: A pedagogical cycle of recognition, reduction and removal. Higher Education of Social Science, 8(2), 1–8.
Graesser, A. C., McNamara, D. S., & Kulikowich, J. M. (2011). Coh-Metrix: Providing multilevel analyses of text characteristics. Educational Researcher, 40(5), 223–234.
Holling, H., Blank, H., Kuchenbacker, K., & Kuhn, J. T. (2008). Rule-based item design of statistical word problems: A review and first implementation. Psychology Science, 50(3), 363.
Krell, M., Khan, S., & van Driel, J. (2021). Analyzing cognitive demands of a scientific reasoning test using the linear logistic test model (LLTM). Education Sciences, 11(9), 472.
Lane, S., Raymond, M. R., & Haladyna, T. M. (Eds.). (2016). Handbook of test development (pp. 3–18). Routledge.
Linacre, J. M. (2019a). A User’s guide to WINSTEPS® Rasch-model computer programs: Program manual 4.4. 6. MESA Press.
Linacre, J. M. (2019b). Facets computer program for many-facet Rasch measurement, version 3.81.2. Winsteps.com
Liu, X. (2020). Using and develo** measurement instruments in science education: A Rasch modeling approach (2nd ed.). Information Age Publishing, Inc.
Liu, X., & Boone, W. J. (2006). Applications of Rasch measurement in science education. JAM Press.
National Academies of Sciences, Engineering, and Medicine. (2012). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. The National Academies Press. https://doi.org/10.17226/13165
McNamara, D. S., Louwerse, M. M., Cai, Z., & Graesser, A. (2005). Coh-Metrix Version 1.4. Retrieved from http://cohmetrix.com
Park, M., & Liu, X. (2021). An investigation of item difficulties in energy aspects across biology, chemistry, environmental science, and physics. Springer Research in Science Education, 51(Supplement 1), S43–S60.
Pinheiro, J., & Bates, D. (2006). Mixed-effects models in S and S-PLUS. Springer.
Rasch, G. (1960/1980). Probabilistic models for some intelligence and attainment tests. Danish Institute for Educational Research (Expanded edition, Chicago: University of Chicago Press, 1980).
R Development Core Team. (2004). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-00-3, URL http://www.R-project.org
Spada, H., & May, R. (1982). The linear logistic test model and its application in educational research. The improvement of measurement in education and psychology (pp. 67–84).
Stenner, A. J., Horabin, I., Smith, D. R., & Smith, R. (1988). The Lexile framework. Metametrics, Inc.
Tutz, G. (2011). Regression for categorical data. Cambridge University Press.
Wang, J., & Engelhard, G. (2019). Exploring the impersonal judgments and personal preferences of raters in rater-mediated assessments with unfolding models. Educational and Psychological Measurement, 79(4), 773–795.
Webb, N. (1999). Alignment of science and mathematics standards and assessments in four states (Research monograph No. 18). CCSSO.
Wilson, M. (2005). Constructing measures: An item response modeling approach (2nd ed.). Erlbaum.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendices
Appendices
17.1.1 Appendix A
17.1.1.1 Sample Items from Cell Domain (Levels 2 and 3)
Level 2 ( Skills and concepts ) |
|
Level 3 (Strategic thinking) |
|
17.1.1.2 Sample Items from Cell Domain (Levels 2 and 3) (Cont.)
Level 2 ( Skills and concepts ) |
|
Level 3 (Strategic thinking) |
|
17.1.2 Appendix B Description of the Coh-Metrix Text Features
Text feature | Description |
|---|---|
1.Word count | This is the total number of words in the item stem. |
2. Narrativity | Narrative text tells a story, with characters, events, places, and things that are familiar to the reader. Nonnarrative texts on less familiar topics lie at the opposite end of the continuum. |
3. Syntactic simplicity | This component reflects the degree to which the sentences in the text contain fewer words and use simpler, familiar syntactic structures, which are less challenging to process. At the opposite end of the continuum are texts that contain sentences with more words and use complex, unfamiliar syntactic structures. |
4. Word concreteness | Texts that contain content words that are concrete, meaningful, and evoke mental images are easier to process and understand. Abstract words on the other end are more difficult to process. |
5. Referential cohesion | A text with high referential cohesion contains words and ideas that overlap across sentences and the entire text, forming explicit threads that connect the text for the reader. |
6. Deep cohesion | This component reflects the degree to which the text contains causal and intentional connectives when there are causal and logical relationships within the text. These connectives help the reader to form a more coherent and deeper understanding of the causal events, processes, and actions in the text. If the text is high in deep cohesion, then those relationships and global cohesion are more explicit. |
7. Verb cohesion | This component reflects the degree to which there are overlap** verbs in the text. When there are repeated verbs, the text likely includes a more coherent event structure that will facilitate and enhance situation model understanding. |
8. Connectivity | This component reflects the degree to which the text contains explicit adversative, additive, and comparative connectives to express relations in the text. |
17.1.3 Appendix C
17.1.3.1 Many Facets Rasch Model Syntax
17.1.4 Appendix D
17.1.4.1 R syntax for Hierarchical Regression Models
17.1.5 Appendix E
17.1.5.1 R syntax for Linear Logistic Rasch Models
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this chapter
Cite this chapter
Yuan, Y., Engelhard, G. (2023). Using Linear Logistic Rasch Models to Examine Cognitive Complexity and Linguistic Cohesion in Science Items. In: Liu, X., Boone, W.J. (eds) Advances in Applications of Rasch Measurement in Science Education. Contemporary Trends and Issues in Science Education, vol 57. Springer, Cham. https://doi.org/10.1007/978-3-031-28776-3_17
Download citation
DOI: https://doi.org/10.1007/978-3-031-28776-3_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-28775-6
Online ISBN: 978-3-031-28776-3
eBook Packages: EducationEducation (R0)