Abstract
The use of polytomous items as part of background or context questionnaires and complex sampling designs are two features common in international large-scale assessments (ILSA). Popular choices to model polytomous items within ILSA include the partial credit model, the graded response model, and confirmatory factor analysis. However, an absent model in ILSA studies is the continuation ratio model. The continuation ratio model is a flexible alternative and a very extendable response model applicable in different situations. Although existing software can fit this model, not all these tools can incorporate complex sampling design features present in ILSA studies. This study aims to illustrate a method to fit a continuation ratio model including complex sampling design information, thus expanding the modelling tools available for secondary users of large-scale assessment studies.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Adams, R. J. (2005). Reliability as a measurement design effect. Studies in Educational Evaluation, 31(2–3), 162–172. https://doi.org/10.1016/j.stueduc.2005.05.008
Agresti, A. (2019). An introduction to categorical data analysis (3rd ed.). John Wiley & Sons, Inc..
Asparouhov, T. (2005). Sampling weights in latent variable modeling. Structural Equation Modeling: A Multidisciplinary Journal, 12(3), 411–434. https://doi.org/10.1207/s15328007sem1203_4
Asparouhov, T. (2006). General multi-level modeling with sampling weights. Communications in Statistics – Theory and Methods, 35(3), 439–460. https://doi.org/10.1080/03610920500476598
Bates, D., Mächler, M., Bolker, B. M., & Walker, S. C. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1). https://doi.org/10.18637/jss.v067.i01
Calderón, F., & González, J. (2021). Polytomous IRT models versus IRTree models for scoring non-cognitive latent traits. In M. Wiberg, D. Molenaar, J. González, U. Böckenholt, & J. Kim (Eds.), Quantitative psychology (pp. 113–125). https://doi.org/10.1007/978-3-030-74772-5_11
Chalmers, R. P. (2012). Mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6). https://doi.org/10.18637/jss.v048.i06
De Boeck, P., & Partchev, I. (2012). IRTrees: Tree-based item response models of the GLMM family. Journal of Statistical Software, 48(Code Snippet 1). https://doi.org/10.18637/jss.v048.c01
De Boeck, P., Bakker, M., Zwitser, R., Nivard, M., Hofman, A., Tuerlinckx, F., & Partchev, I. (2011). The estimation of item response models with the lmer function from the lme4 package in R. Journal of Statistical Software, 39(12), 1–28. https://doi.org/10.18637/jss.v039.i12
Gochyyev, P. (2015). Essays in psychometrics and behavioral statistics. University of California. https://escholarship.org/content/qt36b8p5nw/qt36b8p5nw.pdf
Gonzalez, E. J. (2012). Rescaling sampling weights and selecting mini-samples from large-scale assessment databases. IERI Monograph Series Issues and Methodologies in Large-Scale Assessments, 5, 115–134.
Hemker, B. T., Andries van der Ark, L., & Sijtsma, K. (2001). On measurement properties of continuation ratio models. Psychometrika, 66(4), 487–506. https://doi.org/10.1007/BF02296191
Hoffman, L. (2015). Longitudinal analysis: Modeling within-person fluctuation and change. Psychology Press. http://www.pilesofvariance.com/
Jeon, M., & De Boeck, P. (2016). A generalized item response tree model for psychological assessments. Behavior Research Methods, 48(3). https://doi.org/10.3758/s13428-015-0631-y
Jöreskog, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34(2), 183–202. https://doi.org/10.1007/BF02289343
Kim, S. (2016). Continuation ratio model in item response theory and selection of models for polytomous items. In L. A. van der Ark, M. Wiberg, S. A. Culpepper, J. A. Douglas, & W.-C. Wang (Eds.), Quantitative psychology research (Vol. 196, pp. 1–13). Springer International Publishing. https://doi.org/10.1007/978-3-319-38759-8_1
Köhler, H., Weber, S., Brese, F., Schulz, W., & Carstens, R. (2018). In H. Köhler, S. Weber, F. Brese, W. Schulz, & R. Carstens (Eds.), ICCS 2016 user guide for the international database. International Association for the Evaluation of Educational Achievement (IEA).
Lietz, P., Cresswell, J. C., Rust, K., & Adams, R. J. (2017). In P. Lietz, J. C. Cresswell, K. Rust, & R. J. Adams (Eds.), Implementation of large-scale education assessments. John Wiley & Sons, Ltd.. https://doi.org/10.1002/9781118762462
Martin, M.O., Mullis, I.V.S., Hooper, M.: Methods and procedures in PIRLS 2016. (2017). https://timssandpirls.bc.edu/publications/pirls/2016-methods.html
Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47(2), 149–174. https://doi.org/10.1007/BF02296272
Masyn, K. E. (2003). Discrete-time survival mixture analysis for single and recurrent events using latent variables. University of California Los Angeles. http://www.statmodel.com/download/masyndissertation.pdf
Masyn, K. E. (2009). Discrete-time survival factor mixture analysis for low-frequency recurrent event histories. Research in Human Development, 6(2–3), 165–194. https://doi.org/10.1080/15427600902911270
Masyn, K. E. (2014). Discrete-time survival analysis in prevention science. In Z. Sloboda & H. Petras (Eds.), Defining prevention science (pp. 513–535). Springer US. https://doi.org/10.1007/978-1-4899-7424-2
Masyn, K. E. (2017). Measurement invariance and differential item functioning in latent class analysis with stepwise multiple indicator multiple cause modeling. Structural Equation Modeling: A Multidisciplinary Journal, 24(2), 180–197. https://doi.org/10.1080/10705511.2016.1254049
Meinck, S. (2020). Sampling, weighting, and variance estimation. In H. Wagemaker (Ed.), Reliability and validity of international large-scale assessment (pp. 113–129). Springer International Publishing. https://doi.org/10.1007/978-3-030-53081-5_7
Muthén, B., & Asparouhov, T. (2014). IRT studies of many groups: The alignment method. Frontiers in Psychology, 5(AUG), 1–7. https://doi.org/10.3389/fpsyg.2014.00978
Muthén, L. K., & Muthén, B. O. (2017). Mplus user’s guide (8th ed.). Muthén & Muthén.
OECD. (2019). PISA 2018 results: What school life means for students’ lives (Vol. III). PISA, OECD Publishing. https://doi.org/10.1787/acd78851-en
OECD. (2020). Scaling procedures and construct validation of context questionnaire data. In PISA 2018 technical report (pp. 1–39). OECD.
Rabe-Hesketh, S., Skrondal, A., & Pickles, A. (2004). GLLAMM manual. In The Berkeley Electronic Press (bepress) (No. 160). The Berkeley Electronic Press (bepress). http://www.bepress.com/ucbbiostat/paper160
Rutkowski, L., Gonzalez, E., Joncas, M., & von Davier, M. (2010). International large-scale assessment data: Issues in secondary analysis and reporting. Educational Research, 39(2), 142–151. https://doi.org/10.3102/0013189X10363170
Rutkowski, D., Rutkowski, L., & Wild, J. (2013). Predictors of school violence internationally: The importance of immigrant status and other factors. In 5th IEA international research conference, 26–28 June 2013. http://www.iea.nl/fileadmin/user_upload/IRC/IRC_2013/Papers/IRC-2013_Rutkowski_etal.pdf
Samejima, F. (1968). Estimation of latent ability using a response pattern of graded scores. ETS Research Bulletin Series, 1968(1), i–169. https://doi.org/10.1002/j.2333-8504.1968.tb00153.x
Schulz, W., & Carstens, R. (2020). Questionnaire development in international large-scale assessment studies. In H. Wagemaker (Ed.), Reliability and validity of international large-scale assessment (Vol. 10, pp. 61–83). Springer International Publishing. https://doi.org/10.1007/978-3-030-53081-5_5
Schulz, W., Carstens, R., Losito, B., & Fraillon, J. (2018). In W. Schulz, R. Carstens, B. Losito, & J. Fraillon (Eds.), ICCS 2016 technical report. International Association for the Evaluation of Educational Achievement (IEA).
Stapleton, L. M. (2013). Incorporating sampling weights into single- and multilevel analyses. In L. Rutkowski, M. von Davier, & D. Rutkowski (Eds.), Handbook of international large scale assessment: Background, technical issues, and methods of data analysis (pp. 363–388). Chapman and Hall/CRC.
Sterba, S. K. (2009). Alternative model-based and design-based frameworks for inference from samples to populations: From polarization to integration. Multivariate Behavioral Research, 44(6), 711–740. https://doi.org/10.1080/00273170903333574
Tutz, G. (1990). Sequential item response models with an ordered response. The British Journal of Mathematical and Statistical Psychology, 43(1), 39–55. https://doi.org/10.1111/j.2044-8317.1990.tb00925.x
Tutz, G. (2016). Sequential models for ordered responses. In W. J. van der Linden (Ed.), Handbook of item response theory (Vol. 1, pp. 139–151). CRC Press. https://doi.org/10.1201/9781315374512
Van de Vijver, F. J. R., Avvisati, F., Davidov, E., Eid, M., Fox, J.-P., Le Donné, N., Lek, K., Meuleman, B., Paccagnella, M., & van de Schoot, R. (2019). Invariance analyses in large-scale studies. OECD Education Working Papers, 201, 1–110. https://doi.org/10.1787/19939019
Van Der Ark, L. A. (2001). Relationships and properties of polytomous item response theory models. Applied Psychological Measurement, 25(3), 273–282. https://doi.org/10.1177/01466210122032073
Vermunt, J. K., & Magidson, J. (2016). Technical guide for latent GOLD choice 5.1: Basic, advanced and syntax. Statistical Innovations Inc..
Yamamoto, K. (1987). A model that combines IRT and latent class models. University of Illinois at Urbana-Champaign. http://ezproxy.puc.cl/dissertations-theses/model-that-combines-irt-latent-class-models/docview/303564885/se-2?accountid=16788
Yin, L., & Fishbein, B. (2020). Creating and interpreting the TIMSS 2019 context questionnaire scales. In M. O. Martin, M. Von Davier, & I. V. S. Mullis (Eds.), Methods and procedures: TIMSS 2019 technical report. TIMSS & PIRLS International Study Center, Lynch School of Education and Human Development, Boston College and International Association for the Evaluation of Educational Achievement (IEA).
Zheng, X., & Yang, J. S. (2016, January). Using sample weights in item response data analysis under complex sample designs. In L. A. van der Ark, D. M. Bolt, W.-C. Wang, J. A. Douglas, & M. Wiberg (Eds.), Quantitative psychology research (pp. 123–137). Springer International Publishing. https://doi.org/10.1007/978-3-319-38759-8_10
Acknowledgements
Research funded by the Fondo Nacional de Desarrollo Científico y Tecnológico FONDECYT N° 1201129 and FONDECYT N° 11180792. David Torres-Irribarra and Jorge González were partially supported by the Agencia Nacional de Investigacion y Desarrollo (ANID) Millennium Nucleus on Intergenerational Mobility: From Modelling to Policy (MOVI); (NCS2021072).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Carrasco, D., Irribarra, D.T., González, J. (2022). Continuation Ratio Model for Polytomous Items Under Complex Sampling Design. In: Wiberg, M., Molenaar, D., González, J., Kim, JS., Hwang, H. (eds) Quantitative Psychology. IMPS 2021. Springer Proceedings in Mathematics & Statistics, vol 393. Springer, Cham. https://doi.org/10.1007/978-3-031-04572-1_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-04572-1_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-04571-4
Online ISBN: 978-3-031-04572-1
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)