SpringerLink
Log in

Assessing the interchangeability of linked scores in multivariable statistical analyses

  • Published:
Quality of Life Research Aims and scope Submit manuscript

Abstract

Purpose

Using the lens of classical test theory, we examine a linkage’s generalizability with respect to use in multivariable analyses, including multiple regression and structural equation modeling, rather than comparison of established subpopulations as is most common in the literature.

Methods

To aid in this evaluation, we present a structural-equation-modeling based statistical method to examine the suitability of a given linkage for use cases involving continuous and categorical variables external to the linkage itself.

Results

Using the PROMIS® Parent Proxy and Early Childhood Global Health measures, we show that, although a high correlation between the scores (here, r = .829) may imply a general suitability for linking, a more detailed investigation of content, measurement structure, and results of the proposed methodology reveal important differences between the measures which can compromise interchangeability in certain use cases.

Conclusion

In addition to the statistical quality of a linkage, users of linking methodology should also assess the question of whether the linkage is appropriate to apply to particular use cases of interest.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Data availability

Crosswalk tables are publicly available at the APA OSF repository (url: https://osf.io/nf8bx/). All measures used can be obtained from the Health Measures website at healthmeasures.net or modified therefrom, with modifications described in the manuscript. Study data and analysis code are not publicly available.

Notes

  1. The actual item parameters used for PROMIS scoring are proprietary and could not be included. To obtain these parameters, contact HealthMeasures.net. Perturbation was conducted to mask the actual values by adding noise generated, separately for each item, from a random uniform distribution with limits of − .25 and .25.

References

  1. Dorans, N. J., Pommerich, M., & Holland, P. W. (2007). Linking and aligning scores and scales. Springer.

    Book  Google Scholar 

  2. Feuer, M. J. (2005). E pluribus Unum: Linking tests and democratic education. Measurement and research in the accountability era (pp. 173–192). Routledge.

    Google Scholar 

  3. Huggins, A. C., & Penfield, R. D. (2012). An NCME instructional module on population invariance in linking and equating. Educational Measurement: Issues and Practice, 31(1), 27–40.

    Article  Google Scholar 

  4. Curran, P. J., & Hussong, A. M. (2009). Integrative data analysis: The simultaneous analysis of multiple data sets. Psychological Methods, 14(2), 81.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Choi, S. W., Schalet, B., Cook, K. F., & Cella, D. (2014). Establishing a common metric for depressive symptoms: Linking the BDI, CESD, and PHQ9 to PROMIS depression. Psychological Assessment, 26(2), 513.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Kaat, A. J., Newcomb, M. E., Ryan, D. T., & Mustanski, B. (2017). Expanding a common metric for depression reporting: Linking two scales to PROMIS® depression. Quality of Life Research, 26(5), 1119–1128.

    Article  PubMed  Google Scholar 

  7. Blackwell, C. K., Tang, X., Elliott, A. J., Thomes, T., Louwagie, H., Gershon, R., & Cella, D. (2021). Develo** a common metric for depression across adulthood: Linking PROMIS depression with the Edinburgh Postnatal Depression Scale. Psychological Assessment, 33(7), 610.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Holland, P. W. (2007). A framework and history for score linking. Linking and aligning scores and scales (pp. 5–30). Springer.

    Chapter  Google Scholar 

  9. Bentler, P. M. (2017). Specificity-enhanced reliability coefficients. Psychological Methods, 22(3), 527.

    Article  PubMed  Google Scholar 

  10. Kallen, M. A., Lai, J. S., Blackwell, C. K., Schuchard, J. R., Forrest, C. B., Wakschlag, L. S., & Cella, D. (2022). Measuring PROMIS® global health in early childhood. Journal of Pediatric Psychology, 47(5), 523–533.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Forrest, C. B., Bevans, K. B., Pratiwadi, R., Moon, J., Teneralli, R. E., Minton, J. M., & Tucker, C. A. (2014). Development of the PROMIS® pediatric global health (PGH-7) measure. Quality of Life Research, 23, 1221–1231.

    Article  PubMed  Google Scholar 

  12. Cella, D., Blackwell, C. K., & Wakschlag, L. S. (2022). Bringing PROMIS to early childhood: Introduction and qualitative methods for the development of early childhood parent report instruments. Journal of Pediatric Psychology, 47(5), 500–509.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Lai, J.-S., Kallen, M. A., Blackwell, C. K., Wakschlag, L. S., & Cella, D. (2022). Psychometric considerations in develo** PROMIS® measures for early childhood. Journal of Pediatric Psychology, 47(5), 510–522.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Albano, A. D. (2016). equate: An R package for observed-score linking and equating. Journal of Statistical Software, 74(8), 1–36. https://doi.org/10.18637/jss.v074.i08

    Article  Google Scholar 

  15. R Core Team. (2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing.

    Google Scholar 

  16. Huber, P. J. (1967). The behavior of maximum likelihood estimates under nonstandard conditions. Proceedings of the fifth Berkeley symposium on mathematical statistics and probability. University of California Press.

    Google Scholar 

  17. White, H. (1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica, 48(817), 838. https://doi.org/10.2307/1912934

    Article  Google Scholar 

  18. Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. https://doi.org/10.18637/jss.v048.i02

    Article  Google Scholar 

  19. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70.

    Article  Google Scholar 

  20. Dorans, N. J., & Holland, P. W. (2000). Population invariance and the equitability of tests: Basic theory and the linear case. Journal of Educational Measurement, 37, 281–306.

    Article  Google Scholar 

  21. Dorans, N. J., & Feigenbaum, M. D. (1994). Equating issues engendered by changes to the new SAT and PSAT/NMSQT. In I. M. Lawrence, N. J. Dorans, M. D. Feigenbaum, N. J. Feryok, A. P. Schmitt, & N. K. Wright (Eds.), Technical issues related to the introduction of the new SAT and PSAT/NMSQT (ETS Research Memorandum No. RM-94-10). Educational Testing Service.

    Google Scholar 

  22. Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, 863.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Routledge Academic.

    Google Scholar 

Download references

Acknowledgements

Research reported in this publication was supported by the Environmental influences on Child Health Outcomes (ECHO) program, Office of The Director, National Institutes of Health, under Award Number U24OD023319 with co-funding from the Office of Behavioral and Social Sciences Research (OBSSR; Person Reported Outcomes Core). We have no conflicts of interest to disclose.

Funding

Research reported in this publication was supported by the Environmental influences on Child Health Outcomes (ECHO) program, Office of The Director, National Institutes of Health, under Award Number U24OD023319 with co-funding from the Office of Behavioral and Social Sciences Research (OBSSR; Person Reported Outcomes Core).

Author information

Authors and Affiliations

  1. Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, 625 N. Michigan Ave Fl 27, Chicago, IL, 60611, USA

    Maxwell Mansolf, Courtney K. Blackwell, David Cella & **-Shei Lai

Authors
  1. Maxwell Mansolf
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Courtney K. Blackwell
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David Cella
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. **-Shei Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. All analyses and summarizations of results were performed by MM, who also wrote the first draft of the manuscript. All authors contributed to proofreading and improving the manuscript to its final form, and all authors reviewed and approved the final manuscript.

Corresponding author

Correspondence to Maxwell Mansolf.

Ethics declarations

Competing interests

We have no conflict of interest to disclose.

Ethical approval

This study was performed in line with the principles of the Declaration of Helsinki. We received approval from Northwestern University’s Institutional Review Board. Study materials and data are not publicly available.

Consent to participate

All participants provided informed consent to participate in this research.

Consent to publication

No individual person’s data are presented, only aggregated results across participants.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 2272 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mansolf, M., Blackwell, C.K., Cella, D. et al. Assessing the interchangeability of linked scores in multivariable statistical analyses. Qual Life Res 33, 1121–1131 (2024). https://doi.org/10.1007/s11136-023-03592-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11136-023-03592-x

Keywords

Search

Navigation