Log in

Evaluating Dynamic Discrimination Performance of Risk Prediction Models for Survival Outcomes

  • Published:
Statistics in Biosciences Aims and scope Submit manuscript

Abstract

Risk prediction models for survival outcomes are widely applied in medical research to predict future risk for the occurrence of the event. In many clinical studies, the biomarker data are measured repeatedly over time. To facilitate timely disease prognosis and decision making, many dynamic prediction models have been developed and generate predictions on a real-time basis. As a dynamic prediction model updates an individual’s risk prediction over time based on new measurements, it is often important to examine how well the model performs at different measurement times and prediction times. In this article, we propose a two-dimensional area under curve (AUC) measure for dynamic prediction models and develop associated estimation and inference procedures. The estimation procedures are discussed under two types of biomarker measurement schedules: regular visits and irregular visits. The model parameters are estimated effectively by maximizing a pseudo-partial likelihood function. We apply the proposed method to a renal transplantation study to evaluate the discrimination performance of dynamic prediction models based on longitudinal biomarkers for graft failure.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Begg CB, Cramer LD, Venkatraman E et al (2000) Comparing tumour staging and grading systems: a case study and a review of the issues, using thymoma as a model. Stat Med 19(15):1997–2014

    Article  Google Scholar 

  2. Chen TK, Knicely DH, Grams ME (2019) Chronic kidney disease diagnosis and management: a review. JAMA 322(13):1294–1304

    Article  Google Scholar 

  3. Harrell FE, Califf RM, Pryor DB et al (1982) Evaluating the yield of medical tests. JAMA 247(18):2543–2546

    Article  Google Scholar 

  4. Heagerty PJ, Zheng Y (2005) Survival model predictive accuracy and ROC curves. Biometrics 61(1):92–105

    Article  MathSciNet  MATH  Google Scholar 

  5. Li L, Luo S, Hu B et al (2017) Dynamic prediction of renal failure using longitudinal biomarkers in a cohort study of chronic kidney disease. Stat Biosci 9(2):357–378

    Article  Google Scholar 

  6. Li R, Ning J, Feng Z (2022) Estimation and inference of predictive discrimination for survival outcome risk prediction models. Lifetime Data Anal 28(2):219–240

    Article  MathSciNet  MATH  Google Scholar 

  7. Mayo Clinic (2021) Kidney transplant. https://www.mayoclinic.org/tests-procedures/kidney-transplant/about/pac-20384777. Accessed 1 Nov 2021

  8. Rizopoulos D (2012) Joint models for longitudinal and time-to-event data: with applications in R. CRC Press, Boca Raton

    Book  MATH  Google Scholar 

  9. Rizopoulos D, Ghosh P (2011) A Bayesian semiparametric multivariate joint model for multiple longitudinal outcomes and a time-to-event. Stat Med 30(12):1366–1380

    Article  MathSciNet  Google Scholar 

  10. Saha-Chaudhuri P, Heagerty P (2010) Time-dependent predictive accuracy in the presence of competing risks. Biometrics 66(4):999–1011

    Article  MathSciNet  MATH  Google Scholar 

  11. Saha-Chaudhuri P, Heagerty P (2012) Non-parametric estimation of a time-dependent predictive accuracy curve. Biostatistics 14(1):42–59

    Article  Google Scholar 

  12. Santos J, Martins LS (2015) Estimating glomerular filtration rate in kidney transplantation: still searching for the best marker. World J Nephrol 4(3):345

    Article  Google Scholar 

  13. Schoop R, Graf E, Schumacher M (2008) Quantifying the predictive performance of prognostic models for censored survival data with time-dependent covariates. Biometrics 64(2):603–610

    Article  MathSciNet  MATH  Google Scholar 

  14. Thisted RA (2017) Elements of statistical computing: numerical computation. Routledge, New York

    Book  MATH  Google Scholar 

  15. Uno H, Cai T, Pencina MJ et al (2011) On the C-statistics for evaluating overall adequacy of risk prediction procedures with censored survival data. Stat Med 30(10):1105–1117

    Article  MathSciNet  Google Scholar 

  16. Van Houwelingen HC (2007) Dynamic prediction by landmarking in event history analysis. Scand J Stat 34(1):70–85

    Article  MathSciNet  MATH  Google Scholar 

  17. Zhang J, Ning J, Huang X et al (2021) On the time-varying predictive performance of longitudinal biomarkers: measure and estimation. Stat Med 40(23):5065–5077

    Article  MathSciNet  Google Scholar 

  18. Zheng Y, Heagerty PJ (2004) Semiparametric estimation of time-dependent ROC curves for longitudinal marker data. Biostatistics 5(4):615–632

    Article  MATH  Google Scholar 

  19. Zheng Y, Heagerty PJ (2005) Partly conditional survival models for longitudinal data. Biometrics 61(2):379–391

    Article  MathSciNet  MATH  Google Scholar 

  20. Zheng Y, Heagerty PJ (2007) Prospective accuracy for longitudinal markers. Biometrics 63(2):332–341

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

Authors are grateful to the Editor, Associate Editor, and reviewers for their helpful comments. Authors acknowledge the Texas Advanced Computing Center (TACC) at The University of Texas at Austin for providing computational resources that have contributed to the research results reported within this paper.

Funding

This work was partially supported by awards from the National Institutes of Health (Grant R01DK117209 to Li, and U01CA230997 and U24086368 to Ning).

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to **g Ning or Ruosha Li.

Ethics declarations

Conflict of interest

All authors declare that they have no conflict of interest or financial ties to disclose.

Supplementary Information

Below is the link to the electronic supplementary material.

(PDF 287 kb)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, J., Ning, J. & Li, R. Evaluating Dynamic Discrimination Performance of Risk Prediction Models for Survival Outcomes. Stat Biosci 15, 353–371 (2023). https://doi.org/10.1007/s12561-023-09362-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12561-023-09362-0

Keywords

Navigation