Abstract
Risk prediction models for survival outcomes are widely applied in medical research to predict future risk for the occurrence of the event. In many clinical studies, the biomarker data are measured repeatedly over time. To facilitate timely disease prognosis and decision making, many dynamic prediction models have been developed and generate predictions on a real-time basis. As a dynamic prediction model updates an individual’s risk prediction over time based on new measurements, it is often important to examine how well the model performs at different measurement times and prediction times. In this article, we propose a two-dimensional area under curve (AUC) measure for dynamic prediction models and develop associated estimation and inference procedures. The estimation procedures are discussed under two types of biomarker measurement schedules: regular visits and irregular visits. The model parameters are estimated effectively by maximizing a pseudo-partial likelihood function. We apply the proposed method to a renal transplantation study to evaluate the discrimination performance of dynamic prediction models based on longitudinal biomarkers for graft failure.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs12561-023-09362-0/MediaObjects/12561_2023_9362_Fig1_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs12561-023-09362-0/MediaObjects/12561_2023_9362_Fig2_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs12561-023-09362-0/MediaObjects/12561_2023_9362_Fig3_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs12561-023-09362-0/MediaObjects/12561_2023_9362_Fig4_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs12561-023-09362-0/MediaObjects/12561_2023_9362_Fig5_HTML.png)
Similar content being viewed by others
References
Begg CB, Cramer LD, Venkatraman E et al (2000) Comparing tumour staging and grading systems: a case study and a review of the issues, using thymoma as a model. Stat Med 19(15):1997–2014
Chen TK, Knicely DH, Grams ME (2019) Chronic kidney disease diagnosis and management: a review. JAMA 322(13):1294–1304
Harrell FE, Califf RM, Pryor DB et al (1982) Evaluating the yield of medical tests. JAMA 247(18):2543–2546
Heagerty PJ, Zheng Y (2005) Survival model predictive accuracy and ROC curves. Biometrics 61(1):92–105
Li L, Luo S, Hu B et al (2017) Dynamic prediction of renal failure using longitudinal biomarkers in a cohort study of chronic kidney disease. Stat Biosci 9(2):357–378
Li R, Ning J, Feng Z (2022) Estimation and inference of predictive discrimination for survival outcome risk prediction models. Lifetime Data Anal 28(2):219–240
Mayo Clinic (2021) Kidney transplant. https://www.mayoclinic.org/tests-procedures/kidney-transplant/about/pac-20384777. Accessed 1 Nov 2021
Rizopoulos D (2012) Joint models for longitudinal and time-to-event data: with applications in R. CRC Press, Boca Raton
Rizopoulos D, Ghosh P (2011) A Bayesian semiparametric multivariate joint model for multiple longitudinal outcomes and a time-to-event. Stat Med 30(12):1366–1380
Saha-Chaudhuri P, Heagerty P (2010) Time-dependent predictive accuracy in the presence of competing risks. Biometrics 66(4):999–1011
Saha-Chaudhuri P, Heagerty P (2012) Non-parametric estimation of a time-dependent predictive accuracy curve. Biostatistics 14(1):42–59
Santos J, Martins LS (2015) Estimating glomerular filtration rate in kidney transplantation: still searching for the best marker. World J Nephrol 4(3):345
Schoop R, Graf E, Schumacher M (2008) Quantifying the predictive performance of prognostic models for censored survival data with time-dependent covariates. Biometrics 64(2):603–610
Thisted RA (2017) Elements of statistical computing: numerical computation. Routledge, New York
Uno H, Cai T, Pencina MJ et al (2011) On the C-statistics for evaluating overall adequacy of risk prediction procedures with censored survival data. Stat Med 30(10):1105–1117
Van Houwelingen HC (2007) Dynamic prediction by landmarking in event history analysis. Scand J Stat 34(1):70–85
Zhang J, Ning J, Huang X et al (2021) On the time-varying predictive performance of longitudinal biomarkers: measure and estimation. Stat Med 40(23):5065–5077
Zheng Y, Heagerty PJ (2004) Semiparametric estimation of time-dependent ROC curves for longitudinal marker data. Biostatistics 5(4):615–632
Zheng Y, Heagerty PJ (2005) Partly conditional survival models for longitudinal data. Biometrics 61(2):379–391
Zheng Y, Heagerty PJ (2007) Prospective accuracy for longitudinal markers. Biometrics 63(2):332–341
Acknowledgements
Authors are grateful to the Editor, Associate Editor, and reviewers for their helpful comments. Authors acknowledge the Texas Advanced Computing Center (TACC) at The University of Texas at Austin for providing computational resources that have contributed to the research results reported within this paper.
Funding
This work was partially supported by awards from the National Institutes of Health (Grant R01DK117209 to Li, and U01CA230997 and U24086368 to Ning).
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of interest
All authors declare that they have no conflict of interest or financial ties to disclose.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhang, J., Ning, J. & Li, R. Evaluating Dynamic Discrimination Performance of Risk Prediction Models for Survival Outcomes. Stat Biosci 15, 353–371 (2023). https://doi.org/10.1007/s12561-023-09362-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12561-023-09362-0