Abstract
Objective
To construct and validate an update of the Simplified Acute Physiology Score II (SAPS II) for the evaluation of clinical performance of Intensive Care Units (ICU).
Design and setting
Retrospective analysis of prospectively collected multicenter data in 32 ICUs located in the Paris area belonging to the Cub-Rea database and participating in a performance evaluation project.
Patients
33,471 patients treated between 1999 and 2000.
Measurements and results
Two logistic regression models based on SAPS II were developed to estimate in-hospital mortality among ICU patients. The second model comprised reevaluation of original items of SAPS II and integration of the preadmission location and chronic comorbidity. Internal and external validation were performed. In the two validation samples the most complex model had better calibration than the original SAPS II for in-hospital mortality but its discrimination was not significantly higher (area under ROC curve 0.89 vs. 0.87 for SAPS II). Second-level customization and integration of new items improved uniformity of fit for various categories of patients except for diagnosis-related groups. The rank order of ICUs was modified according to the model used.
Conclusions
The overall performance of SAPS II derived models was good, even in the context of a community cohort and routinely gathered data. However, one-half the variation of outcome remains unexplained after controlling for admission characteristics, and uniformity of prediction across diagnostic subgroups was not achieved. Differences in case-mix still limit comparisons of quality of care.
Similar content being viewed by others
References
Knaus WA, Wagner DP, Draper EA, Zimmerman JE, Bergner M, Bastos PG, Sirio CA, Murphy DJ, Lotring T, Damiano A (1991) The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest 100:1619–1636
Le Gall JR, Lemeshow S, Saulnier F (1993) A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study. JAMA 270:2957–2963
Lemeshow S, Teres D, Klar J, Avrunin JS, Gehlbach SH, Rapoport J (1993) Mortality probability models (MPM II) based on an international cohort of intensive care unit patients. JAMA 270:2478–2486
Teres D, Lemeshow S (1999) When to customise a severity model. Intensive Care Med 25:140–142
Rowan KM, Kerr JH, Major E, McPherson K, Short A, Vessey MP (1994) Intensive Care Society’s Acute Physiology and Chronic Health Evaluation (APACHE II) study in Britain and Ireland: a prospective, multicenter, cohort study comparing two methods for predicting outcome for adult intensive care patients. Crit Care Med 22:1392–1401
Moreno R, Apolone G, Reis Miranda D (1998) Evaluation of the uniformity of fit of general outcome prediction models. Intensive Care Med 24:40–47
Pappachan JV, Millar B, Bennett ED, Smith GB (1999) Comparison of outcome from intensive care admission after adjustment for case mix by the Apache III prognostic system. Chest 115:802–810
Metnitz PG, Lang T, Vesely H, Valentin A, LeGall JR (2000) Ratios of observed to expected mortality are affected by differences in case mix and quality of care. Intensive Care Med 26:1466–1472
Moreno R, Agthé D (1999) ICU discharge decision-making: are we able to decrease post-ICU mortality? Intensive Care Med 25:1035–1036
Beck DH, Smith GB, Papachan JV (2002) The effect of two methods for customising the original SAPS II model for intensive care patients from south England: a multicentre study. Anaesthesia 57:785–793
Apolone G (2000) The state of research on multipurpose severity of illness scoring systems: are we on target? Intensive Care Med 26:1727–1729
Moreno R, Matos R (2001) New issues in severity scoring: interfacing the ICU and evaluating it. Curr Opin Crit Care 7:469–474
Boumendil A, Guidet B, Aegerter P (2003) Performance of intensive care units: updating the simplified acute physiology score II. Intensive Care Med 29:S1–S127
Annane D, Aegerter P, Jars-Guincestre MC, Guidet B, CUB-Rea network (2003) Current epidemiology of septic shock: the CUB-Rea network. Am J Respir Crit Care Med 168:165–172
Aegerter P, Auvert B, Buonamico G, Sznajder M, Beauchet A, Guidet B, LeGall JR, CUB-Réa (1998) Mise en oeuvre et évaluation d’une base de données commune aux services de réanimation d’Ile-de-France. Rev Epidemiol Sante Publique 46:226–237
Altman DG, Royston P (2000) What do we mean by validating a prognostic model? Stat Med 19:453–473
Black N, Payne M, on behalf of the DoCDat Development Group (2003) Directory of clinical databases: improving and promoting their use. Qual Saf Health Care 12:348–352
Moreno R, Apolone G (1997) The impact of different customization strategies in the performance of a general severity score. Crit Care Med 25:2001–2008
Deyo RA, Cherkin DC (1992) Adapting a clinical morbidity index for use with ICD9-CM administrative database. J Clin Epidemiol 45:613–619
Angus DC, Musthafa AA, Clermont G, Griffin MF, Linde-Zwirble WT, Dremsizov TT, Pinsky MR (2001) Quality-adjusted survival in the first year after the acute respiratory distress syndrome. Am J Respir Crit Care Med 163:1389–1394
Hosmer DW, Lemeshow S (1989) Applied logistic regression. Wiley, New York–
Hanley J, McNeil B (1983) A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 148:839–843
Aegerter P, Muller F, Nakache JP, Boué A (1994) Evaluation of screening methods for Down’s syndrome using bootstrap comparison of ROC curves. Comput Methods Programs Biomed 43:151–157
Nagelkerke NJ (1991) A note on the general definition of the coefficient of determination. Biometrika 78:691–692
Glance LG, Osler TM, Dick AW (2002) Identifying quality outliers in a large, multiple-institution database by using customized versions of the Simplified Acute Physiology Score II and the mortality model II0. Crit Care Med 30:1995–2002
Markgraf R, Deutschinoff G, Pientka L, Scholten T, Lorenz C (2001) Performance of the score systems Apache II and III at an interdisciplinary intensive care unit, after customization. Crit Care 5:31–36
De Keizer NF, Bonsel GJ, Goldfad C, Rowan KM (2000) The added value that increasing levels of diagnostic information provide in prognostic models to estimate hospital mortality for adult intensive care patients. Intensive Care Med 26:577–584
Bronskill SE, Normand SL, Landrum MB, Rosenheck RA (2002) Longitudinal profiles of health care providers. Stat Med 21:1067–1088
Beck DH, Smith GB, Pappachan JV, Millar B (2003) External validation of the SAPS II, APACHE II and APACHE III prognostic models in south England: a multicentre study. Intensive Care Med 29:249–256
Zhu BP, Lemeshow S, Hosmer DW, Klar J, Teres D (1996) Factors affecting the performance of the models in the mortality probability model II system and strategies of customization: a simulation study. Crit Care Med 24:57–63
Minvielle E, Dervaux B, Retbi A, Aegerter P, Boumendil A, Jars-Guincestre MC, Tenaillon A, Guidet B (2005) Culture organization and management in Intensive Care (COMIC). Construction and validation of a multidimensional questionnaire. J Crit Care (in press)
Normand SL, Glickman ME, Gatsonis CA (1997) Statistical methods for profiling providers of medical care: issues and applications. J Am Stat Assoc 92:803–814
Glance LG, Dick AW, Osler TM, Mukamel D (2003) Using hierarchical modeling to measure ICU quality. Intensive Care Med 29:2223–2229
Gibbs J, Clark K, Khuri S, Henderson W, Hur K, Daley J (2001) Validating risk-adjusted outcomes: chart review of process of care. Int J Qual Health Care 13:187–196
Mant J, Hicks N (1995) Detecting differences in quality of care: the sensitivity of measures of process and outcome in treating acute myocardial infarction. BMJ 311:793–796
Acknowledgements
The authors thank the members of 25 ICUs of Cub-Rea for providing support. Reviewers are acknowledged for their constructive comments. This work was partly presented at the 16th Annual Congress of the European Society of Intensive Care Medicine in Amsterdam, 5–8 October 2003 [13].
Author information
Authors and Affiliations
Corresponding author
Additional information
This research was financed by The Clinical Research Program of the French Ministry of Health (PHRC AOM 98-124). ICUs participating in the study are listed in the Appendix
Electronic Supplementary Material
Appendix: members of the CUB-REA database
Appendix: members of the CUB-REA database
F. Jardin, B Page (Hôpital Ambroise Paré), J.P. Bedos, P. Guezennec (Hôpital André Mignot), F. Brivet (Hôpital Antoine Béclère), Y. Cohen, J.P. Fosse (Hôpital Avicenne), C.l. Gibert, B. Regnier, P. Auburtin (Hôpital Bichat), C. Richard, J. Depré-Vassal (Hôpital Bicêtre), J. Labrousse, E. Guerot (Hôpital Boucicaut), J.Y. Fagon (Hôpital Européen Georges Pompidou), J.F. Dhainaut, A. Cariou (Hôpital Cochin), F. Fraisse, G. Moret (Hôpital Delafontaine), P. Kalfon (Hôpital Diaconesses), F. Blin (Hôpital Gonesse), F. Lemaire, C. Brun-Buisson (Hôpital Henri Mondor), A. Rabbat (Hôpital Hotel Dieu), G. Nitenberg, F. Blot (Institut Gustave Roussy), J.L. Pourriat, R. Gauzit (Hôpital Jean Verdier), F. Baud, D. Goldgran-Toledano (Hôpital Lariboisière), D. Dreyfuss (Hôpital Louis Mourier), A. Tenaillon (Hôpital Louise Michel), J.L. Pallot, E. Obadia (Hôpital Montreuil), J.M. Coulaud, L. Donetti (Hôpital Montfermeil), H. Bismuth (Hôpital Paul Brousse), T. Similowski (Hôpital Pitié-Salpétrière), F. Bolgert (Hôpital Pitié-Salpétrière, H. Outin (Medical ICU, Hôpital Poissy/St. Germain), J.P. Terville (Surgical ICU, Hôpital Poissy/St. Germain), P. Gajdos, M.C. Jars-Guincestre (Hôpital Raymond Poincaré), F. Hilpert, P. Manet (Hôpital Robert Ballanger), G. Offenstadt, B. Guidet (Hôpital Saint-Antoine), J. Carlet, B. Misset (Hôpital Saint-Joseph), J.R. Le Gall, G. Leleu (Medical ICU, Hôpital Saint-Louis), L. Jacob (Surgical ICU, Hôpital Saint-Louis), C. Mayaud, A. Parrot (Hôpital Tenon), G. Bleichner, H. Mentec (Hôpital Victor Dupouy).
Rights and permissions
About this article
Cite this article
Aegerter, P., Boumendil, A., Retbi, A. et al. SAPS II revisited. Intensive Care Med 31, 416–423 (2005). https://doi.org/10.1007/s00134-005-2557-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00134-005-2557-9