Abstract
Acute kidney injury (AKI) is a common postoperative complication among patients in the neurological intensive care unit (NICU), often resulting in poor prognosis and high mortality. In this retrospective cohort study, we established a model for predicting AKI following brain surgery based on an ensemble machine learning algorithm using data from 582 postoperative patients admitted to the NICU at the Dongyang People's Hospital from March 1, 2017, to January 31, 2020. Demographic, clinical, and intraoperative data were collected. Four machine learning algorithms (C5.0, support vector machine, Bayes, and XGBoost) were used to develop the ensemble algorithm. The AKI incidence in critically ill patients after brain surgery was 20.8%. Intraoperative blood pressure; postoperative oxygenation index; oxygen saturation; and creatinine, albumin, urea, and calcium levels were associated with the postoperative AKI occurrence. The area under the curve value for the ensembled model was 0.85. The accuracy, precision, specificity, recall, and balanced accuracy values were 0.81, 0.86, 0.44, 0.91, and 0.68, respectively, indicating good predictive ability. Ultimately, the models using perioperative variables exhibited good discriminatory ability for early prediction of postoperative AKI risk in patients admitted to the NICU. Thus, the ensemble machine learning algorithm may be a valuable tool for forecasting AKI.
Similar content being viewed by others
Introduction
Acute kidney injury (AKI) is a common complication among neurocritical patients. According to the Kidney Disease: Improving Global Outcomes (KDIGO) study, the reported incidence of AKI is 11.6% in neurocritical patients, with approximately 3–12% of patients requiring hemodialysis1. Postoperative AKI is a major cause of poor outcomes after brain surgery. Many studies2,3,4 have confirmed that the occurrence of AKI is directly related to patient outcomes and results in a longer duration of postoperative mechanical ventilation, prolongation of the intensive care unit (ICU) and overall hospital stays, increased severity of disability, and significant increases in hospital mortality. The clinical diagnosis of AKI is largely dependent on serum creatinine (sCr) and urine output5. However, an increase in sCr usually indicates severely impaired renal function, and neurocritical patients often develop pituitary disorders, such as diabetes insipidus, leading to obvious abnormalities in urine output and consequently resulting in the inability to accurately diagnose AKI. Therefore, much research effort has been focused on the early and accurate identification of AKI in high-risk patients6.
Artificial intelligence (AI) and machine learning (ML) are regarded as highly valuable tools that can assist in preoperative planning, intraoperative guidance, and outcome prediction for patients undergoing neurosurgery7. Considerable research8,9,10 has focused on the application of ML methods in the prediction of postoperative mortality, recurrence, and adverse events in patients with brain tumors, neurovascular diseases, and traumatic brain injury. In recent years, ML has also been widely applied for the prediction of AKI in critically ill patients. For instance, He et al. extracted 76,957 encounters and relevant clinical variables from the electronic medical record system of a hospital to establish models for predicting AKI in general hospital populations using five ML methods11. Their results indicated that the constructed models provided good predictive performance. Although AI-based ML algorithms have shown promise for predicting AKI risk in several studies, the predictive performance of such models must be improved prior to widespread clinical application. Ensemble modeling, a relatively new approach that combines multiple ML models to enhance model accuracy, may be of value given recent evidence across medical specialties12,13,14.
Currently, there are few universal models for predicting the risk of AKI after brain surgery, making it difficult to utilize the large amounts of data available in modern electronic health record systems. Therefore, this study aimed to establish an ensemble learning-based predictive model using a large number of preoperative, intraoperative, and postoperative ICU variables, including intraoperative vital signs, medication information, and intake and output data. Improved accuracy in postoperative AKI prediction may aid in early evaluation of the patient’s risk level, assist in clinical decision-making, and promote timely intervention.
Methods
Study design and research participants
Our study was reported in accordance with the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) statement15 (Supplementary Table S1). This study was conducted in accordance with the tenets of the Declaration of Helsinki. Data were retrospectively analyzed for 582 postsurgical patients admitted to the ICU at the Dongyang People’s Hospital between March 1, 2017, and January 31, 2020. The inclusion criteria were as follows: (1) age ≥ 18 years and (2) ICU admission after brain surgery. The exclusion criteria were as follows: (1) chronic kidney disease, (2) AKI prior to surgery, (3) length of ICU stay < 24 h, and (4) percentage of missing data > 20%.
Data collection
Data were collected using a medical record information mining software program provided by Shanghai LeJiu Healthcare Technology Co., Ltd (Shanghai, China). The collected data included the following: (1) demographic data: age, sex, diagnosis, comorbidities; (2) surgical data: operative time, type of surgery, intraoperative intake and output, intraoperative urine output, intraoperative vital signs (initial, maximum, minimum, and mean values), intraoperative blood transfusion data, intraoperative medication data; (3) blood gas and biochemical indicators 24 h after ICU admission; (4) use of drugs: commonly used diuretics, including mannitol, furosemide, and torasemide, and contrast media; and (5) clinical outcomes, including hospital mortality, duration of mechanical ventilation, and length of ICU stay.
This study was conducted in accordance with all related local guidelines and regulations, including those related to human genetics. This study was approved by the Ethics Committee of the Dongyang People’s Hospital (Dong Ren Yi 2021-YX-021), which waived the requirement for informed consent due to the retrospective and observational study design. The data were anonymized before analysis.
Diagnostic criteria
AKI was defined and staged in accordance with the 2012 KDIGO criteria16: increases in sCr ≥ 50% or an sCr increase of at least 26.5 mol/L within 2 days. Chronic kidney disease was defined as a baseline estimated glomerular filtration rate < 60 mL/min/1.73 m2. The diagnosis of AKI was based solely on the serum creatinine levels according to the KDIGO criteria. Because of the retrospective nature of the study, we did not have accurate data on urine output. Baseline sCr was defined as the lowest sCr level within 6 months before ICU admission17. For patients without previous sCr data, it was estimated using the following formula18: sCr = 0.74–0.2 (if female) + 0.08 (if black) + 0.0039 × age (in years).
Data processing
Selection of independent variables
A total of 119 variables were preliminarily selected based on their potential to influence AKI as demonstrated in the literature and clinical practice. First, variables with missing data > 20% were eliminated, after which 93 variables remained. Data preprocessing was then performed using the caret package in R language19 (R Software for Statistical Computing, Vienna, Austria) to remove variables strongly correlated with other variables, after which 80 variables remained. Subsequently, the backward feature selection method, random forest sampling, and tenfold cross-validation were adopted. After variable performance (error, forecast accuracy) had been calculated and the variables had been ranked in order of importance, the 10 most important variables were ultimately retained.
Outliers were detected using the interquartile range (IQR) (i.e., the difference between the upper and lower quartiles of the boxplot). The outliers were excluded and treated as missing values. Variables with > 20% missing values were deleted. The missing values of the variables were replaced using multiple imputations.
Model construction
Four different models were first constructed using the C5.0, support vector machine (SVM), Bayes, and extreme gradient boosting (XGBoost) ML algorithms. Hyperparameter tuning was performed using the grid search method (Supplementary Figs. S1–3). Then, the four models were ensembled using the caretEnsemble package14 in R software. The resultant ensemble model was mainly trained using the following R packages: CARET, Ranger, arm, e1071, naivebayes, and gbm. The samples were randomly assigned to the training dataset for model training and test dataset for model validation in a 7:3 ratio. Supplementary Table S2 shows the comparison of baseline data and outcomes between the two datasets. All ML models were evaluated via tenfold cross-validation.
Model validation and evaluation
The performance of each model was evaluated using the area under the receiver operating characteristic curve (AUC). The calibration performance of each model was evaluated using calibration curves. The confusion matrix was evaluated based on accuracy, precision, specificity, recall, and balanced accuracy using a cut-off of 0.5.
Models were interpreted based on variable importance, which was sorted using the function “varImp” within the CARET package in R. A locally weighted regression technique was used to detect the cut-off values of the top four features that predicted AKI. Moreover, iBreakdown algorithms were used for individual interpretation20.
Statistical analysis
Descriptive statistical analyses were performed using the CBCgrps package in R21. Normally distributed measurement data were expressed as x ± s and compared between the groups using two-independent-samples t-tests. Meanwhile, non-normally distributed data were expressed as medians (P25, P75) and compared using the Mann–Whitney U tests. Enumeration data were expressed in terms of the rate and percentage and compared between groups using the χ2 test. All statistical analyses were performed using R software version 4.1.2. A P-value < 0.05 was considered significant.
Approval for human experiments
This study was approved by the Ethics Committee of Dongyang People’s Hospital (Dong Ren Yi 2021-YX-021).
Informed consent
The requirement for informed consent was waived due to the retrospective and observational study design.
Results
Comparison of baseline data
A total of 582 postsurgical neurocritical patients were included in this study. Figure 1 shows the flowchart of the participant selection process. Among the included patients, the incidence of AKI was 20.8% (121 patients). The numbers of patients with Stage I, II, and III AKI were 108 (89.2%), three (2.5%), and 10 (8.3%), respectively. Table 1 shows the comparison of baseline clinical data and clinical outcomes between the AKI and non-AKI groups. Age and sex did not significantly differ between the groups, with the mean age and proportion of men among patients being 56.4 ± 15.2 years and 58%, respectively. The AKI group had a significantly higher mean APACHE score. Overall, 112 of the 582 patients died (overall hospital mortality: 19.2%). While mortality was higher in the AKI group than in the non-AKI group, the difference was not statistically significant (25% vs 18%, P = 0.082). Significant differences were observed for the duration of mechanical ventilation, length of ICU stay, length of hospital stay, and total cost of hospitalization (P < 0.001), with outcomes being poorer in the AKI group.
Comparison of surgical data
Among the 582 postsurgical patients included in this study, 517 (87.4%) underwent emergency surgery. Surgery was performed in the subdural space in 168 patients (28.9%), epidural space in 39 patients (6.7%), supratentorial lesions in 178 patients (30.6%), cranial base and posterior cranial fossa in 23 patients (3.9%), and other sites in 174 patients (29.9%). Table 2 shows the comparison of intraoperative data between the AKI and non-AKI groups.
Evaluation of ML models
ML models were established using the C5.0, SVM, Bayes, and XGBoost ML algorithms and subsequently ensembled. Figure 2 shows the relative influences of the individual ML models on the ensemble model, demonstrating that the Bayes and XGBoost models exerted the greatest influence. Figure 3 shows the receiver operating characteristic curves and calibration plots for the five different predictive models for the validation dataset. The AUC value was greatest for the ensemble model (0.85, 95% CI 0.78–0.91) followed by the SVM model (0.84, 95% CI 0.78–0.91). All models, except C5.0, exhibited good predictive performance. Table 3 shows the confusion matrix for the predictive models. The accuracy, precision, specificity, recall and balanced accuracy values of the ensemble model were 0.81, 0.86, 0.44, 0.91, and 0.68, respectively.
Supplementary Figure S4 shows the crude relationships between the top four features and AKI. Figure 4 and Supplementary Fig. S5 show the order of importance for variables in the five ML models. The creatinine level, oxygenation index, and albumin level on ICU admission and the intraoperative mean systolic blood pressure exerted the greatest influence on the models. Figure 5 shows the iBreakdown results regarding the contribution of each variable to the probability of AKI for Patient 1. The analysis indicated that an initial sCr = 79 was closely correlated with a reduced risk of AKI.
Discussion
In the present study, approximately one in five patients admitted to the ICU following brain surgery developed AKI, which is consistent with previous results22. Given the availability of considerable amounts of perioperative data, we adopted four commonly used ML methods to develop models for the early prediction of postoperative AKI. When the four models were ensembled, the resultant model exhibited better predictive ability, achieving an AUC of approximately 0.85, as well as satisfactory values for accuracy and recall.
C5.0, SVM, Bayes, and XGBoost are the most common ML algorithms. The Bayes method provides the basis for probability learning methods, while the SVM is commonly used for classification. Both C5.0 and XGBoost are classic ML algorithms. These algorithms have good prediction effect for binary classification problems. However, the order of importance for variables differed for the five ML models. For example, the creatinine level on ICU admission played a major role in the C5.0 model. In the confusion matrix, the C5.0 model exhibited the best performance, with the highest accuracy, precision, specificity, and balanced accuracy values. However, the AUC value for the C5.0 model had poor performance and the worst calibration. The accuracy values for the SVM, Bayes, and ensemble models were similar, although the balanced accuracy and specificity values were higher for the ensemble model.
Similar to the present study, Adhikari et al.23 utilized intraoperative physiologic data, laboratory indicators, and medication information from 2,911 adult surgical patients to develop a model for predicting postoperative AKI, which exhibited an AUC of 0.86. The authors also noted that incorporating intraoperative physiologic time series data enabled better identification of patients at risk for AKI. Lei et al.24 adopted ML algorithms for the prediction of postoperative AKI using data from 42,615 patients who underwent major non-cardiac surgery. Their results indicated that the inclusion of preoperative and intraoperative data increased the AUC of the gradient boosting machine from 0.712 to 0.804. Our results support the notion that ensemble models based on perioperative data can promote early and accurate prediction of post-brain surgery AKI.
In the present study, the following variables were ranked highest in importance among the five ML models: initial creatinine level, urea level on ICU admission, intraoperative blood pressure, postoperative oxygenation index, and serum albumin level. Although the cause–effect relationships between each of these factors and postoperative AKI cannot be elucidated owing to the black-box nature of ML algorithms, the presence of significant correlations is certain. Therefore, we used a locally weighted scatterplot smoothing technique to assess the crude relationships between the top four features and AKI. The postoperative oxygenation index was negatively associated with AKI, with lower oxygenation values corresponding to higher AKI risk. The analysis indicated that the risk of AKI was lowest at a creatinine level of 40 μmoI/L, an albumin level of 39 mmol/l, and an average systolic pressure of 116 mmHg. Common causes of AKI include infection, hypovolemia, and hypoperfusion, which tend to cause hypoxia. Therefore, hypoxia is regarded as a frequent pathophysiological manifestation of AKI25. Arterial blood pressure is also closely related to AKI following high-risk surgery26. In a previous study, predictive models for postoperative AKI in the ICU were established using preoperative variables and intraoperative blood pressure27, yielding good predictive effects. Other studies have reported a close relationship between serum albumin level and the occurrence of postoperative AKI28,29. For instance, one retrospective cohort study reported that a low serum albumin level immediately after major abdominal surgery increased the risk of postoperative AKI30. Therefore, monitoring and control of intraoperative blood pressure, postoperative oxygenation index, and serum albumin level may aid in preventing AKI and provide guidance for effective clinical treatment.
The present study has certain limitations. First, this was a single-center retrospective study, in which samples were randomly divided into the training and test datasets in a 7:3 ratio. The lack of validation using external data therefore limits the applicability of the model results. Second, the ML algorithms used in this study may exhibit poor interpretability. To address this issue, the variables included in the models were ranked in the order of importance, and the iBreakdown algorithm was used for direct interpretations in individual patients. Third, given that the ratio between patients with and without AKI was 1:4, the dataset used for the present study was imbalance. However, we attempted to account for this using accuracy, precision, specificity, recall, and balanced accuracy values in addition to the common ROC model. Finally, the models constructed in the present study are applicable only for predicting the risk of AKI occurrence during the ICU stay. In future studies, the possibility of AKI occurring 24 or 48 h postoperatively should be investigated using preoperative and intraoperative variables to enhance the predictability and temporal validity of the models.
In conclusion, our study showed that AKI is a prevalent condition affecting neurocritical patients. Through utilizing a comprehensive suite of perioperative variables, we have successfully developed an ensemble machine learning model that exhibits excellent predictive ability in forecasting AKI following brain surgery. However, future prospective studies will be necessary to validate the results of this study.
Data availability
The data are available from the corresponding author on reasonable request. The codes are available at: https://github.com/fzs1412/Ensemble-Machine-Learning-Algorithm.git.
References
Ramírez-Guerrero, G., Baghetti-Hernández, R. & Ronco, C. Acute kidney injury at the neurocritical care unit. Neurocrit. Care 36, 640–649 (2022).
Büttner, S. et al. Incidence, risk factors, and outcome of acute kidney injury in neurocritical care. J. Intensive Care Med. 35, 338–346 (2020).
Deng, Y. et al. The incidence, risk factors and outcomes of postoperative acute kidney injury in neurosurgical critically ill patients. Sci. Rep. 7, 4245 (2017).
Kovacheva, V. P. et al. Acute kidney injury after craniotomy is associated with increased mortality: a cohort study. Neurosurgery 79, 389–396 (2016).
Chawla, L. S. et al. Acute kidney disease and renal recovery: Consensus report of the Acute Disease Quality Initiative (ADQI) 16 Workgroup. Nat. Rev. Nephrol. 13, 241–257 (2017).
Zhang, Z. Machine learning method for the management of acute kidney injury: more than just treating biomarkers individually. Biomark. Med. 13, 1251–1253 (2019).
Senders, J. An introduction and overview of machine learning in neurosurgical care. Acta Neurochir. 160, 29–38 (2018).
Abujaber, A. et al. Prediction of in-hospital mortality in patients on mechanical ventilation post traumatic brain injury: Machine learning approach. BMC Med. Inform. Decision Mak. 20, 1–11 (2020).
Guo, Y. et al. Distinguishing focal cortical dysplasia from glioneuronal tumors in patients with epilepsy by machine learning. Front. Neurol. 11, 548305 (2020).
He, X. W., Du, C. N., Zhao, K., Yang, M. F. & Ma, Q. F. A novel model for predicting the outcome of intracerebral hemorrhage: Based on 1186 patients. J. Stroke Cerebrovasc. Dis. 29, 104867 (2020).
He, J. et al. Multi-perspective predictive modeling for acute kidney injury in general hospital populations using electronic medical records. JAMIA Open. 2, 115–122 (2019).
Muhlestein, W. E., Akagi, D. S., Davies, J. M. & Chambless, L. B. Predicting inpatient length of stay after brain tumor surgery: Develo** machine learning ensembles to improve predictive performance. Neurosurgery 85, 384–393 (2019).
Bannick, M. S., McGaughey, M. & Flaxman, A. D. Ensemble modelling in descriptive epidemiology: burden of disease estimation. Int. J. Epidemiol. 49, 2065–2073 (2021).
Zhang, Z., Chen, L., Xu, P. & Hong, Y. Predictive analytics with ensemble modeling in laparoscopic surgery: a technical note. Laparoscop. Endoscop. Robotic Surg. 5, 25–34 (2022).
Collins, G. S., Reitsma, J. B., Altman, D. G. & Moons, K. G. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): The TRIPOD statement. J. Brit. Surg. 102, 148–158 (2015).
Khwaja, A. KDIGO Clinical practice guidelines for acute kidney injury. Nephron. Clin. Pract. 120, 179–184 (2012).
Shen, Y., Zhang, W., Cheng, X. & Ying, M. Association between postoperative fluid balance and acute kidney injury in patients after cardiac surgery: A retrospective cohort study. J. Crit. Care. 44, 273–277 (2018).
Závada, J. et al. A comparison of three methods to estimate baseline creatinine for RIFLE classification. Nephrol. Dialys. Transpl. 25, 3911–3918 (2010).
Kuhn, M. Caret: classification and regression training. Astrophysics Source Code Library, ascl-1505 (2015).
Staniak M, Biecek P. Explanations of model predictions with live and breakDown packages. The R Foundation (2019).
Zhang, Z., Gayle, A. A., Wang, J., Zhang, H. & Cardinal-Fernandez, P. Comparing baseline characteristics between groups: An introduction to the CBCgrps package. Ann. Transl. Med. 5, 484 (2017).
Zorrilla-Vaca, A. et al. Acute kidney injury following acute ischemic stroke and intracerebral hemorrhage: A meta-analysis of prevalence rate and mortality risk. Cerebrovasc. Dis. 45, 1–9 (2018).
Adhikari, L. et al. Improved predictive models for acute kidney injury with IDEA: Intraoperative data embedded analytics. PLoS ONE 14, e0214904 (2019).
Lei, V. J. Risk stratification for postoperative acute kidney injury in major noncardiac surgery using preoperative and intraoperative data. JAMA Netw. Open. 2, e1916921 (2019).
Basile, D. P., Anderson, M. D. & Sutton, T. A. Pathophysiology of acute kidney injury. Comprehens. Physiol. 2, 1303–1353 (2012).
Greco, M., Falini, S., Angelotti, G. & Cecconi, M. Association of mean arterial pressure and acute kidney injury after high risk surgery. Intensive Care Medicine Experimental Conference: 32nd European Society of Intensive Care Medicine Annual Congress, ESICM. 7 (2019).
Bihorac, A. Database communication enables machine learning classifiers to predict postoperative AKI in ICU. Crit. Care Med. 41, A176 (2013).
Thongprayoon, C., Cheungpasitporn, W., Mao, M. A., Sakhuja, A. & Kashani, K. U-shape association of serum albumin level and acute kidney injury risk in hospitalized patients. PLoS ONE 13, e0199153 (2018).
Shao, M., Wang, S. & Parameswaran, P. K. Hypoalbuminemia: a risk factor for acute kidney injury development and progression to chronic kidney disease in critically ill patients. Int. Urol. Nephrol. 49, 295–302 (2017).
Li, W., Li, N. & Li, S. Relationship between postoperative immediate serum albumin level and postoperative acute kidney injury after major abdominal surgery in critically ill patients. Zhonghua Wei Zhong Bing Ji Jiu Yi Xue 33, 955–961 (2021).
Acknowledgements
We would like to thank Editage (www.editage.cn) for English language editing. This research was supported by the Zhejiang Medical Association (Grant Nos. 2020ZYC-B44, and 2019ZYC-A172) and the **hua Science and Technology Bureau (grant numbers 2022-3-027).
Author information
Authors and Affiliations
Contributions
M.Y.W. designed the study, X.D.J and K.L.D. performed the data extraction, analyzed and drafted the manuscript. Y.T.X. critically revised the manuscript. W.M.Z. supervised the study. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Wu, M., Jiang, X., Du, K. et al. Ensemble machine learning algorithm for predicting acute kidney injury in patients admitted to the neurointensive care unit following brain surgery. Sci Rep 13, 6705 (2023). https://doi.org/10.1038/s41598-023-33930-5
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-023-33930-5
- Springer Nature Limited