A prediction model for moderate to severe acute kidney injury in people with heart failure

Dear Editor,

Heart failure (HF) is a common multi-faceted and life-threatening syndrome, of which up to 23% occur acute kidney injury (AKI) [1]. HF-related AKI is largely overlooked or delayed in identification [2]. Approximately 85% of AKI cases that occurred during cardiac hospitalization in China were either ignored or identified too late [3]. Currently, there are no specific guidelines for the management of HF-related AKI. Hence, it is essential to identify patients at the risk of developing AKI and intervene promptly, to reduce social and economic burden.

Machine learning (ML)-based prediction models can predict patients at risk or likely to benefit from treatment, due to their superior capability in integrating data with multidimensional interactions. A variety of prediction models have been successfully used in HF patients, however, there remains a lack of widely accepted models for predicting HF-related AKI. Previous models have several critical limitations, including relatively low discrimination, miscellaneous patient selection, small sample sizes, and insufficient predictors [4, 5]. Hence, this study aimed to develop a prediction model of AKI in the HF population using ML algorithms. 

This study developed and validated a prediction model of moderate to severe AKI and AKI requiring dialysis, based on readily available variables from HF patients (Additional file 1: Table S1), using 4 ML algorithms with optimal hyperparameters (Fig. 1a; Additional file 1: Table S2). A total of 120,479 participants from the China Renal Data System (CDRS) database were included as the development cohort and 4327 patients from the local hospital as the external validation cohort. Additional file 1: Fig. S1 outlines the patient selection process for the development and validation cohorts. Additional file 1: Table S3 summarizes the baseline characteristics of the participants. In the development cohort [age 68.0 (58.0, 78.0) years; 71,895 men], 5975 (5.0%) patients developed moderate to severe AKI, and 2956 (2.5%) required dialysis. In the external validation cohort [age 72.0 (63.0, 79.0); 2722 men], patients were older, had worse baseline renal function, more severe HF conditions, and were less likely to take HF-associated medications, had a higher incidence of moderate to severe AKI (7.1%) and a lower risk of AKI requiring dialysis (1.1%), compared to the development cohort. Additional file 1: Table S4 demonstrate the comparison of baseline characteristics among patients in the three cohorts. Potential continuous variables with high multicollinearity and categorized variables with near-zero variance were removed (Additional file 1: Fig. S2).

Fig. 1

Development and validation of heart failure (HF)-related acute kidney injury (AKI) prediction model. a Study design. The area under the receiver operating characteristic curves (AUCs) for moderate to severe AKI (b, d) and AKI requiring dialysis (c, e) in the internal validation and external validation cohorts. f SHapley Additive exPlanation (SHAP) summary plot of the XGBoost model. The plot depicts the dot estimation on the model output of the XGBoost model. Each dot represents an individual patient from the dataset. Red represents the higher SHAP value of specific features; blue represents the lower SHAP value of specific features. The higher the SHAP values, the greater the risk of developing AKI development. LR logistic regression, RF random forest, SVM supported vector machine, XGBoost eXtreme gradient boosting, proBNP pro-brain natriuretic peptide, eGFR estimated glomerular filtration rate, LDH lactate dehydrogenase

The discrimination performance metrics of prediction models in the three cohorts are presented in Additional file 1: Table S5, Fig. S3, and Fig. 1b–e. Compared with the other three prediction models using logistic regression, supported vector machine and random forest, the eXtreme gradient boosting (XGBoost) model had the best predictive performance in discrimination with the highest area under the receiver operating characteristic curve (AUC) of 0.868 and 0.973; accuracy of 87.7% and 93.6%; sensitivity of 61.3% and 92.1%; specificity of 89.0% and 93.7%; positive predictive value (PPV) of 22.5% and 26.8%; negative predictive value (NPV) of 97.8% and 99.8%; for moderate to severe AKI (Fig. 1b) and AKI requiring dialysis (Fig. 1c) in the internal validation cohort; respectively. In the external validation cohort, the XGBoost model also attained the highest AUC of 0.956 and 0.958; accuracy of 95.7% and 99.1%; sensitivity of 43.8% and 30.4%; specificity of 99.6% and 99.8%; PPV of 90.0% and 60.9%; NPV of 95.9% and 99.3%; for moderate to severe AKI (Fig. 1d) and AKI requiring dialysis (Fig. 1e); respectively. These results demonstrated the trade-off of high NPV against low PPV, in order to minimize underreporting and heighten clinicians’ vigilance among high-risk patient groups, considering the life-threatening complications of HF-related AKI.

In addition, the XGBoost model was well-calibrated, as suggested by large calibration slopes (0.898, 1.044), small calibration intercept (0.026, −0.075), and small Brier’s score (0.039, 0.016) for moderate to severe AKI and AKI requiring dialysis in the internal validation cohort, respectively. Similar good calibrations with the calibration slopes of 1.740 and 1.039, the intercept of −0.096 and −0.138, and Brier’s scores of 0.038 and 0.012 were observed in the external validation cohort (Additional file 1: Table S6). It indicated that the model had preserved calibration.

To allow for the interpretation of our model’s predictions, we used SHapley Additive exPlanation values to assess feature importance and identified a feature’s relative contribution to uncover key features. Figure 1f shows the top 20 predictors in the XGBoost model, not only including renal health-related variables, but also several specific medication variables for the treatment of the primary causes of HF, HF symptoms, and HF-related complications, which might not be comparable to the general population.

In conclusion, this study developed, internally and externally validated a novel ML risk model for predicting HF-related AKI, based on a large dataset of Chinese admitted HF patients and uses of easily available variables. It is expected that the model could be integrated into the routine clinic workflow for risk stratification of the HF population and selecting individuals at high risk of AKI, facilitating early kidney-specific care, timely diagnosis, and treatment of HF-related AKI.