Explainable Machine Learning-Based Risk Prediction Model for In-Hospital Mortality after Continuous Renal Replacement Therapy Initiation

Assessing Axillary Lymph Node Burden and Prognosis in cT1-T2 Stage Breast Cancer Using Machine Learning Methods: A Retrospective Dual-Institutional MRI Study

BACKGROUND

Pathological axillary lymph node (pALN) burden is an important factor for treatment decision-making in clinical T1-T2 (cT1-T2) stage breast cancer. Preoperative assessment of the pALN burden and prognosis aids in the individualized selection of therapeutic approaches.

PURPOSE

To develop and validate a machine learning (ML) model based on clinicopathological and MRI characteristics for assessing pALN burden and survival in patients with cT1-T2 stage breast cancer.

STUDY TYPE

Retrospective.

POPULATION

A total of 506 females (range: 24-83 years) with cT1-T2 stage breast cancer from two institutions, forming the training (N = 340), internal validation (N = 85), and external validation cohorts (N = 81), respectively.

FIELD STRENGTH/SEQUENCE

This study used 1.5-T, axial fat-suppressed T2-weighted turbo spin-echo sequence and axial three-dimensional dynamic contrast-enhanced fat-suppressed T1-weighted gradient echo sequence.

ASSESSMENT

Four ML methods (eXtreme Gradient Boosting [XGBoost], Support Vector Machine, k-Nearest Neighbor, Classification and Regression Tree) were employed to develop models based on clinicopathological and MRI characteristics. The performance of these models was evaluated by their discriminative ability. The best-performing model was further analyzed to establish interpretability and used to calculate the pALN score. The relationships between the pALN score and disease-free survival (DFS) were examined.

STATISTICAL TESTS

Chi-squared test, Fisher's exact test, univariable logistic regression, area under the curve (AUC), Delong test, net reclassification improvement, integrated discrimination improvement, Hosmer-Lemeshow test, log-rank, Cox regression analyses, and intraclass correlation coefficient were performed. A P-value <0.05 was considered statistically significant.

RESULTS

The XGB II model, developed based on the XGBoost algorithm, outperformed the other models with AUCs of 0.805, 0.803, and 0.818 in the three cohorts. The Shapley additive explanation plot indicated that the top variable in the XGB II model was the Node Reporting and Data System score. In multivariable Cox regression analysis, the pALN score was significantly associated with DFS (hazard ratio: 4.013, 95% confidence interval: 1.059-15.207).

DATA CONCLUSION

The XGB II model may allow to evaluate pALN burden and could provide prognostic information in cT1-T2 stage breast cancer patients.

LEVEL OF EVIDENCE

3 TECHNICAL EFFICACY: Stage 2.

Prediction of mortality risk in critically ill patients with systemic lupus erythematosus: a machine learning approach using the MIMIC-IV database

OBJECTIVE

Early prediction of long-term outcomes in patients with systemic lupus erythematosus (SLE) remains a great challenge in clinical practice. Our study aims to develop and validate predictive models for the mortality risk.

METHODS

This observational study identified patients with SLE requiring hospital admission from the Medical Information Mart for Intensive Care (MIMIC-IV) database. We downloaded data from Fujian Provincial Hospital as an external validation set. Variable selection was performed using the Least Absolute Shrinkage and Selection Operator (LASSO) regression. Then, we constructed two predictive models: a traditional nomogram based on logistic regression and a machine learning model employing a stacking ensemble approach. The predictive ability of the models was evaluated by the areas under the receiver operating characteristic curve (AUC) and the calibration curve.

RESULTS

A total of 395 patients and 100 patients were enrolled respectively from MIMIC-IV database and the validation cohort. The LASSO regression identified 18 significant variables. Both models demonstrated good discrimination, with AUCs above 0.8. The machine learning model outperformed the nomogram in terms of precision and specificity, highlighting its potential superiority in risk prediction. The SHapley additive explanations analysis further elucidated the contribution of each variable to the model's predictions, emphasising the importance of factors such as urine output, age, weight and alanine aminotransferase.

CONCLUSIONS

The machine learning model provides a superior tool for predicting mortality risk in patients with SLE, offering a basis for clinical decision-making and potential improvements in patient outcomes.

Acute Kidney Injury Prognosis Prediction Using Machine Learning Methods: A Systematic Review

Rationale & Objective

Accurate estimation of in-hospital outcomes for patients with acute kidney injury (AKI) is crucial for aiding physicians in making optimal clinical decisions. We aimed to review prediction models constructed by machine learning methods for predicting AKI prognosis using administrative databases.

Study Design

A systematic review following PRISMA guidelines.

Setting & Study Populations

Adult patients diagnosed with AKI who are admitted to either hospitals or intensive care units.

Search Strategy & Sources

We searched PubMed, Embase, Web of Science, Scopus, and Cumulative Index to Nursing and Allied Health for studies published between January 1, 2014 and February 29, 2024. Eligible studies employed machine learning models to predict in-hospital outcomes of AKI based on administrative databases.

Data Extraction

Extracted data included prediction outcomes and population, prediction models with performance, feature selection methods, and predictive features.

Analytical Approach

The included studies were qualitatively synthesized with assessments of quality and bias. We calculated the pooled model discrimination of different AKI prognoses using random-effects models.

Results

Of 3,029 studies, 27 studies were eligible for qualitative review. In-hospital outcomes for patients with AKI included acute kidney disease, chronic kidney disease, renal function recovery or kidney failure, and mortality. Compared with models predicting the mortality of patients with AKI during hospitalization, the prediction performance of models on kidney function recovery was less accurate. Meta-analysis showed that machine learning methods outperformed traditional approaches in mortality prediction (area under the receiver operating characteristic curve, 0.831; 95% CI, 0.799-0.859 vs 0.772; 95% CI, 0.744-0.797). The overlapping predictive features for in-hospital mortality identified from ≥6 studies were age, serum creatinine level, serum urea nitrogen level, anion gap, and white blood cell count. Similarly, age, serum creatinine level, AKI stage, estimated glomerular filtration rate, and comorbid conditions were the common predictive features for kidney function recovery.

Limitations

Many studies developed prediction models within specific hospital settings without broad validation, restricting their generalizability and clinical application.

Conclusions

Machine learning models outperformed traditional approaches in predicting mortality for patients with AKI, although they are less accurate in predicting kidney function recovery. Overall, these models demonstrate significant potential to help physicians improve clinical decision making and patient outcomes.

Registration

CRD42024535965.

Prediction of successful weaning from renal replacement therapy in critically ill patients based on machine learning

BACKGROUND

Predicting the successful weaning of acute kidney injury (AKI) patients from renal replacement therapy (RRT) has emerged as a research focus, and we successfully built predictive models for RRT withdrawal in patients with severe AKI by machine learning.

METHODS

This retrospective single-center study utilized data from our general intensive care unit (ICU) Database, focusing on patients diagnosed with severe AKI who underwent RRT. We evaluated RRT weaning success based on patients being free of RRT in the subsequent week and their overall survival. Multiple logistic regression (MLR) and machine learning algorithms were adopted to construct the prediction models.

RESULTS

A total of 976 patients were included, with 349 patients successfully weaned off RRT. Longer RRT duration (7.0 vs. 9.6 d, p = 0.002, OR = 0.94), higher serum cystatin C levels (1.2 vs. 3.2 mg/L, p < 0.001, OR = 0.46), and the presence of septic shock (28.1% vs. 41.5%, p < 0.001, OR = 0.63) were associated with reduced likelihood of RRT weaning. Conversely, a positive furosemide stress test (FST) (60.2% vs. 40.7%, p < 0.001, OR = 2.75) and higher total urine volume 3 d before RRT withdrawal (755 vs. 125 mL/d, p < 0.001, OR = 2.12) were associated with an increased likelihood of successful weaning from RRT. Next, we demonstrated that machine learning models, especially Random Forest and XGBoost, achieving an AUROC of 0.95. The XGBoost model exhibited superior accuracy, yielding an AUROC of 0.849.

CONCLUSION

High-risk factors for unsuccessful RRT weaning in severe AKI patients include prolonged RRT duration. Machine learning prediction models, when compared to models based on multivariate logistic regression using these indicators, offer distinct advantages in predictive accuracy.

Impact of the prognostic nutritional index on renal replacement therapy-free survival and mortality in patients on continuous renal replacement therapy

BACKGROUND

The survival of critically ill patients with acute kidney injury (AKI) undergoing continuous renal replacement therapy (CRRT) is highly dependent on their nutritional status.

OBJECTIVES

The prognostic nutritional index (PNI) is an indicator used to assess nutritional status and is calculated as: PNI = (serum albumin in g/dL) × 10 + (total lymphocyte count in/mm3) × 0.005. In this retrospective study, we investigated the correlation between this index and clinical outcomes in critically ill patients with AKI receiving CRRT.

METHODS

We analyzed data from 2076 critically ill patients admitted to the intensive care unit at Changhua Christian Hospital, a tertiary hospital in central Taiwan, between January 1, 2010, and April 30, 2021. All these patients met the inclusion criteria of the study. The relationship between PNI and renal replacement therapy-free survival (RRTFS) and mortality was examined using logistic regression models, Cox proportional hazard models, and propensity score matching. High utilization rate of parenteral nutrition (PN) was observed in our study. Subgroup analysis was performed to explore the interaction effect between PNI and PN on mortality.

RESULTS

Patients with higher PNI levels exhibited a greater likelihood of achieving RRTFS, with an adjusted odds ratio of 2.43 (95% confidence interval [CI]: 1.98-2.97, p-value < 0.001). Additionally, these patients demonstrated higher survival rates, with an adjusted hazard ratio of 0.84 (95% CI: 0.72-0.98) for 28-day mortality and 0.80 (95% CI: 0.69-0.92) for 90-day mortality (all p-values < 0.05), compared to those in the low PNI group. While a high utilization rate of parenteral nutrition (PN) was observed, with 78.86% of CRRT patients receiving PN, subgroup analysis showed that high PNI had an independent protective effect on mortality outcomes in AKI patients receiving CRRT, regardless of their PN status.

CONCLUSIONS

PNI can serve as an easy, simple, and efficient measure of lymphocytes and albumin levels to predict RRTFS and mortality in AKI patients with require CRRT.

Interpretable multiphasic CT-based radiomic analysis for preoperatively differentiating benign and malignant solid renal tumors: a multicenter study

BACKGROUND

To develop and compare machine learning models based on triphasic contrast-enhanced CT (CECT) for distinguishing between benign and malignant renal tumors.

MATERIALS AND METHODS

In total, 427 patients were enrolled from two medical centers: Center 1 (serving as the training set) and Center 2 (serving as the external validation set). First, 1781 radiomic features were individually extracted from corticomedullary phase (CP), nephrographic phase (NP), and excretory phase (EP) CECT images, after which 10 features were selected by the minimum redundancy maximum relevance method. Second, random forest (RF) models were constructed from single-phase features (CP, NP, and EP) as well as from the combination of features from all three phases (TP). Third, the RF models were assessed in the training and external validation sets. Finally, the internal prediction mechanisms of the models were explained by the SHapley Additive exPlanations (SHAP) approach.

RESULTS

A total of 266 patients with renal tumors from Center 1 and 161 patients from Center 2 were included. In the training set, the AUCs of the RF models constructed from the CP, NP, EP, and TP features were 0.886, 0.912, 0.930, and 0.944, respectively. In the external validation set, the models achieved AUCs of 0.860, 0.821, 0.921, and 0.908, respectively. The "original_shape_Flatness" feature played the most important role in the prediction outcome for the RF model based on EP features according to the SHAP method.

CONCLUSIONS

The four RF models efficiently differentiated benign from malignant solid renal tumors, with the EP feature-based RF model displaying the best performance.

A radiomics-based interpretable model to predict the pathological grade of pancreatic neuroendocrine tumors

OBJECTIVES

To develop a computed tomography (CT) radiomics-based interpretable machine learning (ML) model to predict the pathological grade of pancreatic neuroendocrine tumors (pNETs) in a non-invasive manner.

METHODS

Patients with pNETs who underwent contrast-enhanced abdominal CT between 2010 and 2022 were included in this retrospective study. Radiomics features were extracted, and five radiomics-based ML models, namely logistic regression (LR), random forest (RF), support vector machine (SVM), XGBoost, and GaussianNB, were developed. The performance of these models was evaluated using a time-independent testing set, and metrics such as sensitivity, specificity, accuracy, and the area under the receiver operating characteristic curve (AUC) were calculated. The accuracy of the radiomics model was compared to that of needle biopsy. The Shapley Additive Explanation (SHAP) tool and the correlation between radiomics and biological features were employed to explore the interpretability of the model.

RESULTS

A total of 122 patients (mean age: 50 ± 14 years; 53 male) were included in the training set, whereas 100 patients (mean age: 48 ± 13 years; 50 male) were included in the testing set. The AUCs for LR, SVM, RF, XGBoost, and GaussianNB were 0.758, 0.742, 0.779, 0.744, and 0.745, respectively, with corresponding accuracies of 73.0%, 70.0%, 77.0%, 71.9%, and 72.9%. The SHAP tool identified two features of the venous phase as the most significant, which showed significant differences among the Ki-67 index or mitotic count subgroups (p < 0.001).

CONCLUSIONS

An interpretable radiomics-based RF model can effectively differentiate between G1 and G2/3 of pNETs, demonstrating favorable interpretability.

CLINICAL RELEVANCE STATEMENT

The radiomics-based interpretable model developed in this study has significant clinical relevance as it offers a non-invasive method for assessing the pathological grade of pancreatic neuroendocrine tumors and holds promise as an important complementary tool to traditional tissue biopsy.

KEY POINTS

• A radiomics-based interpretable model was developed to predict the pathological grade of pNETs and compared with preoperative needle biopsy in terms of accuracy. • The model, based on CT radiomics, demonstrated favorable interpretability. • The radiomics model holds potential as a valuable complementary technique to preoperative needle biopsy; however, it should not be considered a replacement for biopsy.

A computed tomography urography-based machine learning model for predicting preoperative pathological grade of upper urinary tract urothelial carcinoma

OBJECTIVES

Development and validation of a computed tomography urography (CTU)-based machine learning (ML) model for prediction of preoperative pathology grade of upper urinary tract urothelial carcinoma (UTUC).

METHODS

A total of 140 patients with UTUC who underwent CTU examination from January 2017 to August 2023 were retrospectively enrolled. Tumor lesions on the unenhanced, medullary, and excretory periods of CTU were used to extract Features, respectively. Feature selection was screened by the Pearson and Spearman correlation analysis, least absolute shrinkage and selection operator algorithm, random forest (RF), support vector machine (SVM), and eXtreme Gradient Boosting (XGBoost). The logistic regression (LR) was used to screen for independent influencing factors of clinical baseline characteristics. Machine learning models based on different feature datasets were constructed and validated using algorithms such as LR, RF, SVM, and XGBoost. By computing the selected features, a radiomics score was generated, and a diverse feature dataset was constructed. Based on the training set, 16 ML models were created, and their performance was evaluated using the validation set for metrics including sensitivity, specificity, accuracy, area under the receiver operating characteristic curve (AUC), and others.

RESULTS

The training set consisted of 98 patients (mean age: 64.5 ± 10.5 years; 30 males), whereas the validation set consisted of 42 patients (mean age: 65.3 ± 9.78 years; 17 males). Hydronephrosis was the best independent influence factor (p < 0.05). The RF model had the best performance in predicting high-grade UTUC, with AUC of 0.914 (95% Confidence Interval [95%CI] 0.852-0.977) and 0.903 (95%CI 0.809-0.997) in the training set and validation set, and accuracy of 0.878 and 0.857, respectively.

CONCLUSIONS

An ML model based on the RF algorithm exhibits excellent predictive performance, offering a non-invasive approach for predicting preoperative high-grade UTUC.

A critical analysis of Discovery Health's claims-based risk adjustment of mortality rates in South African private sector hospitals

In 2019, Discovery Health published a risk adjustment model to determine standardised mortality rates across South African private hospital systems, with the aim of contributing towards quality improvement in the private healthcare sector. However, the model suffers from limitations due to its design and its reliance on administrative data. The publication's aim of facilitating transparency is unfortunately undermined by shortcomings in reporting. When designing a risk prediction model, patient-proximate variables with a sound theoretical or proven association with the outcome of interest should be used. The addition of key condition-specific clinical data points at the time of hospital admission will dramatically improve model performance. Performance could be further improved by using summary risk prediction scores such as the EUROSCORE II for coronary artery bypass graft surgery or the GRACE risk score for acute coronary syndrome. In general, model reporting should conform to published reporting standards, and attempts should be made to test model validity by using sensitivity analyses. In particular, the limitations of machine learning prediction models should be understood, and these models should be appropriately developed, evaluated and reported.