Automated Machine Learning for the Early Prediction of the Severity of Acute Pancreatitis in Hospitals

Predictive value of machine learning for the severity of acute pancreatitis: A systematic review and meta-analysis

Background

Predicting the severity of acute pancreatitis (AP) early poses a challenge in clinical practice. While there are well-established clinical scoring tools, their actual predictive performance remains uncertain. Various studies have explored the application of machine-learning methods for early AP prediction. However, a more comprehensive evidence-based assessment is needed to determine their predictive accuracy. Hence, this systematic review and meta-analysis aimed to evaluate the predictive accuracy of machine learning in assessing the severity of AP.

Methods

PubMed, EMBASE, Cochrane Library, and Web of Science were systematically searched until December 5, 2023. The risk of bias in eligible studies was assessed using the Prediction Model Risk of Bias Assessment Tool (PROBAST). Subgroup analyses, based on different machine learning types, were performed. Additionally, the predictive accuracy of mainstream scoring tools was summarized.

Results

This systematic review ultimately included 33 original studies. The pooled c-index in both the training and validation sets was 0.87 (95 % CI: 0.84-0.89) and 0.88 (95 % CI: 0.86-0.90), respectively. The sensitivity in the training set was 0.81 (95 % CI: 0.77-0.84), and in the validation set, it was 0.79 (95 % CI: 0.71-0.85). The specificity in the training set was 0.84 (95 % CI: 0.78-0.89), and in the validation set, it was 0.90 (95 % CI: 0.86-0.93). The primary model incorporated was logistic regression; however, its predictive accuracy was found to be inferior to that of neural networks, random forests, and xgboost. The pooled c-index of the APACHE II, BISAP, and Ranson were 0.74 (95 % CI: 0.68-0.80), 0.77 (95 % CI: 0.70-0.85), and 0.74 (95 % CI: 0.68-0.79), respectively.

Conclusions

Machine learning demonstrates excellent accuracy in predicting the severity of AP, providing a reference for updating or developing a straightforward clinical prediction tool.

Development and validation of a multimodal model in predicting severe acute pancreatitis based on radiomics and deep learning

OBJECTIVE

Aim to establish a multimodal model for predicting severe acute pancreatitis (SAP) using machine learning (ML) and deep learning (DL).

METHODS

In this multicentre retrospective study, patients diagnosed with acute pancreatitis at admission were enrolled from January 2017 to December 2021. Clinical information within 24 h and CT scans within 72 h of admission were collected. First, we trained Model α based on clinical features selected by least absolute shrinkage and selection operator analysis. Second, radiomics features were extracted from 3D-CT scans and Model β was developed on the features after dimensionality reduction using principal component analysis. Third, Model γ was trained on 2D-CT images. Lastly, a multimodal model, namely PrismSAP, was constructed based on aforementioned features in the training set. The predictive accuracy of PrismSAP was verified in the validation and internal test sets and further validated in the external test set. Model performance was evaluated using area under the curve (AUC), accuracy, sensitivity, specificity, recall, precision and F1-score.

RESULTS

A total of 1,221 eligible patients were randomly split into a training set (n = 864), a validation set (n = 209) and an internal test set (n = 148). Data of 266 patients were for external testing. In the external test set, PrismSAP performed best with the highest AUC of 0.916 (0.873-0.960) among all models [Model α: 0.709 (0.618-0.800); Model β: 0.749 (0.675-0.824); Model γ: 0.687 (0.592-0.782); MCTSI: 0.778 (0.698-0.857); RANSON: 0.642 (0.559-0.725); BISAP: 0.751 (0.668-0.833); SABP: 0.710 (0.621-0.798)].

CONCLUSION

The proposed multimodal model outperformed any single-modality models and traditional scoring systems.

Predicting the risk of early intensive care unit admission for patients hospitalized with acute pancreatitis using supervised machine learning

Background

Acute pancreatitis (AP) is a complex and life-threatening disease. Early recognition of factors predicting morbidity and mortality is crucial. We aimed to develop and validate a pragmatic model to predict the individualized risk of early intensive care unit (ICU) admission for patients with AP.

Methods

The 2019 Nationwide Readmission Database was used to identify patients hospitalized with a primary diagnosis of AP without ICU admission. A matched comparison cohort of AP patients with ICU admission within 7 days of hospitalization was identified from the National Inpatient Sample after 1:N propensity score matching. The least absolute shrinkage and selection operator (LASSO) regression was used to select predictors and develop an ICU acute pancreatitis risk (IAPR) score validated by 10-fold cross-validation.

Results

A total of 1513 patients hospitalized for AP were included. The median age was 50.0 years (interquartile range: 39.0-63.0). The three predictors that were selected included hypoxia (area under the curve [AUC] 0.78), acute kidney injury (AUC 0.72), and cardiac arrhythmia (AUC 0.61). These variables were used to develop a nomogram that displayed excellent discrimination (AUC 0.874) (bootstrap bias-corrected 95% confidence interval 0.824-0.876). There was no evidence of miscalibration (test statistic = 2.88; P = 0.09). For high-risk patients (total score >6 points), the sensitivity was 68.94% and the specificity was 92.66%.

Conclusions

This supervised machine learning-based model can help recognize high-risk AP hospitalizations. Clinicians may use the IAPR score to identify patients with AP at high risk of ICU admission within the first week of hospitalization.

Automated machine learning for early prediction of acute kidney injury in acute pancreatitis

BACKGROUND

Acute kidney injury (AKI) represents a frequent and grave complication associated with acute pancreatitis (AP), substantially elevating both mortality rates and the financial burden of hospitalization. The aim of our study is to construct a predictive model utilizing automated machine learning (AutoML) algorithms for the early prediction of AKI in patients with AP.

METHODS

We retrospectively analyzed patients who were diagnosed with AP in our hospital from January 2017 to December 2021. These patients were randomly allocated into a training set and a validation set at a ratio of 7:3. To develop predictive models for each set, we employed the least absolute shrinkage and selection operator (LASSO) algorithm along with AutoML. A nomogram was developed based on multivariate logistic regression analysis outcomes. The model's efficacy was assessed using receiver operating characteristic (ROC) curves, calibration curves, and decision curve analysis (DCA). Additionally, the performance of the model constructed via AutoML was evaluated using decision curve analysis (DCA), feature importance, SHapley Additive exPlanations (SHAP) plots, and locally interpretable model-agnostic explanations (LIME).

RESULTS

This study incorporated a total of 437 patients who met the inclusion criteria. Out of these, 313 were assigned to the training cohort and 124 to the validation cohort. In the training and validation cohorts, AKI occurred in 68 (21.7%) and 29(23.4%) patients, respectively. Comparative analysis revealed that the AutoML models exhibited enhanced performance over traditional logistic regression (LR). Furthermore, the deep learning (DL) model demonstrated superior predictive accuracy, evidenced by an area under the ROC curve of 0.963 in the training set and 0.830 in the validation set, surpassing other comparative models. The key variables identified as significant in the DL model within the training dataset included creatinine (Cr), urea (Urea), international normalized ratio (INR), etiology, smoking, alanine aminotransferase (ALT), hypertension, prothrombin time (PT), lactate dehydrogenase (LDH), and diabetes.

CONCLUSION

The AutoML model, utilizing DL algorithm, offers considerable clinical significance in the early detection of AKI among patients with AP.

Acute pancreatitis: A review of diagnosis, severity prediction and prognosis assessment from imaging technology, scoring system and artificial intelligence

Acute pancreatitis (AP) is a potentially life-threatening inflammatory disease of the pancreas, with clinical management determined by the severity of the disease. Diagnosis, severity prediction, and prognosis assessment of AP typically involve the use of imaging technologies, such as computed tomography, magnetic resonance imaging, and ultrasound, and scoring systems, including Ranson, Acute Physiology and Chronic Health Evaluation II, and Bedside Index for Severity in AP scores. Computed tomography is considered the gold standard imaging modality for AP due to its high sensitivity and specificity, while magnetic resonance imaging and ultrasound can provide additional information on biliary obstruction and vascular complications. Scoring systems utilize clinical and laboratory parameters to classify AP patients into mild, moderate, or severe categories, guiding treatment decisions, such as intensive care unit admission, early enteral feeding, and antibiotic use. Despite the central role of imaging technologies and scoring systems in AP management, these methods have limitations in terms of accuracy, reproducibility, practicality and economics. Recent advancements of artificial intelligence (AI) provide new opportunities to enhance their performance by analyzing vast amounts of clinical and imaging data. AI algorithms can analyze large amounts of clinical and imaging data, identify scoring system patterns, and predict the clinical course of disease. AI-based models have shown promising results in predicting the severity and mortality of AP, but further validation and standardization are required before widespread clinical application. In addition, understanding the correlation between these three technologies will aid in developing new methods that can accurately, sensitively, and specifically be used in the diagnosis, severity prediction, and prognosis assessment of AP through complementary advantages.

Machine learning-based decision tool for selecting patients with idiopathic acute pancreatitis for endosonography to exclude a biliary aetiology

BACKGROUND

Biliary microlithiasis/sludge is detected in approximately 30% of patients with idiopathic acute pancreatitis (IAP). As recurrent biliary pancreatitis can be prevented, the underlying aetiology of IAP should be established.

AIM

To develop a machine learning (ML) based decision tool for the use of endosonography (EUS) in pancreatitis patients to detect sludge and microlithiasis.

METHODS

We retrospectively used routinely recorded clinical and laboratory parameters of 218 consecutive patients with confirmed AP admitted to our tertiary care hospital between 2015 and 2020. Patients who did not receive EUS as part of the diagnostic work-up and whose pancreatitis episode could be adequately explained by other causes than biliary sludge and microlithiasis were excluded. We trained supervised ML classifiers using H2O.ai automatically selecting the best suitable predictor model to predict microlithiasis/sludge. The predictor model was further validated in two independent retrospective cohorts from two tertiary care centers (117 patients).

RESULTS

Twenty-eight categorized patients' variables recorded at admission were identified to compute the predictor model with an accuracy of 0.84 [95% confidence interval (CI): 0.791-0.9185], positive predictive value of 0.84, and negative predictive value of 0.80 in the identification cohort (218 patients). In the validation cohort, the robustness of the prediction model was confirmed with an accuracy of 0.76 (95%CI: 0.673-0.8347), positive predictive value of 0.76, and negative predictive value of 0.78 (117 patients).

CONCLUSION

We present a robust and validated ML-based predictor model consisting of routinely recorded parameters at admission that can predict biliary sludge and microlithiasis as the cause of AP.

Construction and validation of machine learning models for sepsis prediction in patients with acute pancreatitis

BACKGROUND

This study aimed to construct predictive models for the risk of sepsis in patients with Acute pancreatitis (AP) using machine learning methods and compared optimal one with the logistic regression (LR) model and scoring systems.

METHODS

In this retrospective cohort study, data were collected from the Medical Information Mart for Intensive Care III (MIMIC III) database between 2001 and 2012 and the MIMIC IV database between 2008 and 2019. Patients were randomly divided into training and test sets (8:2). The least absolute shrinkage and selection operator (LASSO) regression plus 5-fold cross-validation were used to screen and confirm the predictive factors. Based on the selected predictive factors, 6 machine learning models were constructed, including support vector machine (SVM), K-nearest neighbour (KNN), multi-layer perceptron (MLP), LR, gradient boosting decision tree (GBDT) and adaptive enhancement algorithm (AdaBoost). The models and scoring systems were evaluated and compared using sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy, and the area under the curve (AUC).

RESULTS

A total of 1, 672 patients were eligible for participation. In the training set, 261 AP patients (19.51%) were diagnosed with sepsis. The predictive factors for the risk of sepsis in AP patients included age, insurance, vasopressors, mechanical ventilation, Glasgow Coma Scale (GCS), heart rate, respiratory rate, temperature, SpO2, platelet, red blood cell distribution width (RDW), International Normalized Ratio (INR), and blood urea nitrogen (BUN). The AUC of the GBDT model for sepsis prediction in the AP patients in the testing set was 0.985. The GBDT model showed better performance in sepsis prediction than the LR, systemic inflammatory response syndrome (SIRS) score, bedside index for severity in acute pancreatitis (BISAP) score, sequential organ failure assessment (SOFA) score, quick-SOFA (qSOFA), and simplified acute physiology score II (SAPS II).

CONCLUSION

The present findings suggest that compared to the classical LR model and SOFA, qSOFA, SAPS II, SIRS, and BISAP scores, the machine learning model-GBDT model had a better performance in predicting sepsis in the AP patients, which is a useful tool for early identification of high-risk patients and timely clinical interventions.