Machine learning approaches for the prediction of postoperative complication risk in liver resection patients

Machine learning improves prediction of postoperative outcomes after gastrointestinal surgery: a systematic review and meta-analysis

BACKGROUND

Machine learning (ML) approaches have become increasingly popular in predicting surgical outcomes. However, it is unknown whether they are superior to traditional statistical methods such as logistic regression (LR). This study aimed to perform a systematic review and meta-analysis to compare the performance of ML vs LR models in predicting postoperative outcomes for patients undergoing gastrointestinal (GI) surgery.

METHODS

A systematic search of Embase, MEDLINE, Cochrane, Web of Science, and Google Scholar was performed through December 2022. The primary outcome was the discriminatory performance of ML vs LR models as measured by the area under the receiver operating characteristic curve (AUC). A meta-analysis was then performed using a random effects model.

RESULTS

A total of 62 LR models and 143 ML models were included across 38 studies. On average, the best-performing ML models had a significantly higher AUC than the LR models (ΔAUC, 0.07; 95% CI, 0.04-0.09; P < .001). Similarly, on average, the best-performing ML models had a significantly higher logit (AUC) than the LR models (Δlogit [AUC], 0.41; 95% CI, 0.23-0.58; P < .001). Approximately half of studies (44%) were found to have a low risk of bias. Upon a subset analysis of only low-risk studies, the difference in logit (AUC) remained significant (ML vs LR, Δlogit [AUC], 0.40; 95% CI, 0.14-0.66; P = .009).

CONCLUSION

We found a significant improvement in discriminatory ability when using ML over LR algorithms in predicting postoperative outcomes for patients undergoing GI surgery. Subsequent efforts should establish standardized protocols for both developing and reporting studies using ML models and explore the practical implementation of these models.

Experts vs. machine - comparison of machine learning to expert-informed prediction of outcome after major liver surgery

BACKGROUND

Machine learning (ML) has been successfully implemented for classification tasks (e.g., cancer diagnosis). ML performance for more challenging predictions is largely unexplored. This study's objective was to compare machine learning vs. expert-informed predictions for surgical outcome in patients undergoing major liver surgery.

METHODS

Single tertiary center data on preoperative parameters and postoperative complications for elective hepatic surgery patients were included (2008-2021). Expert-informed prediction models were established on 14 parameters identified by two expert liver surgeons to impact on postoperative outcome. ML models used all available preoperative patient variables (n = 62). Model performance was compared for predicting 3-month postoperative overall morbidity. Temporal validation and additional analysis in major liver resection patients were conducted.

RESULTS

889 patients included. Expert-informed models showed low average bias (2-5 CCI points) with high over/underprediction. ML models performed similarly: average prediction 5-10 points higher than observed CCI values with high variability (95% CI -30 to 50). No performance improvement for major liver surgery patients.

CONCLUSION

No clinical relevance in the application of ML for predicting postoperative overall morbidity was found. Despite being a novel hype, ML has the potential for application in clinical practice. However, at this stage it does not replace established approaches of prediction modelling.

Using Machine Learning to Predict Unplanned Hospital Utilization and Chemotherapy Management From Patient-Reported Outcome Measures

PURPOSE

Adverse effects of chemotherapy often require hospital admissions or treatment management. Identifying factors contributing to unplanned hospital utilization may improve health care quality and patients' well-being. This study aimed to assess if patient-reported outcome measures (PROMs) improve performance of machine learning (ML) models predicting hospital admissions, triage events (contacting helpline or attending hospital), and changes to chemotherapy.

MATERIALS AND METHODS

Clinical trial data were used and contained responses to three PROMs (European Organisation for Research and Treatment of Cancer Core Quality of Life Questionnaire [QLQ-C30], EuroQol Five-Dimensional Visual Analogue Scale [EQ-5D], and Functional Assessment of Cancer Therapy-General [FACT-G]) and clinical information on 508 participants undergoing chemotherapy. Six feature sets (with following variables: [1] all available; [2] clinical; [3] PROMs; [4] clinical and QLQ-C30; [5] clinical and EQ-5D; [6] clinical and FACT-G) were applied in six ML models (logistic regression [LR], decision tree, adaptive boosting, random forest [RF], support vector machines [SVMs], and neural network) to predict admissions, triage events, and chemotherapy changes.

RESULTS

The comprehensive analysis of predictive performances of the six ML models for each feature set in three different methods for handling class imbalance indicated that PROMs improved predictions of all outcomes. RF and SVMs had the highest performance for predicting admissions and changes to chemotherapy in balanced data sets, and LR in imbalanced data set. Balancing data led to the best performance compared with imbalanced data set or data set with balanced train set only.

CONCLUSION

These results endorsed the view that ML can be applied on PROM data to predict hospital utilization and chemotherapy management. If further explored, this study may contribute to health care planning and treatment personalization. Rigorous comparison of model performance affected by different imbalanced data handling methods shows best practice in ML research.

Predicting Safe Liver Resection Volume for Major Hepatectomy Using Artificial Intelligence

(1) Background: Advancements in the field of liver surgery have led to a critical need for precise estimations of preoperative liver function to prevent post-hepatectomy liver failure (PHLF), a significant cause of morbidity and mortality. This study introduces a novel application of artificial intelligence (AI) in determining safe resection volumes according to a patient's liver function in major hepatectomies. (2) Methods: We incorporated a deep learning approach, incorporating a unique liver-specific loss function, to analyze patient characteristics, laboratory data, and liver volumetry from computed tomography scans of 52 patients. Our approach was evaluated against existing machine and deep learning techniques. (3) Results: Our approach achieved 68.8% accuracy in predicting safe resection volumes, demonstrating superior performance over traditional models. Furthermore, it significantly reduced the mean absolute error in under-predicted volumes to 23.72, indicating a more precise estimation of safe resection limits. These findings highlight the potential of integrating AI into surgical planning for liver resections. (4) Conclusion: By providing more accurate predictions of safe resection volumes, our method aims to minimize the risk of PHLF, thereby improving clinical outcomes for patients undergoing hepatectomy.

Prediction of Complications and Prognostication in Perioperative Medicine: A Systematic Review and PROBAST Assessment of Machine Learning Tools

BACKGROUND

The utilization of artificial intelligence and machine learning as diagnostic and predictive tools in perioperative medicine holds great promise. Indeed, many studies have been performed in recent years to explore the potential. The purpose of this systematic review is to assess the current state of machine learning in perioperative medicine, its utility in prediction of complications and prognostication, and limitations related to bias and validation.

METHODS

A multidisciplinary team of clinicians and engineers conducted a systematic review using the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) protocol. Multiple databases were searched, including Scopus, Cumulative Index to Nursing and Allied Health Literature (CINAHL), the Cochrane Library, PubMed, Medline, Embase, and Web of Science. The systematic review focused on study design, type of machine learning model used, validation techniques applied, and reported model performance on prediction of complications and prognostication. This review further classified outcomes and machine learning applications using an ad hoc classification system. The Prediction model Risk Of Bias Assessment Tool (PROBAST) was used to assess risk of bias and applicability of the studies.

RESULTS

A total of 103 studies were identified. The models reported in the literature were primarily based on single-center validations (75%), with only 13% being externally validated across multiple centers. Most of the mortality models demonstrated a limited ability to discriminate and classify effectively. The PROBAST assessment indicated a high risk of systematic errors in predicted outcomes and artificial intelligence or machine learning applications.

CONCLUSIONS

The findings indicate that the development of this field is still in its early stages. This systematic review indicates that application of machine learning in perioperative medicine is still at an early stage. While many studies suggest potential utility, several key challenges must be first overcome before their introduction into clinical practice.

EDITOR’S PERSPECTIVE

Machine learning approach for the detection of vitamin D level: a comparative study

BACKGROUND

After the World Health Organization declared the COVID-19 pandemic, the role of Vitamin D has become even more critical for people worldwide. The most accurate way to define vitamin D level is 25-hydroxy vitamin D(25-OH-D) blood test. However, this blood test is not always feasible. Most data sets used in health science research usually contain highly correlated features, which is referred to as multicollinearity problem. This problem can lead to misleading results and overfitting problems in the ML training process. Therefore, the proposed study aims to determine a clinically acceptable ML model for the detection of the vitamin D status of the North Cyprus adult participants accurately, without the need to determine 25-OH-D level, taking into account the multicollinearity problem.

METHOD

The study was conducted with 481 observations who applied voluntarily to Internal Medicine Department at NEU Hospital. The classification performance of four conventional supervised ML models, namely, Ordinal logistic regression(OLR), Elastic-net ordinal regression(ENOR), Support Vector Machine(SVM), and Random Forest (RF) was compared. The comparative analysis is performed regarding the model's sensitivity to the participant's metabolic syndrome(MtS)'positive status, hyper-parameter tuning, sensitivities to the size of training data, and the classification performance of the models.

RESULTS

Due to the presence of multicollinearity, the findings showed that the performance of the SVM(RBF) is obviously negatively affected when the test is examined. Moreover, it can be obviously detected that RF is more robust than other models when the variations in the size of training data are examined. This experiment's result showed that the selected RF and ENOR showed better performances than the other two models when the size of training samples was reduced. Since the multicollinearity is more severe in the small samples, it can be concluded that RF and ENOR are not affected by the presence of the multicollinearity problem. The comparative analysis revealed that the RF classifier performed better and was more robust than the other proposed models in terms of accuracy (0.94), specificity (0.96), sensitivity or recall (0.94), precision (0.95), F1-score (0.95), and Cohen's kappa (0.90).

CONCLUSION

It is evident that the RF achieved better than the SVM(RBF), ENOR, and OLR. These comparison findings will be applied to develop a Vitamin D level intelligent detection system for being used in routine clinical, biochemical tests, and lifestyle characteristics of individuals to decrease the cost and time of vitamin D level detection.

Combined application of inflammation-related biomarkers to predict postoperative complications of rectal cancer patients: a retrospective study by machine learning analysis

BACKGROUND

Postoperative complications in patients of rectal cancer pose challenges to postoperative recovery. Accurately predicting these complications is crucial for developing effective treatment plans for patients.

METHODS

In this retrospective study, 493 patients with rectal cancer who underwent radical resection between January 2020 and December 2021 were examined. We evaluated logistic regression, support vector machines, regression trees, and random forests to predict the incidence of postoperative complications in patients and evaluate the performance of the model. The results will be analyzed to make recommendations for reducing complications.

RESULTS

Among the four machine learning models, random forest demonstrated the highest results. The performance of this model was showed with an AUC of 0.880 (95% CI 0.807-0.949), an accuracy of 88.0% (95% CI 0.815-0.929), a sensitivity of 96.6%, and a specificity of 45.8%. Notably, factors such as inflammation related prognostic index, prognostic nutritional index, tumor location, and T stage were found to significantly increase the probability of postoperative complications.

CONCLUSION

Our study provided evidence that machine learning models can effectively evaluate early postoperative complications of the patients after surgery.

Identifying Effective Biomarkers for Accurate Pancreatic Cancer Prognosis Using Statistical Machine Learning

Pancreatic cancer (PC) has one of the lowest survival rates among all major types of cancer. Consequently, it is one of the leading causes of mortality worldwide. Serum biomarkers historically correlate well with the early prognosis of post-surgical complications of PC. However, attempts to identify an effective biomarker panel for the successful prognosis of PC were almost non-existent in the current literature. The current study investigated the roles of various serum biomarkers including carbohydrate antigen 19-9 (CA19-9), chemokine (C-X-C motif) ligand 8 (CXCL-8), procalcitonin (PCT), and other relevant clinical data for identifying PC progression, classified into sepsis, recurrence, and other post-surgical complications, among PC patients. The most relevant biochemical and clinical markers for PC prognosis were identified using a random-forest-powered feature elimination method. Using this informative biomarker panel, the selected machine-learning (ML) classification models demonstrated highly accurate results for classifying PC patients into three complication groups on independent test data. The superiority of the combined biomarker panel (Max AUC-ROC = 100%) was further established over using CA19-9 features exclusively (Max AUC-ROC = 75%) for the task of classifying PC progression. This novel study demonstrates the effectiveness of the combined biomarker panel in successfully diagnosing PC progression and other relevant complications among Egyptian PC survivors.

Interpretable prediction of cardiopulmonary complications after non-small cell lung cancer surgery based on machine learning and SHapley additive exPlanations

Introduction

Lung cancer is a prevalent malignancy globally, with approximately 20% of patients developing cardiopulmonary complications after lobectomy. In order to prevent complications, an accurate and personalized method based on machine learning (ML) is required.

Methods

During the period of 2017-2021, a retrospective analysis was conducted on the medical records of patients who had undergone lobectomy for non-small cell lung cancer (NSCLC). We performed logical regression, decision tree (DT), random forest (RF), gradient boost DT, and eXtreme gradient boosting analyses to establish an ML model. The ten-fold cross-validation was used to evaluate the performance of multiple ML models based on various evaluation metrics, including accuracy, precision, recall, F1 score, and area under the receiver operating (AUC). Additionally, we also calculated the Kappa value of these model. Each model used grid search to optimize hyper-parameters and then used the interpretability method to provide explanations for the model's Decisions.

Results

The study included 718 eligible patients, among whom the incidence of postoperative cardiopulmonary complications was 20.89%. The RF model showed the best comprehensive performance among all models, and its ten-fold cross-validation accuracy, precision, recall, F1 score, and AUC were (OR and 95% confidence interval [CI]) 0.786 (0.738-0.834), 0.803 (0.735-0.872), 0.738 (0.678-0.797), 0.766 (0.714-0.818), 0.856 (0.815-0.898), respectively. The kappa value of the RF model was 0.696 (0.617-0.768). The SHAP method showed that gender, age, and intraoperative blood loss were closely associated with postoperative cardiopulmonary complications.

Conclusion

The application of ML methods for predicting postoperative cardiopulmonary complications based on clinical data of patients with NSCLC showed a good performance. The results indicate that ML combined with the SHAP individualized interpretation method has practical clinical value.

Prediction of Postoperative Pulmonary Edema Risk Using Machine Learning

Postoperative pulmonary edema (PPE) is a well-known postoperative complication. We hypothesized that a machine learning model could predict PPE risk using pre- and intraoperative data, thereby improving postoperative management. This retrospective study analyzed the medical records of patients aged > 18 years who underwent surgery between January 2011 and November 2021 at five South Korean hospitals. Data from four hospitals (n = 221,908) were used as the training dataset, whereas data from the remaining hospital (n = 34,991) were used as the test dataset. The machine learning algorithms used were extreme gradient boosting, light-gradient boosting machine, multilayer perceptron, logistic regression, and balanced random forest (BRF). The prediction abilities of the machine learning models were assessed using the area under the receiver operating characteristic curve, feature importance, and average precisions of precision-recall curve, precision, recall, f1 score, and accuracy. PPE occurred in 3584 (1.6%) and 1896 (5.4%) patients in the training and test sets, respectively. The BRF model exhibited the best performance (area under the receiver operating characteristic curve: 0.91, 95% confidence interval: 0.84-0.98). However, its precision and f1 score metrics were not good. The five major features included arterial line monitoring, American Society of Anesthesiologists physical status, urine output, age, and Foley catheter status. Machine learning models (e.g., BRF) could predict PPE risk and improve clinical decision-making, thereby enhancing postoperative management.