Machine Learning and Surgical Outcomes Prediction: A Systematic Review

Can Machine Learning Correctly Predict Outcomes of Flexible Ureteroscopy with Laser Lithotripsy for Kidney Stone Disease? Results from a Large Endourology University Centre

Background and objective

The integration of machine learning (ML) in health care has garnered significant attention because of its unprecedented opportunities to enhance patient care and outcomes. In this study, we trained ML algorithms for automated prediction of outcomes of ureteroscopic laser lithotripsy (URSL) on the basis of preoperative characteristics.

Methods

Data were retrieved for patients treated with ureteroscopy for urolithiasis by a single experienced surgeon over a 7-yr period. Sixteen ML classification algorithms were trained to investigate correlation between preoperative characteristics and postoperative outcomes. The outcomes assessed were primary stone-free status (SFS, defined as the presence of only stone fragments <2 mm on endoscopic visualisation and at 3-mo imaging) and postoperative complications. An ensemble model was constructed from the best-performing algorithms for prediction of complications and for prediction of SFS. Simultaneous prediction of postoperative characteristics was then investigated using a multitask neural network, and explainable artificial intelligence (AI) was used to demonstrate the predictive power of the best models.

Key findings and limitations

An ensemble ML model achieved accuracy of 93% and precision of 87% for prediction of SFS. Complications were mainly associated with a preoperative positive urine culture (1.44). Logistic regression revealed that SFS was impacted by the total stone burden (0.34), the presence of a preoperative stent (0.106), a positive preoperative urine culture (0.14), and stone location (0.09). Explainable AI results emphasised the key features and their contributions to the output.

Conclusions and clinical implications

Technological advances are helping urologists to overcome the classic limits of ureteroscopy, namely stone size and the risk of complications. ML represents an excellent aid for correct prediction of outcomes after training on pre-existing data sets. Our ML model achieved accuracy of >90% for prediction of SFS and complications, and represents a basis for the development of an accessible predictive model for endourologists and patients in the URSL setting.

Patient summary

We tested the ability of artificial intelligence to predict treatment outcomes for patients with kidney stones. We trained 16 different machine learning tools with data before surgery, such as patient age and the stone characteristics. Our final model was >90% accurate in predicting stone-free status after surgery and the occurrence of complications.

Machine learning improves prediction of postoperative outcomes after gastrointestinal surgery: a systematic review and meta-analysis

BACKGROUND

Machine learning (ML) approaches have become increasingly popular in predicting surgical outcomes. However, it is unknown whether they are superior to traditional statistical methods such as logistic regression (LR). This study aimed to perform a systematic review and meta-analysis to compare the performance of ML vs LR models in predicting postoperative outcomes for patients undergoing gastrointestinal (GI) surgery.

METHODS

A systematic search of Embase, MEDLINE, Cochrane, Web of Science, and Google Scholar was performed through December 2022. The primary outcome was the discriminatory performance of ML vs LR models as measured by the area under the receiver operating characteristic curve (AUC). A meta-analysis was then performed using a random effects model.

RESULTS

A total of 62 LR models and 143 ML models were included across 38 studies. On average, the best-performing ML models had a significantly higher AUC than the LR models (ΔAUC, 0.07; 95% CI, 0.04-0.09; P < .001). Similarly, on average, the best-performing ML models had a significantly higher logit (AUC) than the LR models (Δlogit [AUC], 0.41; 95% CI, 0.23-0.58; P < .001). Approximately half of studies (44%) were found to have a low risk of bias. Upon a subset analysis of only low-risk studies, the difference in logit (AUC) remained significant (ML vs LR, Δlogit [AUC], 0.40; 95% CI, 0.14-0.66; P = .009).

CONCLUSION

We found a significant improvement in discriminatory ability when using ML over LR algorithms in predicting postoperative outcomes for patients undergoing GI surgery. Subsequent efforts should establish standardized protocols for both developing and reporting studies using ML models and explore the practical implementation of these models.

Meta-lasso: new insight on infection prediction after minimally invasive surgery

Surgical site infection (SSI) after minimally invasive lung cancer surgery constitutes an important factor influencing the direct and indirect economic implications, patient prognosis, and the 5-year survival rate for early-stage lung cancer patients. In the realm of predictive healthcare, machine learning algorithms have been instrumental in anticipating various surgical outcomes, including SSI. However, accurately predicting infection after minimally invasive surgery remains a clinical challenge due to the multitude of physiological and surgical factors associated with it. Furthermore, clinical patient data, in addition to being high-dimensional, often exists the long-tail problem, posing difficulties for traditional machine learning algorithms in effectively processing such data. Based on this insight, we propose a novel approach called meta-lasso for infection prediction following minimally invasive surgery. Our approach leverages the sparse learning algorithm lasso regression to select informative features and introduces a meta-learning framework to mitigate bias towards the dominant class. We conducted a retrospective cohort study on patients who had undergone minimally invasive surgery for lung cancer at Shanghai Chest Hospital between 2018 and 2020. The evaluation encompassed key performance metrics, including sensitivity, specificity, precision (PPV), negative predictive value (NPV), and accuracy. Our approach has surpassed the performance of logistic regression, random forest, Naive Bayes classifier, gradient boosting decision tree, ANN, and lasso regression, with sensitivity at 0.798, specificity at 0.779, precision at 0.789, NPV at 0.798, and accuracy at 0.788 and has greatly improved the classification performance of the inferior class.

Predicting Outcomes Following Lower Extremity Endovascular Revascularization Using Machine Learning

BACKGROUND

Lower extremity endovascular revascularization for peripheral artery disease carries nonnegligible perioperative risks; however, outcome prediction tools remain limited. Using machine learning, we developed automated algorithms that predict 30-day outcomes following lower extremity endovascular revascularization.

METHODS AND RESULTS

The National Surgical Quality Improvement Program targeted vascular database was used to identify patients who underwent lower extremity endovascular revascularization (angioplasty, stent, or atherectomy) for peripheral artery disease between 2011 and 2021. Input features included 38 preoperative demographic/clinical variables. The primary outcome was 30-day postprocedural major adverse limb event (composite of major reintervention, untreated loss of patency, or major amputation) or death. Data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, 6 machine learning models were trained using preoperative features. The primary model evaluation metric was area under the receiver operating characteristic curve. Overall, 21 886 patients were included, and 30-day major adverse limb event/death occurred in 1964 (9.0%) individuals. The best performing model for predicting 30-day major adverse limb event/death was extreme gradient boosting, achieving an area under the receiver operating characteristic curve of 0.93 (95% CI, 0.92-0.94). In comparison, logistic regression had an area under the receiver operating characteristic curve of 0.72 (95% CI, 0.70-0.74). The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.09. The top 3 predictive features in our algorithm were (1) chronic limb-threatening ischemia, (2) tibial intervention, and (3) congestive heart failure.

CONCLUSIONS

Our machine learning models accurately predict 30-day outcomes following lower extremity endovascular revascularization using preoperative data with good discrimination and calibration. Prospective validation is warranted to assess for generalizability and external validity.

Identification and Evaluation of Blood-Based Biomarkers for Abdominal Aortic Aneurysm

INTRODUCTION

Blood-based biomarkers for abdominal aortic aneurysm (AAA) have been studied individually; however, we considered a panel of proteins to investigate AAA prognosis and its potential to improve predictive accuracy.

MATERIALS AND METHODS

Using a prospectively recruited cohort of patients with/without AAA (n = 452), we conducted a prognostic study to develop a model that accurately predicts AAA outcomes using clinical features and circulating biomarker levels. Serum concentrations of 9 biomarkers were measured at baseline, and the cohort was followed for 2 years. The primary outcome was major adverse aortic event (MAAE; composite of rapid AAA expansion [>0.5 cm/6 months or >1 cm/12 months], AAA intervention, or AAA rupture). Using 10-fold cross-validation, we trained a random forest model to predict 2 year MAAE using (1) clinical characteristics, (2) biomarkers, and (3) clinical characteristics and biomarkers.

RESULTS

Two-year MAAE occurred in 114 (25%) patients. Two proteins were significantly elevated in patients with AAA compared with those without AAA (angiopoietin-2 and aggrecan), composing the protein panel. For predicting 2 year MAAE, our random forest model achieved area under the receiver operating characteristic curve (AUROC) 0.74 using clinical features alone, and the addition of the 2-protein panel improved performance to AUROC 0.86.

CONCLUSIONS

Using a combination of clinical/biomarker data, we developed a model that accurately predicts 2 year MAAE.

Development and evaluation of a prediction model for peripheral artery disease-related major adverse limb events using novel biomarker data

OBJECTIVE

Prognostic tools for individuals with peripheral artery disease (PAD) remain limited. We developed prediction models for 3-year PAD-related major adverse limb events (MALE) using demographic, clinical, and biomarker data previously validated by our group.

METHODS

We performed a prognostic study using a prospectively recruited cohort of patients with PAD (n = 569). Demographic/clinical data were recorded including sex, age, comorbidities, previous procedures, and medications. Plasma concentrations of three biomarkers (N-terminal pro-B-type natriuretic peptide [NT-proBNP], fatty acid binding protein 3 [FABP3], and FABP4) were measured at baseline. The cohort was followed for 3 years. MALE was the primary outcome (composite of open/endovascular vascular intervention or major amputation). We trained three machine learning models with 10-fold cross-validation using demographic, clinical, and biomarker data (random forest, decision trees, and Extreme Gradient Boosting [XGBoost]) to predict 3-year MALE in patients. Area under the receiver operating characteristic curve (AUROC) was the primary model evaluation metric.

RESULTS

Three-year MALE was observed in 162 patients (29%). XGBoost was the top-performing predictive model for 3-year MALE, achieving the following performance metrics: AUROC = 0.88 (95% confidence interval [CI], 0.84-0.94); sensitivity, 88%; specificity, 84%; positive predictive value, 83%; and negative predictive value, 91% on test set data. On an independent validation cohort of patients with PAD, XGBoost attained an AUROC of 0.87 (95% CI, 0.82-0.90). The 10 most important predictors of 3-year MALE consisted of: (1) FABP3; (2) FABP4; (3) age; (4) NT-proBNP; (5) active smoking; (6) diabetes; (7) hypertension; (8) dyslipidemia; (9) coronary artery disease; and (10) sex.

CONCLUSIONS

We built robust machine learning algorithms that accurately predict 3-year MALE in patients with PAD using demographic, clinical, and novel biomarker data. Our algorithms can support risk stratification of patients with PAD for additional vascular evaluation and early aggressive medical management, thereby improving outcomes. Further validation of our models for clinical implementation is warranted.

Using Machine Learning (XGBoost) to Predict Outcomes After Infrainguinal Bypass for Peripheral Artery Disease

OBJECTIVE

To develop machine learning (ML) algorithms that predict outcomes after infrainguinal bypass.

BACKGROUND

Infrainguinal bypass for peripheral artery disease carries significant surgical risks; however, outcome prediction tools remain limited.

METHODS

The Vascular Quality Initiative database was used to identify patients who underwent infrainguinal bypass for peripheral artery disease between 2003 and 2023. We identified 97 potential predictor variables from the index hospitalization [68 preoperative (demographic/clinical), 13 intraoperative (procedural), and 16 postoperative (in-hospital course/complications)]. The primary outcome was 1-year major adverse limb event (composite of surgical revision, thrombectomy/thrombolysis, or major amputation) or death. Our data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, we trained 6 ML models using preoperative features. The primary model evaluation metric was the area under the receiver operating characteristic curve (AUROC). The top-performing algorithm was further trained using intraoperative and postoperative features. Model robustness was evaluated using calibration plots and Brier scores.

RESULTS

Overall, 59,784 patients underwent infrainguinal bypass, and 15,942 (26.7%) developed 1-year major adverse limb event/death. The best preoperative prediction model was XGBoost, achieving an AUROC (95% CI) of 0.94 (0.93-0.95). In comparison, logistic regression had an AUROC (95% CI) of 0.61 (0.59-0.63). Our XGBoost model maintained excellent performance at the intraoperative and postoperative stages, with AUROCs (95% CI's) of 0.94 (0.93-0.95) and 0.96 (0.95-0.97), respectively. Calibration plots showed good agreement between predicted and observed event probabilities with Brier scores of 0.08 (preoperative), 0.07 (intraoperative), and 0.05 (postoperative).

CONCLUSIONS

ML models can accurately predict outcomes after infrainguinal bypass, outperforming logistic regression.

LightGBM outperforms other machine learning techniques in predicting graft failure after liver transplantation: Creation of a predictive model through large-scale analysis

BACKGROUND

The incidence of graft failure following liver transplantation (LTx) is consistent. While traditional risk scores for LTx have limited accuracy, the potential of machine learning (ML) in this area remains uncertain, despite its promise in other transplant domains. This study aims to determine ML's predictive limitations in LTx by replicating methods used in previous heart transplant research.

METHODS

This study utilized the UNOS STAR database, selecting 64,384 adult patients who underwent LTx between 2010 and 2020. Gradient boosting models (XGBoost and LightGBM) were used to predict 14, 30, and 90-day graft failure compared to conventional logistic regression model. Models were evaluated using both shuffled and rolling cross-validation (CV) methodologies. Model performance was assessed using the AUC across validation iterations.

RESULTS

In a study comparing predictive models for 14-day, 30-day and 90-day graft survival, LightGBM consistently outperformed other models, achieving the highest AUC of.740,.722, and.700 in shuffled CV methods. However, in rolling CV the accuracy of the model declined across every ML algorithm. The analysis revealed influential factors for graft survival prediction across all models, including total bilirubin, medical condition, recipient age, and donor AST, among others. Several features like donor age and recipient diabetes history were important in two out of three models.

CONCLUSIONS

LightGBM enhances short-term graft survival predictions post-LTx. However, due to changing medical practices and selection criteria, continuous model evaluation is essential. Future studies should focus on temporal variations, clinical implications, and ensure model transparency for broader medical utility.

Identifying Age-Specific Risk Factors for Poor Outcomes After Trauma With Machine Learning

INTRODUCTION

Risk stratification for poor outcomes is not currently age-specific. Risk stratification of older patients based on observational cohorts primarily composed of young patients may result in suboptimal clinical care and inaccurate quality benchmarking. We assessed two hypotheses. First, we hypothesized that risk factors for poor outcomes after trauma are age-dependent and, second, that the relative importance of various risk factors are also age-dependent.

METHODS

A cohort study of severely injured adult trauma patients admitted to the intensive care unit 2014-2018 was performed using trauma registry data. Random forest algorithms predicting poor outcomes (death or complication) were built and validated using three cohorts: (1) patients of all ages, (2) younger patients, and (3) older patients. Older patients were defined as aged 55 y or more to maintain consistency with prior trauma literature. Complications assessed included acute renal failure, acute respiratory distress syndrome, cardiac arrest, unplanned intubation, unplanned intensive care unit admission, and unplanned return to the operating room, as defined by the trauma quality improvement program. Mean decrease in model accuracy (MDA), if each variable was removed and scaled to a Z-score, was calculated. MDA change ≥4 standard deviations between age cohorts was considered significant.

RESULTS

Of 5489 patients, 25% were older. Poor outcomes occurred in 12% of younger and 33% of older patients. Head injury was the most important predictor of poor outcome in all cohorts. In the full cohort, age was the most important predictor of poor outcomes after head injury. Within age cohorts, the most important predictors of poor outcomes, after head injury, were surgery requirement in younger patients and arrival Glasgow Coma Scale in older patients. Compared to younger patients, head injury and arrival Glasgow Coma Scale had the greatest increase in importance for older patients, while systolic blood pressure had the greatest decrease in importance.

CONCLUSIONS

Supervised machine learning identified differences in risk factors and their relative associations with poor outcomes based on age. Age-specific models may improve hospital benchmarking and identify quality improvement targets for older trauma patients.

A machine learning algorithm for peripheral artery disease prognosis using biomarker data

Peripheral artery disease (PAD) biomarkers have been studied in isolation; however, an algorithm that considers a protein panel to inform PAD prognosis may improve predictive accuracy. Biomarker-based prediction models were developed and evaluated using a model development (n = 270) and prospective validation cohort (n = 277). Plasma concentrations of 37 proteins were measured at baseline and the patients were followed for 2 years. The primary outcome was 2-year major adverse limb event (MALE; composite of vascular intervention or major amputation). Of the 37 proteins tested, 6 were differentially expressed in patients with vs. without PAD (ADAMTS13, ICAM-1, ANGPTL3, Alpha 1-microglobulin, GDF15, and endostatin). Using 10-fold cross-validation, we developed a random forest machine learning model that accurately predicts 2-year MALE in a prospective validation cohort of PAD patients using a 6-protein panel (AUROC 0.84). This algorithm can support PAD risk stratification, informing clinical decisions on further vascular evaluation and management.

Predicting Outcomes Following Endovascular Abdominal Aortic Aneurysm Repair Using Machine Learning

OBJECTIVE

To develop machine learning (ML) models that predict outcomes following endovascular aneurysm repair (EVAR) for abdominal aortic aneurysm (AAA).

BACKGROUND

EVAR carries non-negligible perioperative risks; however, there are no widely used outcome prediction tools.

METHODS

The National Surgical Quality Improvement Program targeted database was used to identify patients who underwent EVAR for infrarenal AAA between 2011 and 2021. Input features included 36 preoperative variables. The primary outcome was 30-day major adverse cardiovascular event (composite of myocardial infarction, stroke, or death). Data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, 6 ML models were trained using preoperative features. The primary model evaluation metric was area under the receiver operating characteristic curve. Model robustness was evaluated with calibration plot and Brier score. Subgroup analysis was performed to assess model performance based on age, sex, race, ethnicity, and prior AAA repair.

RESULTS

Overall, 16,282 patients were included. The primary outcome of 30-day major adverse cardiovascular event occurred in 390 (2.4%) patients. Our best-performing prediction model was XGBoost, achieving an area under the receiver operating characteristic curve (95% CI) of 0.95 (0.94-0.96) compared with logistic regression [0.72 [0.70-0.74)]. The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.06. Model performance remained robust on all subgroup analyses.

CONCLUSIONS

Our newer ML models accurately predict 30-day outcomes following EVAR using preoperative data and perform better than logistic regression. Our automated algorithms can guide risk mitigation strategies for patients being considered for EVAR.

Using machine learning to predict outcomes following suprainguinal bypass

OBJECTIVE

Suprainguinal bypass for peripheral artery disease (PAD) carries important surgical risks; however, outcome prediction tools remain limited. We developed machine learning (ML) algorithms that predict outcomes following suprainguinal bypass.

METHODS

The Vascular Quality Initiative database was used to identify patients who underwent suprainguinal bypass for PAD between 2003 and 2023. We identified 100 potential predictor variables from the index hospitalization (68 preoperative [demographic/clinical], 13 intraoperative [procedural], and 19 postoperative [in-hospital course/complications]). The primary outcomes were major adverse limb events (MALE; composite of untreated loss of patency, thrombectomy/thrombolysis, surgical revision, or major amputation) or death at 1 year following suprainguinal bypass. Our data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, we trained six ML models using preoperative features (Extreme Gradient Boosting [XGBoost], random forest, Naïve Bayes classifier, support vector machine, artificial neural network, and logistic regression). The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). The best performing algorithm was further trained using intra- and postoperative data. Model robustness was evaluated using calibration plots and Brier scores. Performance was assessed on subgroups based on age, sex, race, ethnicity, rurality, median Area Deprivation Index, symptom status, procedure type, prior intervention for PAD, concurrent interventions, and urgency.

RESULTS

Overall, 16,832 patients underwent suprainguinal bypass, and 3136 (18.6%) developed 1-year MALE or death. Patients with 1-year MALE or death were older (mean age, 64.9 vs 63.5 years; P < .001) with more comorbidities, had poorer functional status (65.7% vs 80.9% independent at baseline; P < .001), and were more likely to have chronic limb-threatening ischemia (67.4% vs 47.6%; P < .001) than those without an outcome. Despite being at higher cardiovascular risk, they were less likely to receive acetylsalicylic acid or statins preoperatively and at discharge. Our best performing prediction model at the preoperative stage was XGBoost, achieving an AUROC of 0.92 (95% confidence interval [CI], 0.91-0.93). In comparison, logistic regression had an AUROC of 0.67 (95% CI, 0.65-0.69). Our XGBoost model maintained excellent performance at the intra- and postoperative stages, with AUROCs of 0.93 (95% CI, 0.92-0.94) and 0.98 (95% CI, 0.97-0.99), respectively. Calibration plots showed good agreement between predicted and observed event probabilities with Brier scores of 0.12 (preoperative), 0.11 (intraoperative), and 0.10 (postoperative). Of the top 10 predictors, nine were preoperative features including chronic limb-threatening ischemia, previous procedures, comorbidities, and functional status. Model performance remained robust on all subgroup analyses.

CONCLUSIONS

We developed ML models that accurately predict outcomes following suprainguinal bypass, performing better than logistic regression. Our algorithms have potential for important utility in guiding perioperative risk mitigation strategies to prevent adverse outcomes following suprainguinal bypass.

ChatGPT in surgery: a revolutionary innovation?

ChatGPT has brought about a new era of digital health, as this model has become prominent and been rapidly developing since its release. ChatGPT may be able to facilitate improvements in surgery as well; however, the influence of ChatGPT on surgery is largely unknown at present. Therefore, the present study reports on the current applications of ChatGPT in the field of surgery, evaluating its workflow, practical implementations, limitations, and future perspectives. A literature search was performed using the PubMed and Embase databases. The initial search was performed from its inception until July 2023. This study revealed that ChatGPT has promising capabilities in areas of surgical research, education, training, and practice. In daily practice, surgeons and surgical residents can be aided in performing logistics and administrative tasks, and patients can be more efficiently informed about the details of their condition. However, priority should be given to establishing proper policies and protocols to ensure the safe and reliable use of this model.

Decision Curve Analysis of In-Hospital Mortality Prediction Models: The Relative Value of Pre- and Intraoperative Data For Decision-Making

BACKGROUND

Clinical prediction modeling plays a pivotal part in modern clinical care, particularly in predicting the risk of in-hospital mortality. Recent modeling efforts have focused on leveraging intraoperative data sources to improve model performance. However, the individual and collective benefit of pre- and intraoperative data for clinical decision-making remains unknown. We hypothesized that pre- and intraoperative predictors contribute equally to the net benefit in a decision curve analysis (DCA) of in-hospital mortality prediction models that include pre- and intraoperative predictors.

METHODS

Data from the VitalDB database featuring a subcohort of 6043 patients were used. A total of 141 predictors for in-hospital mortality were grouped into preoperative (demographics, intervention characteristics, and laboratory measurements) and intraoperative (laboratory and monitor data, drugs, and fluids) data. Prediction models using either preoperative, intraoperative, or all data were developed with multiple methods (logistic regression, neural network, random forest, gradient boosting machine, and a stacked learner). Predictive performance was evaluated by the area under the receiver-operating characteristic curve (AUROC) and under the precision-recall curve (AUPRC). Clinical utility was examined with a DCA in the predefined risk preference range (denoted by so-called treatment threshold probabilities) between 0% and 20%.

RESULTS

AUROC performance of the prediction models ranged from 0.53 to 0.78. AUPRC values ranged from 0.02 to 0.25 (compared to the incidence of 0.09 in our dataset) and high AUPRC values resulted from prediction models based on preoperative laboratory values. A DCA of pre- and intraoperative prediction models highlighted that preoperative data provide the largest overall benefit for decision-making, whereas intraoperative values provide only limited benefit for decision-making compared to preoperative data. While preoperative demographics, comorbidities, and surgery-related data provide the largest benefit for low treatment thresholds up to 5% to 10%, preoperative laboratory measurements become the dominant source for decision support for higher thresholds.

CONCLUSIONS

When it comes to predicting in-hospital mortality and subsequent decision-making, preoperative demographics, comorbidities, and surgery-related data provide the largest benefit for clinicians with risk-averse preferences, whereas preoperative laboratory values provide the largest benefit for decision-makers with more moderate risk preferences. Our decision-analytic investigation of different predictor categories moves beyond the question of whether certain predictors provide a benefit in traditional performance metrics (eg, AUROC). It offers a nuanced perspective on for whom these predictors might be beneficial in clinical decision-making. Follow-up studies requiring larger datasets and dedicated deep-learning models to handle continuous intraoperative data are essential to examine the robustness of our results.

Predicting outcomes following lower extremity open revascularization using machine learning

Lower extremity open revascularization is a treatment option for peripheral artery disease that carries significant peri-operative risks; however, outcome prediction tools remain limited. Using machine learning (ML), we developed automated algorithms that predict 30-day outcomes following lower extremity open revascularization. The National Surgical Quality Improvement Program targeted vascular database was used to identify patients who underwent lower extremity open revascularization for chronic atherosclerotic disease between 2011 and 2021. Input features included 37 pre-operative demographic/clinical variables. The primary outcome was 30-day major adverse limb event (MALE; composite of untreated loss of patency, major reintervention, or major amputation) or death. Our data were split into training (70%) and test (30%) sets. Using tenfold cross-validation, we trained 6 ML models. Overall, 24,309 patients were included. The primary outcome of 30-day MALE or death occurred in 2349 (9.3%) patients. Our best performing prediction model was XGBoost, achieving an area under the receiver operating characteristic curve (95% CI) of 0.93 (0.92-0.94). The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.08. Our ML algorithm has potential for important utility in guiding risk mitigation strategies for patients being considered for lower extremity open revascularization to improve outcomes.

Prediction of multiclass surgical outcomes in glaucoma using multimodal deep learning based on free-text operative notes and structured EHR data

OBJECTIVE

Surgical outcome prediction is challenging but necessary for postoperative management. Current machine learning models utilize pre- and post-op data, excluding intraoperative information in surgical notes. Current models also usually predict binary outcomes even when surgeries have multiple outcomes that require different postoperative management. This study addresses these gaps by incorporating intraoperative information into multimodal models for multiclass glaucoma surgery outcome prediction.

MATERIALS AND METHODS

We developed and evaluated multimodal deep learning models for multiclass glaucoma trabeculectomy surgery outcomes using both structured EHR data and free-text operative notes. We compare those to baseline models that use structured EHR data exclusively, or neural network models that leverage only operative notes.

RESULTS

The multimodal neural network had the highest performance with a macro AUROC of 0.750 and F1 score of 0.583. It outperformed the baseline machine learning model with structured EHR data alone (macro AUROC of 0.712 and F1 score of 0.486). Additionally, the multimodal model achieved the highest recall (0.692) for hypotony surgical failure, while the surgical success group had the highest precision (0.884) and F1 score (0.775).

DISCUSSION

This study shows that operative notes are an important source of predictive information. The multimodal predictive model combining perioperative notes and structured pre- and post-op EHR data outperformed other models. Multiclass surgical outcome prediction can provide valuable insights for clinical decision-making.

CONCLUSIONS

Our results show the potential of deep learning models to enhance clinical decision-making for postoperative management. They can be applied to other specialties to improve surgical outcome predictions.

Machine learning models to predict success of endoscopic sleeve gastroplasty using total and excess weight loss percent achievement: a multicentre study

BACKGROUND

The large amount of heterogeneous data collected in surgical/endoscopic practice calls for data-driven approaches as machine learning (ML) models. The aim of this study was to develop ML models to predict endoscopic sleeve gastroplasty (ESG) efficacy at 12 months defined by total weight loss (TWL) % and excess weight loss (EWL) % achievement. Multicentre data were used to enhance generalizability: evaluate consistency among different center of ESG practice and assess reproducibility of the models and possible clinical application. Models were designed to be dynamic and integrate follow-up clinical data into more accurate predictions, possibly assisting management and decision-making.

METHODS

ML models were developed using data of 404 ESG procedures performed at 12 centers across Europe. Collected data included clinical and demographic variables at the time of ESG and at follow-up. Multicentre/external and single center/internal and temporal validation were performed. Training and evaluation of the models were performed on Python's scikit-learn library. Performance of models was quantified as receiver operator curve (ROC-AUC), sensitivity, specificity, and calibration plots.

RESULTS

Multicenter external validation: ML models using preoperative data show poor performance. Best performances were reached by linear regression (LR) and support vector machine models for TWL% and EWL%, respectively, (ROC-AUC: TWL% 0.87, EWL% 0.86) with the addition of 6-month follow-up data. Single-center internal validation: Preoperative data only ML models show suboptimal performance. Early, i.e., 3-month follow-up data addition lead to ROC-AUC of 0.79 (random forest classifiers model) and 0.81 (LR models) for TWL% and EWL% achievement prediction, respectively. Single-center temporal validation shows similar results.

CONCLUSIONS

Although preoperative data only may not be sufficient for accurate postoperative predictions, the ability of ML models to adapt and evolve with the patients changes could assist in providing an effective and personalized postoperative care. ML models predictive capacity improvement with follow-up data is encouraging and may become a valuable support in patient management and decision-making.

Using machine learning to predict outcomes following open abdominal aortic aneurysm repair

OBJECTIVE

Prediction of outcomes following open abdominal aortic aneurysm (AAA) repair remains challenging with a lack of widely used tools to guide perioperative management. We developed machine learning (ML) algorithms that predict outcomes following open AAA repair.

METHODS

The Vascular Quality Initiative (VQI) database was used to identify patients who underwent elective open AAA repair between 2003 and 2023. Input features included 52 preoperative demographic/clinical variables. All available preoperative variables from VQI were used to maximize predictive performance. The primary outcome was in-hospital major adverse cardiovascular event (MACE; composite of myocardial infarction, stroke, or death). Secondary outcomes were individual components of the primary outcome, other in-hospital complications, and 1-year mortality and any reintervention. We split our data into training (70%) and test (30%) sets. Using 10-fold cross-validation, six ML models were trained using preoperative features (Extreme Gradient Boosting [XGBoost], random forest, Naïve Bayes classifier, support vector machine, artificial neural network, and logistic regression). The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). Model robustness was evaluated with calibration plot and Brier score. The top 10 predictive features in our final model were determined based on variable importance scores. Performance was assessed on subgroups based on age, sex, race, ethnicity, rurality, median area deprivation index, proximal clamp site, prior aortic surgery, and concomitant procedures.

RESULTS

Overall, 12,027 patients were included. The primary outcome of in-hospital MACE occurred in 630 patients (5.2%). Compared with patients without a primary outcome, those who developed in-hospital MACE were older with more comorbidities, demonstrated poorer functional status, had more complex aneurysms, and were more likely to require concomitant procedures. Our best performing prediction model for in-hospital MACE was XGBoost, achieving an AUROC of 0.93 (95% confidence interval, 0.92-0.94). Comparatively, logistic regression had an AUROC of 0.71 (95% confidence interval, 0.70-0.73). For secondary outcomes, XGBoost achieved AUROCs between 0.84 and 0.94. The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.05. These findings highlight the excellent predictive performance of the XGBoost model. The top three predictive features in our algorithm for in-hospital MACE following open AAA repair were: (1) coronary artery disease; (2) American Society of Anesthesiologists classification; and (3) proximal clamp site. Model performance remained robust on all subgroup analyses.

CONCLUSIONS

Open AAA repair outcomes can be accurately predicted using preoperative data with our ML models, which perform better than logistic regression. Our automated algorithms can help guide risk-mitigation strategies for patients being considered for open AAA repair to improve outcomes.

Predicting outcomes following open revascularization for aortoiliac occlusive disease using machine learning

OBJECTIVE

Open surgical treatment options for aortoiliac occlusive disease carry significant perioperative risks; however, outcome prediction tools remain limited. Using machine learning (ML), we developed automated algorithms that predict 30-day outcomes following open aortoiliac revascularization.

METHODS

The National Surgical Quality Improvement Program (NSQIP) targeted vascular database was used to identify patients who underwent open aortoiliac revascularization for atherosclerotic disease between 2011 and 2021. Input features included 38 preoperative demographic/clinical variables. The primary outcome was 30-day major adverse limb event (MALE; composite of untreated loss of patency, major reintervention, or major amputation) or death. The 30-day secondary outcomes were individual components of the primary outcome, major adverse cardiovascular event (MACE; composite of myocardial infarction, stroke, or death), individual components of MACE, wound complication, bleeding, other morbidity, non-home discharge, and unplanned readmission. Our data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, we trained six ML models using preoperative features. The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). Model robustness was evaluated with calibration plot and Brier score. Variable importance scores were calculated to determine the top 10 predictive features. Performance was assessed on subgroups based on age, sex, race, ethnicity, symptom status, procedure type, and urgency.

RESULTS

Overall, 9649 patients were included. The primary outcome of 30-day MALE or death occurred in 1021 patients (10.6%). Our best performing prediction model for 30-day MALE or death was XGBoost, achieving an AUROC of 0.95 (95% confidence interval [CI], 0.94-0.96). In comparison, logistic regression had an AUROC of 0.79 (95% CI, 0.77-0.81). For 30-day secondary outcomes, XGBoost achieved AUROCs between 0.87 and 0.97 (untreated loss of patency [0.95], major reintervention [0.88], major amputation [0.96], death [0.97], MACE [0.95], myocardial infarction [0.88], stroke [0.93], wound complication [0.94], bleeding [0.87], other morbidity [0.96], non-home discharge [0.90], and unplanned readmission [0.91]). The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.05. The strongest predictive feature in our algorithm was chronic limb-threatening ischemia. Model performance remained robust on all subgroup analyses of specific demographic/clinical populations.

CONCLUSIONS

Our ML models accurately predict 30-day outcomes following open aortoiliac revascularization using preoperative data, performing better than logistic regression. They have potential for important utility in guiding risk-mitigation strategies for patients being considered for open aortoiliac revascularization to improve outcomes.

Machine learning to predict outcomes following endovascular abdominal aortic aneurysm repair

BACKGROUND

Endovascular aneurysm repair (EVAR) for abdominal aortic aneurysm (AAA) carries important perioperative risks; however, there are no widely used outcome prediction tools. The aim of this study was to apply machine learning (ML) to develop automated algorithms that predict 1-year mortality following EVAR.

METHODS

The Vascular Quality Initiative database was used to identify patients who underwent elective EVAR for infrarenal AAA between 2003 and 2023. Input features included 47 preoperative demographic/clinical variables. The primary outcome was 1-year all-cause mortality. Data were split into training (70 per cent) and test (30 per cent) sets. Using 10-fold cross-validation, 6 ML models were trained using preoperative features with logistic regression as the baseline comparator. The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). Model robustness was evaluated with calibration plot and Brier score.

RESULTS

Some 63 655 patients were included. One-year mortality occurred in 3122 (4.9 per cent) patients. The best performing prediction model for 1-year mortality was XGBoost, achieving an AUROC (95 per cent c.i.) of 0.96 (0.95-0.97). Comparatively, logistic regression had an AUROC (95 per cent c.i.) of 0.69 (0.68-0.71). The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.04. The top 3 predictive features in the algorithm were 1) unfit for open AAA repair, 2) functional status, and 3) preoperative dialysis.

CONCLUSIONS

In this data set, machine learning was able to predict 1-year mortality following EVAR using preoperative data and outperformed standard logistic regression models.

Predicting Major Adverse Cardiovascular Events Following Carotid Endarterectomy Using Machine Learning

Background Carotid endarterectomy (CEA) is a major vascular operation for stroke prevention that carries significant perioperative risks; however, outcome prediction tools remain limited. The authors developed machine learning algorithms to predict outcomes following CEA. Methods and Results The National Surgical Quality Improvement Program targeted vascular database was used to identify patients who underwent CEA between 2011 and 2021. Input features included 36 preoperative demographic/clinical variables. The primary outcome was 30-day major adverse cardiovascular events (composite of stroke, myocardial infarction, or death). The data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, 6 machine learning models were trained using preoperative features. The primary metric for evaluating model performance was area under the receiver operating characteristic curve. Model robustness was evaluated with calibration plot and Brier score. Overall, 38 853 patients underwent CEA during the study period. Thirty-day major adverse cardiovascular events occurred in 1683 (4.3%) patients. The best performing prediction model was XGBoost, achieving an area under the receiver operating characteristic curve of 0.91 (95% CI, 0.90-0.92). In comparison, logistic regression had an area under the receiver operating characteristic curve of 0.62 (95% CI, 0.60-0.64), and existing tools in the literature demonstrate area under the receiver operating characteristic curve values ranging from 0.58 to 0.74. The calibration plot showed good agreement between predicted and observed event probabilities with a Brier score of 0.02. The strongest predictive feature in our algorithm was carotid symptom status. Conclusions The machine learning models accurately predicted 30-day outcomes following CEA using preoperative data and performed better than existing tools. They have potential for important utility in guiding risk-mitigation strategies to improve outcomes for patients being considered for CEA.

The prediction of sagittal chin point relapse following two-jaw surgery using machine learning

The study aimed to identify critical factors associated with the surgical stability of pogonion (Pog) by applying machine learning (ML) to predict relapse following two-jaw orthognathic surgery (2 J-OGJ). The sample set comprised 227 patients (110 males and 117 females, 207 training and 20 test sets). Using lateral cephalograms taken at the initial evaluation (T0), pretreatment (T1), after (T2) 2 J-OGS, and post treatment (T3), 55 linear and angular skeletal and dental surgical movements (T2-T1) were measured. Six ML modes were utilized, including classification and regression trees (CART), conditional inference tree (CTREE), and random forest (RF). The training samples were classified into three groups; highly significant (HS) (≥ 4), significant (S) (≥ 2 and < 4), and insignificant (N), depending on Pog relapse. RF indicated that the most important variable that affected relapse rank prediction was ramus inclination (RI), CTREE and CART revealed that a clockwise rotation of more than 3.7 and 1.8 degrees of RI was a risk factor for HS and S groups, respectively. RF, CTREE, and CART were practical tools for predicting surgical stability. More than 1.8 degrees of CW rotation of the ramus during surgery would lead to significant Pog relapse.

Algorithmic bias and research integrity; the role of nonhuman authors in shaping scientific knowledge with respect to artificial intelligence: a perspective

Artificial intelligence technologies were developed to assist authors in bettering the organization and caliber of their published papers, which are both growing in quantity and sophistication. Even though the usage of artificial intelligence tools in particular ChatGPT's natural language processing systems has been shown to be beneficial in research, there are still concerns about accuracy, responsibility, and transparency when it comes to the norms regarding authorship credit and contributions. Genomic algorithms quickly examine large amounts of genetic data to identify potential disease-causing mutations. By analyzing millions of medications for potential therapeutic benefits, they can quickly and relatively economically find novel approaches to treatment. Researchers from several fields can collaborate on difficult tasks with the assistance of nonhuman writers, promoting interdisciplinary research. Sadly, there are a number of significant disadvantages associated with employing nonhuman authors, including the potential for algorithmic prejudice. Biased data may be reinforced by the algorithm since machine learning algorithms can only be as objective as the data they are trained on. It is overdue that scholars bring forth basic moral concerns in the fight against algorithmic prejudice. Overall, even if the use of nonhuman authors has the potential to significantly improve scientific research, it is crucial for scientists to be aware of these drawbacks and take precautions to avoid bias and limits. To provide accurate and objective results, algorithms must be carefully designed and implemented, and researchers need to be mindful of the larger ethical ramifications of their usage.

Using machine learning to predict outcomes following carotid endarterectomy

OBJECTIVE

Prediction of outcomes following carotid endarterectomy (CEA) remains challenging, with a lack of standardized tools to guide perioperative management. We used machine learning (ML) to develop automated algorithms that predict outcomes following CEA.

METHODS

The Vascular Quality Initiative (VQI) database was used to identify patients who underwent CEA between 2003 and 2022. We identified 71 potential predictor variables (features) from the index hospitalization (43 preoperative [demographic/clinical], 21 intraoperative [procedural], and 7 postoperative [in-hospital complications]). The primary outcome was stroke or death at 1 year following CEA. Our data were split into training (70%) and test (30%) sets. Using 10-fold cross-validation, we trained six ML models using preoperative features (Extreme Gradient Boosting [XGBoost], random forest, Naïve Bayes classifier, support vector machine, artificial neural network, and logistic regression). The primary model evaluation metric was area under the receiver operating characteristic curve (AUROC). After selecting the best performing algorithm, additional models were built using intra- and postoperative data. Model robustness was evaluated using calibration plots and Brier scores. Performance was assessed on subgroups based on age, sex, race, ethnicity, insurance status, symptom status, and urgency of surgery.

RESULTS

Overall, 166,369 patients underwent CEA during the study period. In total, 7749 patients (4.7%) had the primary outcome of stroke or death at 1 year. Patients with an outcome were older with more comorbidities, had poorer functional status, and demonstrated higher risk anatomic features. They were also more likely to undergo intraoperative surgical re-exploration and have in-hospital complications. Our best performing prediction model at the preoperative stage was XGBoost, achieving an AUROC of 0.90 (95% confidence interval [CI], 0.89-0.91). In comparison, logistic regression had an AUROC of 0.65 (95% CI, 0.63-0.67), and existing tools in the literature demonstrate AUROCs ranging from 0.58 to 0.74. Our XGBoost models maintained excellent performance at the intra- and postoperative stages, with AUROCs of 0.90 (95% CI, 0.89-0.91) and 0.94 (95% CI, 0.93-0.95), respectively. Calibration plots showed good agreement between predicted and observed event probabilities with Brier scores of 0.15 (preoperative), 0.14 (intraoperative), and 0.11 (postoperative). Of the top 10 predictors, eight were preoperative features, including comorbidities, functional status, and previous procedures. Model performance remained robust on all subgroup analyses.

CONCLUSIONS

We developed ML models that accurately predict outcomes following CEA. Our algorithms perform better than logistic regression and existing tools, and therefore, have potential for important utility in guiding perioperative risk mitigation strategies to prevent adverse outcomes.

Prediction of postoperative complications after oesophagectomy using machine-learning methods

BACKGROUND

Oesophagectomy is an operation with a high risk of postoperative complications. The aim of this single-centre retrospective study was to apply machine-learning methods to predict complications (Clavien-Dindo grade IIIa or higher) and specific adverse events.

METHODS

Patients with resectable adenocarcinoma or squamous cell carcinoma of the oesophagus and gastro-oesophageal junction who underwent Ivor Lewis oesophagectomy between 2016 and 2021 were included. The tested algorithms were logistic regression after recursive feature elimination, random forest, k-nearest neighbour, support vector machine, and neural network. The algorithms were also compared with a current risk score (the Cologne risk score).

RESULTS

457 patients had Clavien-Dindo grade IIIa or higher complications (52.9 per cent) versus 407 patients with Clavien-Dindo grade 0, I, or II complications (47.1 per cent). After 3-fold imputation and 3-fold cross-validation, the overall accuracies were: logistic regression after recursive feature elimination, 0.528; random forest, 0.535; k-nearest neighbour, 0.491; support vector machine, 0.511; neural network, 0.688; and Cologne risk score, 0.510. For medical complications, the results were: logistic regression after recursive feature elimination, 0.688; random forest, 0.664; k-nearest neighbour, 0.673; support vector machine, 0.681; neural network, 0.692; and Cologne risk score, 0.650. For surgical complications, the results were: logistic regression after recursive feature elimination, 0.621; random forest, 0.617; k-nearest neighbour, 0.620; support vector machine, 0.634; neural network, 0.667; and Cologne risk score, 0.624. The calculated area under the curve of the neural network was 0.672 for Clavien-Dindo grade IIIa or higher, 0.695 for medical complications, and 0.653 for surgical complications.

CONCLUSION

The neural network scored the highest accuracies compared with all of the other models for the prediction of postoperative complications after oesophagectomy.

Artificial Intelligence in Neurosurgery: A State-of-the-Art Review from Past to Future

In recent years, there has been a significant surge in discussions surrounding artificial intelligence (AI), along with a corresponding increase in its practical applications in various facets of everyday life, including the medical industry. Notably, even in the highly specialized realm of neurosurgery, AI has been utilized for differential diagnosis, pre-operative evaluation, and improving surgical precision. Many of these applications have begun to mitigate risks of intraoperative and postoperative complications and post-operative care. This article aims to present an overview of the principal published papers on the significant themes of tumor, spine, epilepsy, and vascular issues, wherein AI has been applied to assess its potential applications within neurosurgery. The method involved identifying high-cited seminal papers using PubMed and Google Scholar, conducting a comprehensive review of various study types, and summarizing machine learning applications to enhance understanding among clinicians for future utilization. Recent studies demonstrate that machine learning (ML) holds significant potential in neuro-oncological care, spine surgery, epilepsy management, and other neurosurgical applications. ML techniques have proven effective in tumor identification, surgical outcomes prediction, seizure outcome prediction, aneurysm prediction, and more, highlighting its broad impact and potential in improving patient management and outcomes in neurosurgery. This review will encompass the current state of research, as well as predictions for the future of AI within neurosurgery.

Data-Driven Technologies as Enablers for Value Creation in the Prevention of Surgical Site Infections: a Systematic Review

Despite the advances in modern medicine, the use of data-driven technologies (DDTs) to prevent surgical site infections (SSIs) remains a major challenge. Scholars recognise that data management is the next frontier in infection prevention, but many aspects related to the benefits and advantages of using DDTs to mitigate SSI risk factors remain unclear and underexplored in the literature. This study explores how DDTs enable value creation in the prevention of SSIs. This study follows a systematic literature review approach and the PRISMA statement to analyse peer-reviewed articles from seven databases. Fifty-nine articles were included in the review and were analysed through a descriptive and a thematic analysis. The findings suggest a growing interest in DDTs in SSI prevention in the last 5 years, and that machine learning and smartphone applications are widely used in SSI prevention. DDTs are mainly applied to prevent SSIs in clean and clean-contaminated surgeries and often used to manage patient-related data in the postoperative stage. DDTs enable the creation of nine categories of value that are classified in four dimensions: cost/sacrifice, functional/instrumental, experiential/hedonic, and symbolic/expressive. This study offers a unique and systematic overview of the value creation aspects enabled by DDT applications in SSI prevention and suggests that additional research is needed in four areas: value co-creation and product-service systems, DDTs in contaminated and dirty surgeries, data legitimation and explainability, and data-driven interventions.

Supplementary Information

The online version contains supplementary material available at 10.1007/s41666-023-00129-2.

Cardiac surgery risk prediction using ensemble machine learning to incorporate legacy risk scores: A benchmarking study

Objective

The introduction of new clinical risk scores (e.g. European System for Cardiac Operative Risk Evaluation (EuroSCORE) II) superseding original scores (e.g. EuroSCORE I) with different variable sets typically result in disparate datasets due to high levels of missingness for new score variables prior to time of adoption. Little is known about the use of ensemble learning to incorporate disparate data from legacy scores. We tested the hypothesised that Homogenenous and Heterogeneous Machine Learning (ML) ensembles will have better performance than ensembles of Dynamic Model Averaging (DMA) for combining knowledge from EuroSCORE I legacy data with EuroSCORE II data to predict cardiac surgery risk.

Methods

Using the National Adult Cardiac Surgery Audit dataset, we trained 12 different base learner models, based on two different variable sets from either EuroSCORE I (LogES) or EuroScore II (ES II), partitioned by the time of score adoption (1996-2016 or 2012-2016) and evaluated on holdout set (2017-2019). These base learner models were ensembled using nine different combinations of six ML algorithms to produce homogeneous or heterogeneous ensembles. Performance was assessed using a consensus metric.

Results

Xgboost homogenous ensemble (HE) was the highest performing model (clinical effectiveness metric (CEM) 0.725) with area under the curve (AUC) (0.8327; 95% confidence interval (CI) 0.8323-0.8329) followed by Random Forest HE (CEM 0.723; AUC 0.8325; 95%CI 0.8320-0.8326). Across different heterogenous ensembles, significantly better performance was obtained by combining siloed datasets across time (CEM 0.720) than building ensembles of either 1996-2011 (t-test adjusted, p = 1.67×10-6) or 2012-2019 (t-test adjusted, p = 1.35×10-193) datasets alone.

Conclusions

Both homogenous and heterogenous ML ensembles performed significantly better than DMA ensemble of Bayesian Update models. Time-dependent ensemble combination of variables, having differing qualities according to time of score adoption, enabled previously siloed data to be combined, leading to increased power, clinical interpretability of variables and usage of data.

Predicting Length of Stay using machine learning for total joint replacements performed at a rural community hospital

Background

Predicting patient’s Length of Stay (LOS) before total joint replacement (TJR) surgery is vital for hospitals to optimally manage costs and resources. Many hospitals including in rural areas use publicly available models such as National Surgical Quality Improvement Program (NSQIP) calculator which, unfortunately, performs suboptimally when predicting LOS for TJR procedures.

Objective

The objective of this research was to develop a Machine Learning (ML) model to predict LOS for TJR procedures performed at a Perioperative Surgical Home implemented rural community hospital for better accuracy and interpretation than the NSQIP calculator.

Methods

A total of 158 TJR patients were collected and analyzed from a rural community hospital located in Montana. A random forest (RF) model was used to predict patient’s LOS. For interpretation, permuted feature importance and partial dependence plot methods were used to identify the important variables and their relationship with the LOS.

Results

The root mean square error for the RF model (0.7) was lower than the NSQIP calculator (1.21). The five most important variables for predicting LOS were BMI, Duke Activity Status-Index, diabetes, patient’s household income, and patient’s age.

Conclusion

This pilot study is the first of its kind to develop an ML model to predict LOS for TJR procedures that were performed at a small-scale rural community hospital. This pilot study contributes an approach for rural hospitals, making them more independent by developing their own predictions instead of relying on public models.

ICD-10 based machine learning models outperform the Trauma and Injury Severity Score (TRISS) in survival prediction

Background

Precise models are necessary to estimate mortality risk following traumatic injury to inform clinical decision making or quantify hospital performance. The Trauma and Injury Severity Score (TRISS) has been the historical gold standard in survival prediction but its limitations are well-characterized. The present study used International Classification of Diseases 10^thRevision (ICD-10) injury codes with machine learning approaches to develop models whose performance was compared to that of TRISS.

Methods

The 2015–2017 National Trauma Data Bank was used to identify patients following trauma-related admission. Injury codes from ICD-10 were grouped by clinical relevance into 1,495 variables. The TRISS score, which comprises the Injury Severity Score, age, mechanism (blunt vs penetrating) as well as highest 24-hour values for systolic blood pressure (SBP), respiratory rate (RR) and Glasgow Coma Scale (GCS) was calculated for each patient. A base eXtreme gradient boosting model (XGBoost), a machine learning technique, was developed using injury variables as well as age, SBP, RR, mechanism and GCS. Prediction of in-hospital survival and other in-hospital complications were compared between both models using receiver operating characteristic (ROC) and reliability plots. A complete XGBoost model, containing injury variables, vitals, demographic information and comorbidities, was additionally developed.

Results

Of 1,380,740 patients, 1,338,417 (96.9%) survived to discharge. Compared to survivors, those who died were older and had a greater prevalence of penetrating injuries (18.0% vs 9.44%). The base XGBoost model demonstrated a greater receiver-operating characteristic (ROC) than TRISS (0.950 vs 0.907) which persisted across sub-populations and secondary endpoints. Furthermore, it exhibited high calibration across all risk levels (R² = 0.998 vs 0.816). The complete XGBoost model had an exceptional ROC of 0.960.

Conclusions

We report improved performance of machine learning models over TRISS. Our model may improve stratification of injury severity in clinical and quality improvement settings.

At the Crossroads of Minimally Invasive Mitral Valve Surgery—Benching Single Hospital Experience to a National Registry: A Plea for Risk Management Technology

Almost 30 years after the first endoscopic mitral valve repair, Minimally Invasive Mitral Valve Surgery (MIMVS) has become the standard at many institutions due to optimal clinical results and fast recovery. The question that arises is can already good results be further improved by an Institutional Risk Management Performance (IRMP) system in decreasing risks in minimally invasive mitral valve surgery (MIMVS)? As of yet, there are no reports on IRMP and learning systems in the literature. (2) Methods: We described and appraised our five-year single institutional experience with MIMVS in isolated valve surgery included in the Netherlands Heart Registry (NHR) and investigated root causes of high-impact complications. (3) Results: The 120-day and 12-month mortality were 1.1% and 1.9%, respectively, compared to the average of 4.3% and 5.3% reported in the NHR. The regurgitation rate was 1.4% compared to 5.2% nationwide. The few high-impact complications appeared not to be preventable. (4) Discussion: In MIMVS, freedom from major and minor complications is a strong indicator of an effective IRMP but remains concealed from physicians and patients, despite its relevance to shared decision making. Innovation adds to the complexity of MIMVS and challenges surgical competence. An IRMP system may detect and control new risks earlier. (5) Conclusion: An IRMP system contributes to an effective reduction of risks, pain and discomfort; provides relevant input for shared decision making; and warrants the safe introduction of new technology. Crossroads conclusions: investment in machine learning and AI for an effective IRMP system is recommended and the roles for commanding and operating surgeons should be considered.

Conventional regression analysis and machine learning in prediction of anastomotic leakage and pulmonary complications after esophagogastric cancer surgery

Background and Objectives

With the current advanced data‐driven approach to health care, machine learning is gaining more interest. The current study investigates the added value of machine learning to linear regression in predicting anastomotic leakage and pulmonary complications after upper gastrointestinal cancer surgery.

Methods

All patients in the Dutch Upper Gastrointestinal Cancer Audit undergoing curatively intended esophageal or gastric cancer surgeries from 2011 to 2017 were included. Anastomotic leakage was defined as any clinically or radiologically proven anastomotic leakage. Pulmonary complications entailed: pneumonia, pleural effusion, respiratory failure, pneumothorax, and/or acute respiratory distress syndrome. Different machine learning models were tested. Nomograms were constructed using Least Absolute Shrinkage and Selection Operator.

Results

Between 2011 and 2017, 4228 patients underwent surgical resection for esophageal cancer, of which 18% developed anastomotic leakage and 30% a pulmonary complication. Of the 2199 patients with surgical resection for gastric cancer, 7% developed anastomotic leakage and 15% a pulmonary complication. In all cases, linear regression had the highest predictive value with the area under the curves varying between 61.9 and 68.0, but the difference with machine learning models did not reach statistical significance.

Conclusion

Machine learning models can predict postoperative complications in upper gastrointestinal cancer surgery, but they do not outperform the current gold standard, linear regression

Machine learning in vascular surgery: a systematic review and critical appraisal

Machine learning (ML) is a rapidly advancing field with increasing utility in health care. We conducted a systematic review and critical appraisal of ML applications in vascular surgery. MEDLINE, Embase, and Cochrane CENTRAL were searched from inception to March 1, 2021. Study screening, data extraction, and quality assessment were performed by two independent reviewers, with a third author resolving discrepancies. All original studies reporting ML applications in vascular surgery were included. Publication trends, disease conditions, methodologies, and outcomes were summarized. Critical appraisal was conducted using the PROBAST risk-of-bias and TRIPOD reporting adherence tools. We included 212 studies from a pool of 2235 unique articles. ML techniques were used for diagnosis, prognosis, and image segmentation in carotid stenosis, aortic aneurysm/dissection, peripheral artery disease, diabetic foot ulcer, venous disease, and renal artery stenosis. The number of publications on ML in vascular surgery increased from 1 (1991-1996) to 118 (2016-2021). Most studies were retrospective and single center, with no randomized controlled trials. The median area under the receiver operating characteristic curve (AUROC) was 0.88 (range 0.61-1.00), with 79.5% [62/78] studies reporting AUROC ≥ 0.80. Out of 22 studies comparing ML techniques to existing prediction tools, clinicians, or traditional regression models, 20 performed better and 2 performed similarly. Overall, 94.8% (201/212) studies had high risk-of-bias and adherence to reporting standards was poor with a rate of 41.4%. Despite improvements over time, study quality and reporting remain inadequate. Future studies should consider standardized tools such as PROBAST and TRIPOD to improve study quality and clinical applicability.

Artificial intelligence, machine learning, and deep learning for clinical outcome prediction

AI is a broad concept, grouping initiatives that use a computer to perform tasks that would usually require a human to complete. AI methods are well suited to predict clinical outcomes. In practice, AI methods can be thought of as functions that learn the outcomes accompanying standardized input data to produce accurate outcome predictions when trialed with new data. Current methods for cleaning, creating, accessing, extracting, augmenting, and representing data for training AI clinical prediction models are well defined. The use of AI to predict clinical outcomes is a dynamic and rapidly evolving arena, with new methods and applications emerging. Extraction or accession of electronic health care records and combining these with patient genetic data is an area of present attention, with tremendous potential for future growth. Machine learning approaches, including decision tree methods of Random Forest and XGBoost, and deep learning techniques including deep multi-layer and recurrent neural networks, afford unique capabilities to accurately create predictions from high dimensional, multimodal data. Furthermore, AI methods are increasing our ability to accurately predict clinical outcomes that previously were difficult to model, including time-dependent and multi-class outcomes. Barriers to robust AI-based clinical outcome model deployment include changing AI product development interfaces, the specificity of regulation requirements, and limitations in ensuring model interpretability, generalizability, and adaptability over time.