Machine learning improves mortality risk prediction after cardiac surgery: Systematic review and meta-analysis

Exploring Primary and Interaction Effects of Minor Physical Anomalies: Development and Validation of Prediction Models Using Explainable Machine Learning Algorithms for Early-Onset Schizophrenia

BACKGROUND AND HYPOTHESIS

Minor physical abnormalities (MPAs) are neurodevelopmental markers that can be traced to prenatal events and may be significant features of early-onset schizophrenia (EOS). Therefore, our study aimed to (1) find the primary and interaction effects of MPAs for EOS and (2) develop and validate the model for EOS based on explainable machine learning algorithms.

STUDY DESIGN

The study included 549 patients with schizophrenia (193 EOS and 356 AOS) and 420 healthy controls (HC) in southern Taiwan. For the feature selection, variable selection using random forests (varSelRF) and recursive feature elimination (RFE) were applied to identify the important variables of MPAs. We used different machine learning algorithms to build the prediction models based on the selected MPAs variables.

STUDY RESULTS

The results showed that the mouth anomalies are significant MPAs variables and have interaction effects with craniofacial MPAs variables for EOS. The prediction models using the selected MPAs variables performed better in discriminating EOS vs HC compared to AOS vs HC. The AUC values for distinguishing EOS vs HC were 0.85-0.93, AOS vs HC were 0.80-0.87, and EOS vs AOS were 0.67-0.77 in validation sets.

CONCLUSIONS

This risk prediction model provides a clinical decision support system for detecting patients at high risk of developing EOS and enables early intervention in clinical practice.

Recurrence patterns and prediction of survival after recurrence for gallbladder cancer

BACKGROUND

Gallbladder cancer (GBC) is associated with a poor prognosis. Recurrence patterns and their effect on survival remain ill-defined. This study aimed to analyze recurrence patterns and develop a machine learning (ML) model to predict survival after recurrence (SAR) of GBC.

METHODS

Patients who underwent curative-intent resection of GBC between 1999 and 2022 were identified using an international database. An Extreme Gradient Boosting ML model to predict SAR was developed and validated.

RESULTS

Among 348 patients, 110 (31.6%) developed disease recurrence during follow-up. The most common recurrence site was local (29.1%), followed by multiple site (26.4%), liver (21.8%), peritoneal (18.2%), and lung (0.05%). The median SAR was the longest in patients with lung recurrence (36.0 months), followed by those with local recurrence (15.7 months). In contrast, patients with peritoneal (8.9 months), liver (8.5 months), or multiple-site (6.4 months) recurrence had a considerably shorter SAR. Patients with multiple-site recurrence had a worse SAR than individuals with single-site recurrence (6.4 vs 11.10 months, respectively; P =.014). The model demonstrated good performance in the evaluation and bootstrapping cohorts (area under the receiver operating characteristic curve: 71.4 and 71.0, respectively). The most influential variables were American Society of Anesthesiologists classification, local recurrence, receipt of adjuvant chemotherapy, American Joint Committee on Cancer T and N categories, and developing early disease recurrence (<12 months). To enable clinical applicability, an easy-to-use calculator was made available (https://catalano-giovanni.shinyapps.io/SARGB).

CONCLUSION

Except for lung recurrence, SAR for GBC was poor. A subset of patients with less aggressive disease biology may have favorable SAR. ML-based SAR prediction may help individuate candidates for curative re-resection when feasible.

Comprehensive overview of artificial intelligence in surgery: a systematic review and perspectives

The rapid integration of artificial intelligence (AI) into surgical practice necessitates a comprehensive evaluation of its applications, challenges, and physiological impact. This systematic review synthesizes current AI applications in surgery, with a particular focus on machine learning (ML) and its role in optimizing preoperative planning, intraoperative decision-making, and postoperative patient management. Using PRISMA guidelines and PICO criteria, we analyzed key studies addressing AI's contributions to surgical precision, outcome prediction, and real-time physiological monitoring. While AI has demonstrated significant promise-from enhancing diagnostics to improving intraoperative safety-many surgeons remain skeptical due to concerns over algorithmic unpredictability, surgeon autonomy, and ethical transparency. This review explores AI's physiological integration into surgery, discussing its role in real-time hemodynamic assessments, AI-guided tissue characterization, and intraoperative physiological modeling. Ethical concerns, including algorithmic opacity and liability in high-stakes scenarios, are critically examined alongside AI's potential to augment surgical expertise. We conclude that longitudinal validation, improved AI explainability, and adaptive regulatory frameworks are essential to ensure safe, effective, and ethically sound integration of AI into surgical decision-making. Future research should focus on bridging AI-driven analytics with real-time physiological feedback to refine precision surgery and patient safety strategies.

Artificial intelligence in cardiovascular procedures: a bibliometric and visual analysis study

Background

The integration of artificial intelligence (AI) into cardiovascular procedures has significantly advanced diagnostic accuracy, outcome prediction, and robotic-assisted surgeries. However, a comprehensive bibliometric analysis of AI's impact in this field is lacking. This study examines research trends, key contributors, and emerging themes in AI-driven cardiovascular interventions.

Methods

We retrieved relevant publications from the Web of Science Core Collection and analyzed them using VOSviewer, CiteSpace, and Biblioshiny to map research trends and collaborations.

Results

AI-related cardiovascular research has grown substantially from 1993 to 2024, with a sharp increase from 2020 to 2023, peaking at 93 publications in 2023. The USA (127 papers), China (79), and England (31) were the top contributors, with Harvard University leading institutional output (17 papers). Frontiers in Cardiovascular Medicine was the most prolific journal. Core research themes included "machine learning," "mortality," and "cardiac surgery," with emerging trends in "association," "implantation," and "aortic stenosis," underscoring AI's expanding role in predictive modeling and surgical outcomes.

Conclusion

AI demonstrates transformative potential in cardiovascular procedures, particularly in diagnostic imaging, predictive modeling, and patient management. This bibliometric analysis highlights the growing interest in AI applications and provides a framework for integrating AI into clinical workflows to enhance diagnostic accuracy, treatment strategies, and patient outcomes.

Predicting the complexity of minimally invasive liver resection for hepatocellular carcinoma using machine learning

BACKGROUND

Despite technical advancements, minimally invasive liver surgery (MILS) for hepatocellular carcinoma (HCC) remains challenging. Nonetheless, effective tools to assess MILS complexity are still lacking. Machine learning (ML) models could improve the accuracy of such tools.

METHODS

Patients who underwent curative-intent MILS for HCC were identified using an international database. An XGBoost ML model was developed to predict surgical complexity using clinical and radiological characteristics.

RESULTS

Among 845 patients, 186 (22.0 %) were classified as high-risk patients. In this subgroup, median Charlson Comorbidity Index (CCI) (5.0, IQR 3.0-7.0 vs. 2.0, IQR 2.0-5.0, p < 0.001) and tumor burden score (TBS) (median 4.12, IQR 3.0-5.1 vs. 4.22, IQR 3.2-7.1, p < 0.001) were higher. The model was able to effectively predict complexity of surgery in both the training and testing cohorts with high discriminating power (ROC-AUC: 0.86, 95%CI 0.82-0.89 vs. 0.73, 95%CI 0.65-0.81). The most influential variables were CCI, TBS, BMI, extent of resection, and sex. Patients predicted to have a complex surgery were more likely to develop severe complications (OR 4.77, 95%CI 1.82-13.9, p = 0.002). An easy-to-use calculator was developed.

CONCLUSION

Preoperative ML-prediction of complex MILS for HCC may improve preoperative planning, resource allocation, and patient outcomes.

Five steps in performing machine learning for binary outcomes

BACKGROUND

The use of machine learning (ML) in cardiovascular and thoracic surgery is evolving rapidly. Maximizing the capabilities of ML can help improve patient risk stratification and clinical decision making, improve accuracy of predictions, and improve resource utilization in cardiac surgery. The many nuances and intricacies of ML modeling need to be understood to appropriately implement these technologies in the clinical research setting. This primer provides an educational framework of ML for generating predicted probabilities in clinical research and illustrates it with a real-world clinical example.

METHODS

We focus on modeling for binary classification and imbalanced classes, a common scenario in cardiothoracic surgery research. We present a 5-step strategy for successfully harnessing the power of ML and performing such analyses, and demonstrate our strategy using a real-world example based on data from the National Surgical Quality Improvement Program pediatric database.

CONCLUSIONS

Collaboration among surgeons, care providers, statisticians, data scientists, and information technology professionals can help to maximize the impact of ML as a powerful tool in cardiac surgery.

Personalizing patient risk of a life-altering event: An application of machine learning to hemiarch surgery

OBJECTIVE

The study objective was to assess a machine learning model's ability to predict the occurrence of life-altering events in hemiarch surgery and determine contributing patient characteristics and intraoperative factors.

METHODS

In total, 602 patients who underwent hemiarch replacement at a high-volume aortic center from 2009 to 2022 were included. Patients were randomly divided into training (80%) and testing (20%) sets with various eXtreme gradient boosting candidate models constructed to predict the risk of experiencing life-altering events, including stroke, mortality, or new renal replacement therapy requirement. A total of 64 input parameters from the index hospitalization were identified, including 24 demographic characteristics as well as 8 preoperative and 32 intraoperative variables. A SHapley Additive exPlanation beeswarm plot was generated to identify and interpret the impact of individual features on the predictions of the final model.

RESULTS

A life-altering event was noted in 15% (90/602) of patients who underwent hemiarch replacement, including urgent/emergency cases and dissections. The final eXtreme Gradient Boosting model demonstrated a cross-validation accuracy of 88% on the testing set and was well calibrated as evidenced by a low Brier score of 0.12. The best performing model achieved an area under the receiver operating characteristic curve of 0.76 and an area under the precision recall curve of 0.55. The SHapley Additive exPlanation beeswarm plot provided insights into key features that significantly influenced model prediction.

CONCLUSIONS

Machine learning demonstrated superior accuracy in predicting hemiarch patients who would experience a life-altering event. This model may help to guide patients and clinicians in stratifying risk on an individual basis, which may in turn influence clinical decision-making.

Machine Learning vs Traditional Approaches to Predict All-Cause Mortality for Acute Coronary Syndrome: A Systematic Review and Meta-analysis

BACKGROUND

Acute coronary syndrome (ACS) remains one of the leading causes of death globally. Accurate and reliable mortality risk prediction of ACS patients is essential for developing targeted treatment strategies and improve prognostication. Traditional models for risk stratification such as the GRACE and TIMI risk scores offer moderate discriminative value, and do not incorporate contemporary predictors of ACS prognosis. Machine learning (ML) models have emerged as an alternate method that may offer improved risk assessment. This review compares ML models with traditional risk scores for predicting all-cause mortality in patients with ACS.

METHODS

PubMed, Embase, Web of Science, Cochrane, CINAHL, Scopus, and IEEE XPlore databases were searched through October 30, 2024, as well as Google Scholar and manual screening of reference lists from included studies and the grey literature for studies comparing ML models with traditional statistical methods for event prediction of ACS patients. The primary outcome was comparative discrimination measured by C-statistics with 95% confidence intervals (CIs) in estimating risk of all-cause mortality.

RESULTS

Twelve studies were included (250,510 patients). The summary C-statistic of best-performing ML models across all end points was 0.88 (95% CI 0.86-0.91), compared with 0.82 (95% CI 0.80-0.85) for traditional methods. The difference in C-statistic between ML models and traditional methods was 0.06 (P < 0.0007). Five studies undertook external validation. The PROBAST tool demonstrated high risk of bias for all studies. Common sources of bias included reporting bias and selection bias. Best-performing ML models demonstrated superior discrimination of all-cause mortality for ACS patients compared with traditional risk scores.

CONCLUSIONS

Despite outperforming well established prognostic tools such as the GRACE and TIMI scores, current clinical applications of ML approaches remain uncertain, particularly in view of the need for greater model validation.

Machine-learning versus traditional methods for prediction of all-cause mortality after transcatheter aortic valve implantation: a systematic review and meta-analysis

BACKGROUND

Accurate mortality prediction following transcatheter aortic valve implantation (TAVI) is essential for mitigating risk, shared decision-making and periprocedural planning. Surgical risk models have demonstrated modest discriminative value for patients undergoing TAVI and are typically poorly calibrated, with incremental improvements seen in TAVI-specific models. Machine learning (ML) models offer an alternative risk stratification that may offer improved predictive accuracy.

METHODS

PubMed, EMBASE, Web of Science and Cochrane databases were searched until 16 December 2023 for studies comparing ML models with traditional statistical methods for event prediction after TAVI. The primary outcome was comparative discrimination measured by C-statistics with 95% CIs between ML models and traditional methods in estimating the risk of all-cause mortality at 30 days and 1 year.

RESULTS

Nine studies were included (29 608 patients). The summary C-statistic of the top performing ML models was 0.79 (95% CI 0.71 to 0.86), compared with traditional methods 0.68 (95% CI 0.61 to 0.76). The difference in C-statistic between all ML models and traditional methods was 0.11 (p<0.00001). Of the nine studies, two studies provided externally validated models and three studies reported calibration. Prediction Model Risk of Bias Assessment Tool tool demonstrated high risk of bias for all studies.

CONCLUSION

ML models outperformed traditional risk scores in the discrimination of all-cause mortality following TAVI. While integration of ML algorithms into electronic healthcare systems may improve periprocedural risk stratification, immediate implementation in the clinical setting remains uncertain. Further research is required to overcome methodological and validation limitations.

Artificial Intelligence in Cardiac Surgery: Transforming Outcomes and Shaping the Future

Background: Artificial intelligence (AI) has emerged as a transformative technology in healthcare, with its integration into cardiac surgery offering significant advancements in precision, efficiency, and patient outcomes. However, a comprehensive understanding of AI's applications, benefits, challenges, and future directions in cardiac surgery is needed to inform its safe and effective implementation. Methods: A systematic review was conducted following PRISMA guidelines. Literature searches were performed in PubMed, Scopus, Cochrane Library, Google Scholar, and Web of Science, covering publications from January 2000 to November 2024. Studies focusing on AI applications in cardiac surgery, including risk stratification, surgical planning, intraoperative guidance, and postoperative management, were included. Data extraction and quality assessment were conducted using standardized tools, and findings were synthesized narratively. Results: A total of 121 studies were included in this review. AI demonstrated superior predictive capabilities in risk stratification, with machine learning models outperforming traditional scoring systems in mortality and complication prediction. Robotic-assisted systems enhanced surgical precision and minimized trauma, while computer vision and augmented cognition improved intraoperative guidance. Postoperative AI applications showed potential in predicting complications, supporting patient monitoring, and reducing healthcare costs. However, challenges such as data quality, validation, ethical considerations, and integration into clinical workflows remain significant barriers to widespread adoption. Conclusions: AI has the potential to revolutionize cardiac surgery by enhancing decision making, surgical accuracy, and patient outcomes. Addressing limitations related to data quality, bias, validation, and regulatory frameworks is essential for its safe and effective implementation. Future research should focus on interdisciplinary collaboration, robust testing, and the development of ethical and transparent AI systems to ensure equitable and sustainable advancements in cardiac surgery.

Interpretable machine learning-based predictive modeling of patient outcomes following cardiac surgery

BACKGROUND

The clinical applicability of machine learning predictions of patient outcomes following cardiac surgery remains unclear. We applied machine learning to predict patient outcomes associated with high morbidity and mortality after cardiac surgery and identified the importance of variables to the derived model's performance.

METHODS

We applied machine learning to the Society of Thoracic Surgeons Adult Cardiac Surgery Database to predict postoperative hemorrhage requiring reoperation, venous thromboembolism (VTE), and stroke. We used permutation feature importance to identify variables important to model performance and a misclassification analysis to study the limitations of the model.

RESULTS

The study dataset included 662,772 subjects who underwent cardiac surgery between 2015 and 2017 and 240 variables. Hemorrhage requiring reoperation, VTE, and stroke occurred in 2.9%, 1.2%, and 2.0% of subjects, respectively. The model performed remarkably well at predicting all 3 complications (area under the receiver operating characteristic curve, 0.92-0.97). Preoperative and intraoperative variables were not important to model performance; instead, performance for the prediction of all 3 outcomes was driven primarily by several postoperative variables, including known risk factors for the complications, such as mechanical ventilation and new onset of postoperative arrhythmias. Many of the postoperative variables important to model performance also increased the risk of subject misclassification, indicating internal validity.

CONCLUSIONS

A machine learning model accurately and reliably predicts patient outcomes following cardiac surgery. Postoperative, as opposed to preoperative or intraoperative variables, are important to model performance. Interventions targeting this period, including minimizing the duration of mechanical ventilation and early treatment of new-onset postoperative arrhythmias, may help lower the risk of these complications.

Machine learning approaches for risk prediction after percutaneous coronary intervention: a systematic review and meta-analysis

Aims

Accurate prediction of clinical outcomes following percutaneous coronary intervention (PCI) is essential for mitigating risk and peri-procedural planning. Traditional risk models have demonstrated a modest predictive value. Machine learning (ML) models offer an alternative risk stratification that may provide improved predictive accuracy.

Methods and results

This study was reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses, Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies and Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis guidelines. PubMed, EMBASE, Web of Science, and Cochrane databases were searched until 1 November 2023 for studies comparing ML models with traditional statistical methods for event prediction after PCI. The primary outcome was comparative discrimination measured by C-statistics with 95% confidence intervals (CIs) between ML models and traditional methods in estimating the risk of all-cause mortality, major bleeding, and the composite outcome major adverse cardiovascular events (MACE). Thirty-four models were included across 13 observational studies (4 105 916 patients). For all-cause mortality, the pooled C-statistic for top-performing ML models was 0.89 (95%CI, 0.84-0.91), compared with 0.86 (95% CI, 0.80-0.93) for traditional methods (P = 0.54). For major bleeding, the pooled C-statistic for ML models was 0.80 (95% CI, 0.77-0.84), compared with 0.78 (95% CI, 0.77-0.79) for traditional methods (P = 0.02). For MACE, the C-statistic for ML models was 0.83 (95% CI, 0.75-0.91), compared with 0.71 (95% CI, 0.69-0.74) for traditional methods (P = 0.007). Out of all included models, only one model was externally validated. Calibration was inconsistently reported across all models. Prediction Model Risk of Bias Assessment Tool demonstrated a high risk of bias across all studies.

Conclusion

Machine learning models marginally outperformed traditional risk scores in the discrimination of MACE and major bleeding following PCI. While integration of ML algorithms into electronic healthcare systems has been hypothesized to improve peri-procedural risk stratification, immediate implementation in the clinical setting remains uncertain. Further research is required to overcome methodological and validation limitations.

Using artificial intelligence to predict post-operative outcomes in congenital heart surgeries: a systematic review

INTRODUCTION

Congenital heart disease (CHD) represents the most common group of congenital anomalies, constitutes a significant contributor to the burden of non-communicable diseases, highlighting the critical need for improved risk assessment tools. Artificial intelligence (AI) holds promise in enhancing outcome predictions for congenital cardiac surgery. This study aims to systematically review the utilization of AI in predicting post-operative outcomes in this population.

METHODS

Following PRISMA guidelines, a comprehensive search of Pubmed, Scopus, and Web of Science databases was conducted. Two independent reviewers screened articles based on predefined criteria. Included studies focused on AI models predicting various post-operative outcomes in congenital heart surgery.

RESULTS

The review included 35 articles, primarily published within the last four years, indicating growing interest in AI applications. Models predominantly targeted mortality and survival (n = 16), prolonged length of hospital or ICU stay (n = 7), postoperative complications (n = 6), prolonged mechanical ventilatory support time (n = 4), with additional focus on specific outcomes such as peri-ventricular leucomalacia (n = 2) and malnutrition (n = 1). Performance metrics, such as area under the curve (AUC), ranged from 0.52 to 0.997. Notably, these AI models consistently outperformed traditional risk stratification categories. For instance, in assessing the risk of morbidity and mortality, the AI models demonstrated superior performance compared to conventional methods.

CONCLUSION

AI-driven prediction models show significant promise in improving outcome predictions for congenital heart surgery. They surpass traditional risk prediction tools not only in immediate postoperative risks but also in long-term outcomes such as 1-year survival and malnutrition. Further studies with robust external validation are necessary to assess the practical applicability of these models in clinical settings. The protocol of this review was prospectively registered on PROSPERO (CRD42024550942).

Machine learning prediction of early recurrence after surgery for gallbladder cancer

BACKGROUND

Gallbladder cancer is often associated with poor prognosis, especially when patients experience early recurrence after surgery. Machine learning may improve prediction accuracy by analysing complex non-linear relationships. The aim of this study was to develop and evaluate a machine learning model to predict early recurrence risk after resection of gallbladder cancer.

METHODS

In this cross-sectional study, patients who underwent resection of gallbladder cancer with curative intent between 2001 and 2022 were identified using an international database. Patients were assigned randomly to a development and an evaluation cohort. Four machine learning models were trained to predict early recurrence (within 12 months) and compared using the area under the receiver operating curve (AUC).

RESULTS

Among 374 patients, 56 (15.0%) experienced early recurrence; most patients had T1 (51, 13.6%) or T2 (180, 48.1%) disease, and a subset had lymph node metastasis (120, 32.1%). In multivariable Cox analysis, resection margins (HR 2.34, 95% c.i. 1.55 to 3.80; P < 0.001), and greater AJCC T (HR 2.14, 1.41 to 3.25; P < 0.001) and N (HR 1.59, 1.05 to 2.42; P = 0.029) categories were independent predictors of early recurrence. The random forest model demonstrated the highest discrimination in the evaluation cohort (AUC 76.4, 95% c.i. 66.3 to 86.5), compared with XGBoost (AUC 74.4, 53.4 to 85.3), support vector machine (AUC 67.2, 54.4 to 80.0), and logistic regression (AUC 73.1, 60.6 to 85.7), as well as good accuracy after bootstrapping validation (AUC 75.3, 75.0 to 75.6). Patients classified as being at high versus low risk of early recurrence had much worse overall survival (36.1 versus 63.8% respectively; P < 0.001). An easy-to-use calculator was made available (https://catalano-giovanni.shinyapps.io/GallbladderER).

CONCLUSION

Machine learning-based prediction of early recurrence after resection of gallbladder cancer may help stratify patients, as well as help inform postoperative adjuvant therapy and surveillance strategies.

The Emerging and Important Role of Artificial Intelligence in Cardiac Surgery

Artificial Intelligence (AI) has greatly affected our everyday lives and holds great promise to change the landscape of medicine. AI is particularly positioned to improve care for the increasingly complex patients undergoing cardiac surgery using the immense amount of data generated in the course of their care. When deployed, AI can be used to analyze this information at the patient's bedside more expediently and accurately, all while providing new insights. This review summarizes the current applications of AI in cardiac surgery from the vantage point of a patient's journey. Applications of AI include preoperative risk assessment, intraoperative planning, postoperative patient care, and outpatient telemonitoring, encompassing the spectrum of cardiac surgical care. Offloading of administrative processes and enhanced experience with information gathering also represent a unique and under-represented avenue for future use of AI. As clinicians, understanding the nomenclature and applications of AI is important to contextualize issues, to ensure problem-driven solutions, and for clinical benefit. Precision medicine, and thus clinically relevant AI, remains dependent on data curation and warehousing to gather insights from large multicentre repositories while treating privacy with the utmost importance. AI tasks should not be siloed but rather holistically integrated into clinical workflow to retain context and relevance. As cardiac surgeons, AI allows us to look forward to a bright future of more efficient use of our clinical expertise toward high-level decision making and technical prowess.

Logistic regression analysis and machine learning for predicting post-stroke gait independence: a retrospective study

This study investigated whether machine learning (ML) has better predictive accuracy than logistic regression analysis (LR) for gait independence at discharge in subacute stroke patients (n = 843) who could not walk independently at admission. We developed prediction models using LR and five ML algorithms-specifically, the decision tree (DT), support vector machine, artificial neural network, ensemble learning, and k-nearest neighbor methods. Functional Independence Measure sub-items were used to evaluate the ability to walk independently. Model predictive accuracies were evaluated using areas under receiver operating characteristic curves (AUCs) as well as accuracy, precision, recall, F1 score, and specificity. The AUC for DT (0.812) was significantly lower than those for the other algorithms (p < 0.01); however, the AUC for LR (0.895) did not differ significantly from those for the other models (0.893-0.903). Other performance metrics showed no substantial differences between LR and ML algorithms. In conclusion, the DT algorithm had significantly low predictive accuracy, and LR showed no significant difference in predictive accuracy compared with the other ML algorithms. As its predictive accuracy is similar to that of ML, LR can continue to be used for predicting the prognosis of gait independence, with additional advantages of being easily understandable and manually computable.

Using machine learning methods to predict all-cause somatic hospitalizations in adults: A systematic review

AIM

In this review, we investigated how Machine Learning (ML) was utilized to predict all-cause somatic hospital admissions and readmissions in adults.

METHODS

We searched eight databases (PubMed, Embase, Web of Science, CINAHL, ProQuest, OpenGrey, WorldCat, and MedNar) from their inception date to October 2023, and included records that predicted all-cause somatic hospital admissions and readmissions of adults using ML methodology. We used the CHARMS checklist for data extraction, PROBAST for bias and applicability assessment, and TRIPOD for reporting quality.

RESULTS

We screened 7,543 studies of which 163 full-text records were read and 116 met the review inclusion criteria. Among these, 45 predicted admission, 70 predicted readmission, and one study predicted both. There was a substantial variety in the types of datasets, algorithms, features, data preprocessing steps, evaluation, and validation methods. The most used types of features were demographics, diagnoses, vital signs, and laboratory tests. Area Under the ROC curve (AUC) was the most used evaluation metric. Models trained using boosting tree-based algorithms often performed better compared to others. ML algorithms commonly outperformed traditional regression techniques. Sixteen studies used Natural language processing (NLP) of clinical notes for prediction, all studies yielded good results. The overall adherence to reporting quality was poor in the review studies. Only five percent of models were implemented in clinical practice. The most frequently inadequately addressed methodological aspects were: providing model interpretations on the individual patient level, full code availability, performing external validation, calibrating models, and handling class imbalance.

CONCLUSION

This review has identified considerable concerns regarding methodological issues and reporting quality in studies investigating ML to predict hospitalizations. To ensure the acceptability of these models in clinical settings, it is crucial to improve the quality of future studies.

Machine learning prediction of one-year mortality after percutaneous coronary intervention in acute coronary syndrome patients

BACKGROUND

Machine learning (ML) models have the potential to accurately predict outcomes and offer novel insights into inter-variable correlations. In this study, we aimed to design ML models for the prediction of 1-year mortality after percutaneous coronary intervention (PCI) in patients with acute coronary syndrome.

METHODS

This study was performed on 13,682 patients at Tehran Heart Center from 2015 to 2021. Patients were split into 70:30 for testing and training. Four ML models were designed: a traditional Logistic Regression (LR) model, Random Forest (RF), Extreme Gradient Boosting (XGBoost), and Ada Boost models. The importance of features was calculated using the RF feature selector and SHAP based on the XGBoost model. The Area Under the Receiver Operating Characteristic Curve (AUC-ROC) for the prediction on the testing dataset was the main measure of the model's performance.

RESULTS

From a total of 9,073 patients with >1-year follow-up, 340 participants died. Higher age and higher rates of comorbidities were observed in these patients. Body mass index and lipid profile demonstrated a U-shaped correlation with the outcome. Among the models, RF had the best discrimination (AUC 0.866), while the highest sensitivity (80.9%) and specificity (88.3%) were for LR and XGBoost models, respectively. All models had AUCs of >0.8.

CONCLUSION

ML models can predict 1-year mortality after PCI with high performance. A classic LR statistical approach showed comparable results with other ML models. The individual-level assessment of inter-variable correlations provided new insights into the non-linear contribution of risk factors to post-PCI mortality.

Performance Drift in Machine Learning Models for Cardiac Surgery Risk Prediction: Retrospective Analysis

Background

The Society of Thoracic Surgeons and European System for Cardiac Operative Risk Evaluation (EuroSCORE) II risk scores are the most commonly used risk prediction models for in-hospital mortality after adult cardiac surgery. However, they are prone to miscalibration over time and poor generalization across data sets; thus, their use remains controversial. Despite increased interest, a gap in understanding the effect of data set drift on the performance of machine learning (ML) over time remains a barrier to its wider use in clinical practice. Data set drift occurs when an ML system underperforms because of a mismatch between the data it was developed from and the data on which it is deployed.

Objective

In this study, we analyzed the extent of performance drift using models built on a large UK cardiac surgery database. The objectives were to (1) rank and assess the extent of performance drift in cardiac surgery risk ML models over time and (2) investigate any potential influence of data set drift and variable importance drift on performance drift.

Methods

We conducted a retrospective analysis of prospectively, routinely gathered data on adult patients undergoing cardiac surgery in the United Kingdom between 2012 and 2019. We temporally split the data 70:30 into a training and validation set and a holdout set. Five novel ML mortality prediction models were developed and assessed, along with EuroSCORE II, for relationships between and within variable importance drift, performance drift, and actual data set drift. Performance was assessed using a consensus metric.

Results

A total of 227,087 adults underwent cardiac surgery during the study period, with a mortality rate of 2.76% (n=6258). There was strong evidence of a decrease in overall performance across all models (P<.0001). Extreme gradient boosting (clinical effectiveness metric [CEM] 0.728, 95% CI 0.728-0.729) and random forest (CEM 0.727, 95% CI 0.727-0.728) were the overall best-performing models, both temporally and nontemporally. EuroSCORE II performed the worst across all comparisons. Sharp changes in variable importance and data set drift from October to December 2017, from June to July 2018, and from December 2018 to February 2019 mirrored the effects of performance decrease across models.

Conclusions

All models show a decrease in at least 3 of the 5 individual metrics. CEM and variable importance drift detection demonstrate the limitation of logistic regression methods used for cardiac surgery risk prediction and the effects of data set drift. Future work will be required to determine the interplay between ML models and whether ensemble models could improve on their respective performance advantages.

Machine learning improves prediction of postoperative outcomes after gastrointestinal surgery: a systematic review and meta-analysis

BACKGROUND

Machine learning (ML) approaches have become increasingly popular in predicting surgical outcomes. However, it is unknown whether they are superior to traditional statistical methods such as logistic regression (LR). This study aimed to perform a systematic review and meta-analysis to compare the performance of ML vs LR models in predicting postoperative outcomes for patients undergoing gastrointestinal (GI) surgery.

METHODS

A systematic search of Embase, MEDLINE, Cochrane, Web of Science, and Google Scholar was performed through December 2022. The primary outcome was the discriminatory performance of ML vs LR models as measured by the area under the receiver operating characteristic curve (AUC). A meta-analysis was then performed using a random effects model.

RESULTS

A total of 62 LR models and 143 ML models were included across 38 studies. On average, the best-performing ML models had a significantly higher AUC than the LR models (ΔAUC, 0.07; 95% CI, 0.04-0.09; P < .001). Similarly, on average, the best-performing ML models had a significantly higher logit (AUC) than the LR models (Δlogit [AUC], 0.41; 95% CI, 0.23-0.58; P < .001). Approximately half of studies (44%) were found to have a low risk of bias. Upon a subset analysis of only low-risk studies, the difference in logit (AUC) remained significant (ML vs LR, Δlogit [AUC], 0.40; 95% CI, 0.14-0.66; P = .009).

CONCLUSION

We found a significant improvement in discriminatory ability when using ML over LR algorithms in predicting postoperative outcomes for patients undergoing GI surgery. Subsequent efforts should establish standardized protocols for both developing and reporting studies using ML models and explore the practical implementation of these models.

Development and validation of algorithms to predict left ventricular ejection fraction class from healthcare claims data

AIMS

The use of large medical or healthcare claims databases is very useful for population-based studies on the burden of heart failure (HF). Clinical characteristics and management of HF patients differ according to categories of left ventricular ejection fraction (LVEF), but this information is often missing in such databases. We aimed to develop and validate algorithms to identify LVEF in healthcare databases where the information is lacking.

METHODS AND RESULTS

Algorithms were built by machine learning with a random forest approach. Algorithms were trained and reinforced using the French national claims database [Système National des Données de Santé (SNDS)] and a French HF registry. Variables were age, gender, and comorbidities, which could be identified by medico-administrative code-based proxies, Anatomical Therapeutic Chemical codes for drug delivery, International Classification of Diseases (Tenth Revision) coding for hospitalizations, and administrative codes for any other type of reimbursed care. The algorithms were validated by cross-validation and against a subset of the SNDS that includes LVEF information. The areas under the receiver operating characteristic curve were 0.84 for the algorithm identifying LVEF ≤ 40% and 0.79 for the algorithms identifying LVEF < 50% and ≥50%. For LVEF ≤ 40%, the reinforced algorithm identified 50% of patients in the validation dataset with a positive predictive value of 0.88 and a specificity of 0.96. The most important predictive variables were delivery of HF medication, sex, age, hospitalization, and testing for natriuretic peptides with different orders of positive or negative importance according to the LVEF category.

CONCLUSIONS

The algorithms identify reduced or preserved LVEF in HF patients within a nationwide healthcare claims database with high positive predictive value and low rates of false positives.

The State of Artificial Intelligence in Pediatric Surgery: A Systematic Review

BACKGROUND

Artificial intelligence (AI) has been recently shown to improve clinical workflows and outcomes - yet its potential in pediatric surgery remains largely unexplored. This systematic review details the use of AI in pediatric surgery.

METHODS

Nine medical databases were searched from inception until January 2023, identifying articles focused on AI in pediatric surgery. Two authors reviewed full texts of eligible articles. Studies were included if they were original investigations on the development, validation, or clinical application of AI models for pediatric health conditions primarily managed surgically. Studies were excluded if they were not peer-reviewed, were review articles, editorials, commentaries, or case reports, did not focus on pediatric surgical conditions, or did not employ at least one AI model. Extracted data included study characteristics, clinical specialty, AI method and algorithm type, AI model (algorithm) role and performance metrics, key results, interpretability, validation, and risk of bias using PROBAST and QUADAS-2.

RESULTS

Authors screened 8178 articles and included 112. Half of the studies (50%) reported predictive models (for adverse events [25%], surgical outcomes [16%] and survival [9%]), followed by diagnostic (29%) and decision support models (21%). Neural networks (44%) and ensemble learners (36%) were the most commonly used AI methods across application domains. The main pediatric surgical subspecialties represented across all models were general surgery (31%) and neurosurgery (25%). Forty-four percent of models were interpretable, and 6% were both interpretable and externally validated. Forty percent of models had a high risk of bias, and concerns over applicability were identified in 7%.

CONCLUSIONS

While AI has wide potential clinical applications in pediatric surgery, very few published AI algorithms were externally validated, interpretable, and unbiased. Future research needs to focus on developing AI models which are prospectively validated and ultimately integrated into clinical workflows.

LEVEL OF EVIDENCE

2A.

A Machine Learning Algorithm using Clinical and Demographic Data for All-Cause Preterm Birth Prediction

OBJECTIVE

 Preterm birth remains the predominant cause of perinatal mortality throughout the United States and the world, with well-documented racial and socioeconomic disparities. To develop and validate a predictive algorithm for all-cause preterm birth using clinical, demographic, and laboratory data using machine learning.

STUDY DESIGN

 We performed a cohort study of pregnant individuals delivering at a single institution using prospectively collected information on clinical conditions, patient demographics, laboratory data, and health care utilization. Our primary outcome was all-cause preterm birth before 37 weeks. The dataset was randomly divided into a derivation cohort (70%) and a separate validation cohort (30%). Predictor variables were selected amongst 33 that had been previously identified in the literature (directed machine learning). In the derivation cohort, both statistical (logistic regression) and machine learning (XG-Boost) models were used to derive the best fit (C-Statistic) and then validated using the validation cohort. We measured model discrimination with the C-Statistic and assessed the model performance and calibration of the model to determine whether the model provided clinical decision-making benefits.

RESULTS

 The cohort includes a total of 12,440 deliveries among 12,071 individuals. Preterm birth occurred in 2,037 births (16.4%). The derivation cohort consisted of 8,708 (70%) and the validation cohort consisted of 3,732 (30%). XG-Boost was chosen due to the robustness of the model and the ability to deal with missing data and collinearity between predictor variables. The top five predictor variables identified as drivers of preterm birth, by feature importance metric, were multiple gestation, number of emergency department visits in the year prior to the index pregnancy, initial unknown body mass index, gravidity, and prior preterm delivery. Test performance characteristics were similar between the two populations (derivation cohort area under the curve [AUC] = 0.70 vs. validation cohort AUC = 0.63).

CONCLUSION

 Clinical, demographic, and laboratory information can be useful to predict all-cause preterm birth with moderate precision.

KEY POINTS

· Machine learning can be used to create models to predict preterm birth.. · In our model, all-cause preterm birth can be predicted with moderate precision.. · Clinical, demographic, and laboratory information can be useful to predict all-cause preterm birth..

A systematic review of prediction models on arteriovenous fistula: Risk scores and machine learning approaches

OBJECTIVE

Failure-to-mature and early stenosis remains the Achille's heel of hemodialysis arteriovenous fistula (AVF) creation. The maturation and patency of an AVF can be influenced by a variety of demographic, comorbidity, and anatomical factors. This study aims to review the prediction models of AVF maturation and patency with various risk scores and machine learning models.

DATA SOURCES AND REVIEW METHODS

Literature search was performed on PubMed, Scopus, and Embase to identify eligible articles. The quality of the studies was assessed using the Prediction model Risk Of Bias ASsessment (PROBAST) Tool. The performance (discrimination and calibration) of the included studies were extracted.

RESULTS

Fourteen studies (seven studies used risk score approaches; seven studies used machine learning approaches) were included in the review. Among them, 12 studies were rated as high or unclear "risk of bias." Six studies were rated as high concern or unclear for "applicability." C-statistics (Model discrimination metric) was reported in five studies using risk score approach (0.70-0.886) and three utilized machine learning methods (0.80-0.85). Model calibration was reported in three studies. Failure-to-mature risk score developed by one of the studies has been externally validated in three different patient populations, however the model discrimination degraded significantly (C-statistics: 0.519-0.53).

CONCLUSION

The performance of existing predictive models for AVF maturation/patency is underreported. They showed satisfactory performance in their own study population. However, there was high risk of bias in methodology used to build some of the models. The reviewed models also lack external validation or had reduced performance in external cohort.

Application of machine learning for lung cancer survival prognostication-A systematic review and meta-analysis

Introduction

Machine learning (ML) techniques have gained increasing attention in the field of healthcare, including predicting outcomes in patients with lung cancer. ML has the potential to enhance prognostication in lung cancer patients and improve clinical decision-making. In this systematic review and meta-analysis, we aimed to evaluate the performance of ML models compared to logistic regression (LR) models in predicting overall survival in patients with lung cancer.

Methods

We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement. A comprehensive search was conducted in Medline, Embase, and Cochrane databases using a predefined search query. Two independent reviewers screened abstracts and conflicts were resolved by a third reviewer. Inclusion and exclusion criteria were applied to select eligible studies. Risk of bias assessment was performed using predefined criteria. Data extraction was conducted using the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies (CHARMS) checklist. Meta-analytic analysis was performed to compare the discriminative ability of ML and LR models.

Results

The literature search resulted in 3,635 studies, and 12 studies with a total of 211,068 patients were included in the analysis. Six studies reported confidence intervals and were included in the meta-analysis. The performance of ML models varied across studies, with C-statistics ranging from 0.60 to 0.85. The pooled analysis showed that ML models had higher discriminative ability compared to LR models, with a weighted average C-statistic of 0.78 for ML models compared to 0.70 for LR models.

Conclusion

Machine learning models show promise in predicting overall survival in patients with lung cancer, with superior discriminative ability compared to logistic regression models. However, further validation and standardization of ML models are needed before their widespread implementation in clinical practice. Future research should focus on addressing the limitations of the current literature, such as potential bias and heterogeneity among studies, to improve the accuracy and generalizability of ML models for predicting outcomes in patients with lung cancer. Further research and development of ML models in this field may lead to improved patient outcomes and personalized treatment strategies.

Research Concepts and Opportunities for Early-Career Investigators in Cardiac Surgery

Basic, translational or clinic, research is a key component of cardiac surgery. Understanding basic cellular and molecular mechanisms is key to improving patient outcomes, and cardiac surgical procedures must be compared with nonsurgical alternatives. However, guidance for early-career investigators interested in cardiac surgery research is limited. This opinion piece aims at providing basic guidance and principles based on the authors' experience.

Comparative analysis of machine learning vs. traditional modeling approaches for predicting in-hospital mortality after cardiac surgery: temporal and spatial external validation based on a nationwide cardiac surgery registry

AIMS

Preoperative risk assessment is crucial for cardiac surgery. Although previous studies suggested machine learning (ML) may improve in-hospital mortality predictions after cardiac surgery compared to traditional modeling approaches, the validity is doubted due to lacking external validation, limited sample sizes, and inadequate modeling considerations. We aimed to assess predictive performance between ML and traditional modelling approaches, while addressing these major limitations.

METHODS AND RESULTS

Adult cardiac surgery cases (n = 168 565) between 2013 and 2018 in the Chinese Cardiac Surgery Registry were used to develop, validate, and compare various ML vs. logistic regression (LR) models. The dataset was split for temporal (2013-2017 for training, 2018 for testing) and spatial (geographically-stratified random selection of 83 centers for training, 22 for testing) experiments, respectively. Model performances were evaluated in testing sets for discrimination and calibration. The overall in-hospital mortality was 1.9%. In the temporal testing set (n = 32 184), the best-performing ML model demonstrated a similar area under the receiver operating characteristic curve (AUC) of 0.797 (95% CI 0.779-0.815) to the LR model (AUC 0.791 [95% CI 0.775-0.808]; P = 0.12). In the spatial experiment (n = 28 323), the best ML model showed a statistically better but modest performance improvement (AUC 0.732 [95% CI 0.710-0.754]) than LR (AUC 0.713 [95% CI 0.691-0.737]; P = 0.002). Varying feature selection methods had relatively smaller effects on ML models. Most ML and LR models were significantly miscalibrated.

CONCLUSION

ML provided only marginal improvements over traditional modelling approaches in predicting cardiac surgery mortality with routine preoperative variables, which calls for more judicious use of ML in practice.

Machine-learning-based prediction of survival and mitral regurgitation recurrence in patients undergoing mitral valve repair

OBJECTIVES

This study was conducted to assess long-term clinical outcomes after mitral valve repair using machine-learning techniques.

METHODS

We retrospectively evaluated 436 consecutive patients (mean age: 54.7 ± 15.4; 235 males) who underwent mitral valve repair between January 2000 and December 2017. Actuarial survival and freedom from significant (≥ moderate) mitral regurgitation (MR) were clinical end points. To evaluate the independent risk factors, random survival forest (RSF), extreme gradient boost (XGBoost), support vector machine, Cox proportional hazards model and general linear models with elastic net regularization were used. Concordance indices (C-indices) of each model were estimated.

RESULTS

The operative mortality was 0.9% (N = 4). Reoperation was required in 15 patients (3.5%). In terms of C-index, the overall performance of the XGBoost (C-index 0.806) and RSF models (C-index 0.814) was better than that of the Cox model (C-index 0.733) in overall survival. For the recurrent MR, the C-index for XGBoost was 0.718, which was the highest among the 5 models. Compared to the Cox model (C-index 0.545), the C-indices of the XGBoost (C-index 0.718) and RSF models (C-index 0.692) were higher.

CONCLUSIONS

Machine-learning techniques can be a useful tool for both prediction and interpretation in the survival and recurrent MR. From the machine-learning techniques examined here, the long-term clinical outcomes of mitral valve repair were excellent. The complexity of MV increased the risk of late mitral valve-related reoperation.

Paying attention to cardiac surgical risk: An interpretable machine learning approach using an uncertainty-aware attentive neural network

Machine learning (ML) is increasingly applied to predict adverse postoperative outcomes in cardiac surgery. Commonly used ML models fail to translate to clinical practice due to absent model explainability, limited uncertainty quantification, and no flexibility to missing data. We aimed to develop and benchmark a novel ML approach, the uncertainty-aware attention network (UAN), to overcome these common limitations. Two Bayesian uncertainty quantification methods were tested, generalized variational inference (GVI) or a posterior network (PN). The UAN models were compared with an ensemble of XGBoost models and a Bayesian logistic regression model (LR) with imputation. The derivation datasets consisted of 153,932 surgery events from the Australian and New Zealand Society of Cardiac and Thoracic Surgeons (ANZSCTS) Cardiac Surgery Database. An external validation consisted of 7343 surgery events which were extracted from the Medical Information Mart for Intensive Care (MIMIC) III critical care dataset. The highest performing model on the external validation dataset was a UAN-GVI with an area under the receiver operating characteristic curve (AUC) of 0.78 (0.01). Model performance improved on high confidence samples with an AUC of 0.81 (0.01). Confidence calibration for aleatoric uncertainty was excellent for all models. Calibration for epistemic uncertainty was more variable, with an ensemble of XGBoost models performing the best with an AUC of 0.84 (0.08). Epistemic uncertainty was improved using the PN approach, compared to GVI. UAN is able to use an interpretable and flexible deep learning approach to provide estimates of model uncertainty alongside state-of-the-art predictions. The model has been made freely available as an easy-to-use web application demonstrating that by designing uncertainty-aware models with innately explainable predictions deep learning may become more suitable for routine clinical use.

A machine learning approach to predicting 30-day mortality following paediatric cardiac surgery: findings from the Australia New Zealand Congenital Outcomes Registry for Surgery (ANZCORS)

OBJECTIVES

We aim to develop the first risk prediction model for 30-day mortality for the Australian and New Zealand patient populations and examine whether machine learning (ML) algorithms outperform traditional statistical approaches.

METHODS

Data from the Australia New Zealand Congenital Outcomes Registry for Surgery, which contains information on every paediatric cardiac surgical encounter in Australian and New Zealand for patients aged <18 years between January 2013 and December 2021, were analysed (n = 14 343). The outcome was mortality within the 30-day period following a surgical encounter, with ∼30% of the observations randomly selected to be used for validation of the final model. Three different ML methods were used, all of which employed five-fold cross-validation to prevent overfitting, with model performance judged primarily by the area under the receiver operating curve (AUC).

RESULTS

Among the 14 343 30-day periods, there were 188 deaths (1.3%). In the validation data, the gradient-boosted tree obtained the best performance [AUC = 0.87, 95% confidence interval = (0.82, 0.92); calibration = 0.97, 95% confidence interval = (0.72, 1.27)], outperforming penalized logistic regression and artificial neural networks (AUC of 0.82 and 0.81, respectively). The strongest predictors of mortality in the gradient boosting trees were patient weight, STAT score, age and gender.

CONCLUSIONS

Our risk prediction model outperformed logistic regression and achieved a level of discrimination comparable to the PRAiS2 and Society of Thoracic Surgery Congenital Heart Surgery Database mortality risk models (both which obtained AUC = 0.86). Non-linear ML methods can be used to construct accurate clinical risk prediction tools.

Tree-based survival analysis improves mortality prediction in cardiac surgery

Objectives

Machine learning (ML) classification tools are known to accurately predict many cardiac surgical outcomes. A novel approach, ML-based survival analysis, remains unstudied for predicting mortality after cardiac surgery. We aimed to benchmark performance, as measured by the concordance index (C-index), of tree-based survival models against Cox proportional hazards (CPH) modeling and explore risk factors using the best-performing model.

Methods

144,536 patients with 147,301 surgery events from the Australian and New Zealand Society of Cardiac and Thoracic Surgeons (ANZSCTS) national database were used to train and validate models. Univariate analysis was performed using Student's T-test for continuous variables, Chi-squared test for categorical variables, and stratified Kaplan-Meier estimation of the survival function. Three ML models were tested, a decision tree (DT), random forest (RF), and gradient boosting machine (GBM). Hyperparameter tuning was performed using a Bayesian search strategy. Performance was assessed using 2-fold cross-validation repeated 5 times.

Results

The highest performing model was the GBM with a C-index of 0.803 (0.002), followed by RF with 0.791 (0.003), DT with 0.729 (0.014), and finally CPH with 0.596 (0.042). The 5 most predictive features were age, type of procedure, length of hospital stay, drain output in the first 4 h (ml), and inotrope use greater than 4 h postoperatively.

Conclusion

Tree-based learning for survival analysis is a non-parametric and performant alternative to CPH modeling. GBMs offer interpretable modeling of non-linear relationships, promising to expose the most relevant risk factors and uncover new questions to guide future research.

Comparing Decentralized Learning Methods for Health Data Models to Nondecentralized Alternatives: Protocol for a Systematic Review

BACKGROUND

Considering the soaring health-related costs directed toward a growing, aging, and comorbid population, the health sector needs effective data-driven interventions while managing rising care costs. While health interventions using data mining have become more robust and adopted, they often demand high-quality big data. However, growing privacy concerns have hindered large-scale data sharing. In parallel, recently introduced legal instruments require complex implementations, especially when it comes to biomedical data. New privacy-preserving technologies, such as decentralized learning, make it possible to create health models without mobilizing data sets by using distributed computation principles. Several multinational partnerships, including a recent agreement between the United States and the European Union, are adopting these techniques for next-generation data science. While these approaches are promising, there is no clear and robust evidence synthesis of health care applications.

OBJECTIVE

The main aim is to compare the performance among health data models (eg, automated diagnosis and mortality prediction) developed using decentralized learning approaches (eg, federated and blockchain) to those using centralized or local methods. Secondary aims are comparing the privacy compromise and resource use among model architectures.

METHODS

We will conduct a systematic review using the first-ever registered research protocol for this topic following a robust search methodology, including several biomedical and computational databases. This work will compare health data models differing in development architecture, grouping them according to their clinical applications. For reporting purposes, a PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 flow diagram will be presented. CHARMS (Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies)-based forms will be used for data extraction and to assess the risk of bias, alongside PROBAST (Prediction Model Risk of Bias Assessment Tool). All effect measures in the original studies will be reported.

RESULTS

The queries and data extractions are expected to start on February 28, 2023, and end by July 31, 2023. The research protocol was registered with PROSPERO, under the number 393126, on February 3, 2023. With this protocol, we detail how we will conduct the systematic review. With that study, we aim to summarize the progress and findings from state-of-the-art decentralized learning models in health care in comparison to their local and centralized counterparts. Results are expected to clarify the consensuses and heterogeneities reported and help guide the research and development of new robust and sustainable applications to address the health data privacy problem, with applicability in real-world settings.

CONCLUSIONS

We expect to clearly present the status quo of these privacy-preserving technologies in health care. With this robust synthesis of the currently available scientific evidence, the review will inform health technology assessment and evidence-based decisions, from health professionals, data scientists, and policy makers alike. Importantly, it should also guide the development and application of new tools in service of patients' privacy and future research.

TRIAL REGISTRATION

PROSPERO 393126; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=393126.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/45823.

Comparison of machine learning techniques in prediction of mortality following cardiac surgery: analysis of over 220 000 patients from a large national database

OBJECTIVES

To perform a systematic comparison of in-hospital mortality risk prediction post-cardiac surgery, between the predominant scoring system-European System for Cardiac Operative Risk Evaluation (EuroSCORE) II, logistic regression (LR) retrained on the same variables and alternative machine learning techniques (ML)-random forest (RF), neural networks (NN), XGBoost and weighted support vector machine.

METHODS

Retrospective analyses of prospectively routinely collected data on adult patients undergoing cardiac surgery in the UK from January 2012 to March 2019. Data were temporally split 70:30 into training and validation subsets. Mortality prediction models were created using the 18 variables of EuroSCORE II. Comparisons of discrimination, calibration and clinical utility were then conducted. Changes in model performance, variable-importance over time and hospital/operation-based model performance were also reviewed.

RESULTS

Of the 227 087 adults who underwent cardiac surgery during the study period, there were 6258 deaths (2.76%). In the testing cohort, there was an improvement in discrimination [XGBoost (95% confidence interval (CI) area under the receiver operator curve (AUC), 0.834-0.834, F1 score, 0.276-0.280) and RF (95% CI AUC, 0.833-0.834, F1, 0.277-0.281)] compared with EuroSCORE II (95% CI AUC, 0.817-0.818, F1, 0.243-0.245). There was no significant improvement in calibration with ML and retrained-LR compared to EuroSCORE II. However, EuroSCORE II overestimated risk across all deciles of risk and over time. The calibration drift was lowest in NN, XGBoost and RF compared with EuroSCORE II. Decision curve analysis showed XGBoost and RF to have greater net benefit than EuroSCORE II.

CONCLUSIONS

ML techniques showed some statistical improvements over retrained-LR and EuroSCORE II. The clinical impact of this improvement is modest at present. However the incorporation of additional risk factors in future studies may improve upon these findings and warrants further study.

Machine Learning and Artificial Intelligence in Surgical Research

Machine learning, a subtype of artificial intelligence, is an emerging field of surgical research dedicated to predictive modeling. From its inception, machine learning has been of interest in medical and surgical research. Built on traditional research metrics for optimal success, avenues of research include diagnostics, prognosis, operative timing, and surgical education, in a variety of surgical subspecialties. Machine learning represents an exciting and developing future in the world of surgical research that will not only allow for more personalized and comprehensive medical care.

Machine learning-based prediction of 1-year mortality in hypertensive patients undergoing coronary revascularization surgery

BACKGROUND

Machine learning (ML) has shown promising results in all fields of medicine, including preventive cardiology. Hypertensive patients are at higher risk of mortality after coronary artery bypass graft (CABG) surgery; thus, we aimed to design and evaluate five ML models to predict 1-year mortality among hypertensive patients who underwent CABG.

HYOTHESIS

ML algorithms can significantly improve mortality prediction after CABG.

METHODS

Tehran Heart Center's CABG data registry was used to extract several baseline and peri-procedural characteristics and mortality data. The best features were chosen using random forest (RF) feature selection algorithm. Five ML models were developed to predict 1-year mortality: logistic regression (LR), RF, artificial neural network (ANN), extreme gradient boosting (XGB), and naïve Bayes (NB). The area under the curve (AUC), sensitivity, and specificity were used to evaluate the models.

RESULTS

Among the 8,493 hypertensive patients who underwent CABG (mean age of 68.27 ± 9.27 years), 303 died in the first year. Eleven features were selected as the best predictors, among which total ventilation hours and ejection fraction were the leading ones. LR showed the best prediction ability with an AUC of 0.82, while the least AUC was for the NB model (0.79). Among the subgroups, the highest AUC for LR model was for two age range groups (50-59 and 80-89 years), overweight, diabetic, and smoker subgroups of hypertensive patients.

CONCLUSIONS

All ML models had excellent performance in predicting 1-year mortality among CABG hypertension patients, while LR was the best regarding AUC. These models can help clinicians assess the risk of mortality in specific subgroups at higher risk (such as hypertensive ones).

Machine learning to predict adverse outcomes after cardiac surgery: A systematic review and meta-analysis

BACKGROUND

Machine learning (ML) models are promising tools for predicting adverse postoperative outcomes in cardiac surgery, yet have not translated to routine clinical use. We conducted a systematic review and meta-analysis to assess the predictive performance of ML approaches.

METHODS

We conducted an electronic search to find studies assessing ML and traditional statistical models to predict postoperative outcomes. Our primary outcome was the concordance (C-) index of discriminative performance. Using a Bayesian meta-analytic approach we pooled the C-indices with the 95% credible interval (CrI) across multiple outcomes comparing ML methods to logistic regression (LR) and clinical scoring tools. Additionally, we performed critical difference and sensitivity analysis.

RESULTS

We identified 2792 references from the search of which 51 met inclusion criteria. Two postoperative outcomes were amenable for meta-analysis: 30-day mortality and in-hospital mortality. For 30-day mortality, the pooled C-index and 95% CrI were 0.82 (0.79-0.85), 0.80 (0.77-0.84), 0.78 (0.74-0.82) for ML models, LR, and scoring tools respectively. For in-hospital mortality, the pooled C-index was 0.81 (0.78-0.84) and 0.79 (0.73-0.84) for ML models and LR, respectively. There were no statistically significant results indicating ML superiority over LR.

CONCLUSION

In cardiac surgery patients, for the prediction of mortality, current ML methods do not have greater discriminative power over LR as measured by the C-index.

Prediction of Postoperative Deterioration in Cardiac Surgery Patients Using Electronic Health Record and Physiologic Waveform Data

BACKGROUND

Postoperative hemodynamic deterioration among cardiac surgical patients can indicate or lead to adverse outcomes. Whereas prediction models for such events using electronic health records or physiologic waveform data are previously described, their combined value remains incompletely defined. The authors hypothesized that models incorporating electronic health record and processed waveform signal data (electrocardiogram lead II, pulse plethysmography, arterial catheter tracing) would yield improved performance versus either modality alone.

METHODS

Intensive care unit data were reviewed after elective adult cardiac surgical procedures at an academic center between 2013 and 2020. Model features included electronic health record features and physiologic waveforms. Tensor decomposition was used for waveform feature reduction. Machine learning-based prediction models included a 2013 to 2017 training set and a 2017 to 2020 temporal holdout test set. The primary outcome was a postoperative deterioration event, defined as a composite of low cardiac index of less than 2.0 ml min-1 m-2, mean arterial pressure of less than 55 mmHg sustained for 120 min or longer, new or escalated inotrope/vasopressor infusion, epinephrine bolus of 1 mg or more, or intensive care unit mortality. Prediction models analyzed data 8 h before events.

RESULTS

Among 1,555 cases, 185 (12%) experienced 276 deterioration events, most commonly including low cardiac index (7.0% of patients), new inotrope (1.9%), and sustained hypotension (1.4%). The best performing model on the 2013 to 2017 training set yielded a C-statistic of 0.803 (95% CI, 0.799 to 0.807), although performance was substantially lower in the 2017 to 2020 test set (0.709, 0.705 to 0.712). Test set performance of the combined model was greater than corresponding models limited to solely electronic health record features (0.641; 95% CI, 0.637 to 0.646) or waveform features (0.697; 95% CI, 0.693 to 0.701).

CONCLUSIONS

Clinical deterioration prediction models combining electronic health record data and waveform data were superior to either modality alone, and performance of combined models was primarily driven by waveform data. Decreased performance of prediction models during temporal validation may be explained by data set shift, a core challenge of healthcare prediction modeling.

EDITOR’S PERSPECTIVE

Machine Learning to Predict Outcomes and Cost by Phase of Care After Coronary Artery Bypass Grafting

BACKGROUND

Machine learning may enhance prediction of outcomes after coronary artery bypass grafting (CABG). We sought to develop and validate a dynamic machine learning model to predict CABG outcomes at clinically relevant pre- and postoperative time points.

METHODS

The Society of Thoracic Surgeons (STS) registry data elements from 2086 isolated CABG patients were divided into training and testing datasets and input into Extreme Gradient Boosting decision-tree machine learning algorithms. Two prediction models were developed based on data from preoperative (80 parameters) and postoperative (125 parameters) phases of care. Outcomes included operative mortality, major morbidity or mortality, high cost, and 30-day readmission. Machine learning and STS model performance were assessed using accuracy and the area under the precision-recall curve (AUC-PR).

RESULTS

Preoperative machine learning models predicted mortality (accuracy, 98%; AUC-PR = 0.16; F1 = 0.24), major morbidity or mortality (accuracy, 75%; AUC-PR = 0.33; F1 = 0.42), high cost (accuracy, 83%; AUC-PR = 0.51; F1 = 0.52), and 30-day readmission (accuracy, 70%; AUC-PR = 0.47; F1 = 0.49) with high accuracy. Preoperative machine learning models performed similarly to the STS for prediction of mortality (STS AUC-PR = 0.11; P = .409) and outperformed STS for prediction of mortality or major morbidity (STS AUC-PR = 0.28; P < .001). Addition of intraoperative parameters further improved machine learning model performance for major morbidity or mortality (AUC-PR = 0.39; P < .01) and high cost (AUC-PR = 0.64; P < .01), with cross-clamp and bypass times emerging as important additive predictive parameters.

CONCLUSIONS

Machine learning can predict mortality, major morbidity, high cost, and readmission after isolated CABG. Prediction based on the phase of care allows for dynamic risk assessment through the hospital course, which may benefit quality assessment and clinical decision-making.

Machine learning algorithms to predict major bleeding after isolated coronary artery bypass grafting

Objectives

Postoperative major bleeding is a common problem in patients undergoing cardiac surgery and is associated with poor outcomes. We evaluated the performance of machine learning (ML) methods to predict postoperative major bleeding.

Methods

A total of 1,045 patients who underwent isolated coronary artery bypass graft surgery (CABG) were enrolled. Their datasets were assigned randomly to training (70%) or a testing set (30%). The primary outcome was major bleeding defined as the universal definition of perioperative bleeding (UDPB) classes 3–4. We constructed a reference logistic regression (LR) model using known predictors. We also developed several modern ML algorithms. In the test set, we compared the area under the receiver operating characteristic curves (AUCs) of these ML algorithms with the reference LR model results, and the TRUST and WILL-BLEED risk score. Calibration analysis was undertaken using the calibration belt method.

Results

The prevalence of postoperative major bleeding was 7.1% (74/1,045). For major bleeds, the conditional inference random forest (CIRF) model showed the highest AUC [0.831 (0.732–0.930)], and the stochastic gradient boosting (SGBT) and random forest models demonstrated the next best results [0.820 (0.742–0.899) and 0.810 (0.719–0.902)]. The AUCs of all ML models were higher than [0.629 (0.517–0.641) and 0.557 (0.449–0.665)], as achieved by TRUST and WILL-BLEED, respectively.

Conclusion

ML methods successfully predicted major bleeding after cardiac surgery, with greater performance compared with previous scoring models. Modern ML models may enhance the identification of high-risk major bleeding subpopulations.

Machine Learning Methods for Predicting Long-Term Mortality in Patients After Cardiac Surgery

Objective:

This study aims to construct and validate several machine learning (ML) algorithms to predict long-term mortality and identify risk factors in unselected patients post-cardiac surgery.

Methods

The Medical Information Mart for Intensive Care (MIMIC-III) database was used to perform a retrospective administrative database study. Candidate predictors consisted of the demographics, comorbidity, vital signs, laboratory test results, scoring systems, and treatment information on the first day of ICU admission. Four-year mortality was set as the study outcome. We used the ML methods of logistic regression (LR), artificial neural network (NNET), naïve bayes (NB), gradient boosting machine (GBM), adapting boosting (Ada), random forest (RF), bagged trees (BT), and eXtreme Gradient Boosting (XGB). The prognostic capacity and clinical utility of these ML models were compared using the area under the receiver operating characteristic curves (AUC), calibration curves, and decision curve analysis (DCA).

Results

Of 7,368 patients in MIMIC-III included in the final cohort, a total of 1,337 (18.15%) patients died during a 4-year follow-up. Among 65 variables extracted from the database, a total of 25 predictors were selected using recursive feature elimination and included in the subsequent analysis. The Ada model performed best among eight models in both discriminatory ability with the highest AUC of 0.801 and goodness of fit (visualized by calibration curve). Moreover, the DCA shows that the net benefit of the RF, Ada, and BT models surpassed that of other ML models for almost all threshold probability values. Additionally, through the Ada technique, we determined that red blood cell distribution width (RDW), blood urea nitrogen (BUN), SAPS II, anion gap (AG), age, urine output, chloride, creatinine, congestive heart failure, and SOFA were the Top 10 predictors in the feature importance rankings.

Conclusions

The Ada model performs best in predicting 4-year mortality after cardiac surgery among the eight ML models, which might have significant application in the development of early warning systems for patients following operations.

Predicting Major Adverse Cardiovascular Events in Acute Coronary Syndrome: A Scoping Review of Machine Learning Approaches

BACKGROUND

Acute coronary syndrome is the topmost cause of death worldwide; therefore, it is necessary to predict major adverse cardiovascular events and cardiovascular deaths in patients with acute coronary syndrome to make correct and timely clinical decisions.

OBJECTIVE

The current review aimed to highlight algorithms and important predictor variables through examining those studies which used machine learning algorithms for predicting major adverse cardiovascular events in patients with acute coronary syndrome.

METHODS

To predict major adverse cardiovascular events in patients with acute coronary syndrome, the preferred reporting items for scoping reviews guidelines were used. In doing so, PubMed, Embase, Web of Science, Scopus, Springer, and IEEE Xplore databases were searched for articles published between 2005 and 2021. The checklist "Quality assessment of machine learning studies" was used to assess the quality of eligible studies. The findings of the studies are presented in the form of a narrative synthesis of evidence.

RESULTS

In total, among 2,558 retrieved articles, 22 studies were qualified for analysis. Major adverse cardiovascular events and mortality were predicted in 5 and 17 studies, respectively. According to the results, 14 (63.64%) studies did not perform external validation and only used registry data. The algorithms used in this study comprised, inter alia, Regression Logistic, Random Forest, Boosting Ensemble, Non-Boosting Ensemble, Decision Trees, and Naive Bayes. Multiple studies (N = 20) achieved a high area under the ROC curve between 0.8 and 0.99 in predicting mortality and major adverse cardiovascular events. The predictor variables used in these studies were divided into demographic, clinical, and therapeutic features. However, no study reported the integration of machine learning model into clinical practice.

CONCLUSION

Machine learning algorithms rendered acceptable results to predict major adverse cardiovascular events and mortality outcomes in patients with acute coronary syndrome. However, these approaches have never been integrated into clinical practice. Further research is required to develop feasible and effective machine learning prediction models to measure their potentially important implications for optimizing the quality of care in patients with acute coronary syndrome.

Comparing Machine Learning Models and Statistical Models for Predicting Heart Failure Events: A Systematic Review and Meta-Analysis

Objective

To compare the performance, clinical feasibility, and reliability of statistical and machine learning (ML) models in predicting heart failure (HF) events.

Background

Although ML models have been proposed to revolutionize medicine, their promise in predicting HF events has not been investigated in detail.

Methods

A systematic search was performed on Medline, Web of Science, and IEEE Xplore for studies published between January 1, 2011 to July 14, 2021 that developed or validated at least one statistical or ML model that could predict all-cause mortality or all-cause readmission of HF patients. Prediction Model Risk of Bias Assessment Tool was used to assess the risk of bias, and random effect model was used to evaluate the pooled c-statistics of included models.

Result

Two-hundred and two statistical model studies and 78 ML model studies were included from the retrieved papers. The pooled c-index of statistical models in predicting all-cause mortality, ML models in predicting all-cause mortality, statistical models in predicting all-cause readmission, ML models in predicting all-cause readmission were 0.733 (95% confidence interval 0.724–0.742), 0.777 (0.752–0.803), 0.678 (0.651–0.706), and 0.660 (0.633–0.686), respectively, indicating that ML models did not show consistent superiority compared to statistical models. The head-to-head comparison revealed similar results. Meanwhile, the immoderate use of predictors limited the feasibility of ML models. The risk of bias analysis indicated that ML models' technical pitfalls were more serious than statistical models'. Furthermore, the efficacy of ML models among different HF subgroups is still unclear.

Conclusions

ML models did not achieve a significant advantage in predicting events, and their clinical feasibility and reliability were worse.

Machine Learning for the Prediction of Complications in Patients After Mitral Valve Surgery

Background: This study intended to use a machine learning model to identify critical preoperative and intraoperative variables and predict the risk of several severe complications (myocardial infarction, stroke, renal failure, and hospital mortality) after cardiac valvular surgery.

Study Design and Methods: A total of 1,488 patients undergoing cardiac valvular surgery in eight large tertiary hospitals in China were examined. Fifty-four perioperative variables, such as essential demographic characteristics, concomitant disease, preoperative laboratory indicators, operation type, and intraoperative information, were collected. Machine learning models were developed and validated by 10-fold cross-validation. In each fold, Recursive Feature Elimination was used to select key variables. Ten machine learning models and logistic regression were developed. The area under the receiver operating characteristic (AUROC), accuracy (ACC), Youden index, sensitivity, specificity, F1-score, positive predictive value (PPV), and negative predictive value (NPV) were used to compare the prediction performance of different models. The SHapley Additive ex Planations package was applied to interpret the best machine learning model. Finally, a model was trained on the whole dataset with the merged key variables, and a web tool was created for clinicians to use.

Results: In this study, 14 vital variables, namely, intraoperative total input, intraoperative blood loss, intraoperative colloid bolus, Classification of New York Heart Association (NYHA) heart function, preoperative hemoglobin (Hb), preoperative platelet (PLT), age, preoperative fibrinogen (FIB), intraoperative minimum red blood cell volume (Hct), body mass index (BMI), creatinine, preoperative Hct, intraoperative minimum Hb, and intraoperative autologous blood, were finally selected. The eXtreme Gradient Boosting algorithms (XGBOOST) algorithm model presented a significantly better predictive performance (AUROC: 0.90) than the other models (ACC: 81%, Youden index: 70%, sensitivity: 89%, specificity: 81%, F1-score:0.26, PPV: 15%, and NPV: 99%).

Conclusion: A model for predicting several severe complications after cardiac valvular surgery was successfully developed using a machine learning algorithm based on 14 perioperative variables, which could guide clinical physicians to take appropriate preventive measures and diminish the complications for patients at high risk.

Electrocardiography-Based Artificial Intelligence Algorithm Aids in Prediction of Long-term Mortality After Cardiac Surgery

OBJECTIVE

To assess whether an electrocardiography-based artificial intelligence (AI) algorithm developed to detect severe ventricular dysfunction (left ventricular ejection fraction [LVEF] of 35% or below) independently predicts long-term mortality after cardiac surgery among patients without severe ventricular dysfunction (LVEF>35%).

METHODS

Patients who underwent valve or coronary bypass surgery at Mayo Clinic (1993-2019) and had documented LVEF above 35% on baseline electrocardiography were included. We compared patients with an abnormal vs a normal AI-enhanced electrocardiogram (AI-ECG) screen for LVEF of 35% or below on preoperative electrocardiography. The primary end point was all-cause mortality.

RESULTS

A total of 20,627 patients were included, of whom 17,125 (83.0%) had a normal AI-ECG screen and 3502 (17.0%) had an abnormal AI-ECG screen. Patients with an abnormal AI-ECG screen were older and had more comorbidities. Probability of survival at 5 and 10 years was 86.2% and 68.2% in patients with a normal AI-ECG screen vs 71.4% and 45.1% in those with an abnormal screen (log-rank, P<.01). In the multivariate Cox survival analysis, the abnormal AI-ECG screen was independently associated with a higher all-cause mortality overall (hazard ratio [HR], 1.31; 95% CI, 1.24 to 1.37) and in subgroups of isolated valve surgery (HR, 1.30; 95% CI, 1.18 to 1.42), isolated coronary artery bypass grafting (HR, 1.29; 95% CI, 1.20 to 1.39), and combined coronary artery bypass grafting and valve surgery (HR, 1.19; 95% CI, 1.08 to 1.32). In a subgroup analysis, the association between abnormal AI-ECG screen and mortality was consistent in patients with LVEF of 35% to 55% and among those with LVEF above 55%.

CONCLUSION

A novel electrocardiography-based AI algorithm that predicts severe ventricular dysfunction can predict long-term mortality among patients with LVEF above 35% undergoing valve and/or coronary bypass surgery.

Clinical impact and quality of randomized controlled trials involving interventions evaluating artificial intelligence prediction tools: a systematic review

The evidence of the impact of traditional statistical (TS) and artificial intelligence (AI) tool interventions in clinical practice was limited. This study aimed to investigate the clinical impact and quality of randomized controlled trials (RCTs) involving interventions evaluating TS, machine learning (ML), and deep learning (DL) prediction tools. A systematic review on PubMed was conducted to identify RCTs involving TS/ML/DL tool interventions in the past decade. A total of 65 RCTs from 26,082 records were included. A majority of them had model development studies and generally good performance was achieved. The function of TS and ML tools in the RCTs mainly included assistive treatment decisions, assistive diagnosis, and risk stratification, but DL trials were only conducted for assistive diagnosis. Nearly two-fifths of the trial interventions showed no clinical benefit compared to standard care. Though DL and ML interventions achieved higher rates of positive results than TS in the RCTs, in trials with low risk of bias (17/65) the advantage of DL to TS was reduced while the advantage of ML to TS disappeared. The current applications of DL were not yet fully spread performed in medicine. It is predictable that DL will integrate more complex clinical problems than ML and TS tools in the future. Therefore, rigorous studies are required before the clinical application of these tools.