Machine Learning to Predict Outcomes in Patients with Acute Gastrointestinal Bleeding: A Systematic Review

A meta-analysis of the diagnostic test accuracy of artificial intelligence predicting emergency department dispositions

BACKGROUND

The rapid advancement of Artificial Intelligence (AI) has led to its widespread application across various domains, showing encouraging outcomes. Many studies have utilized AI to forecast emergency department (ED) disposition, aiming to forecast patient outcomes earlier and to allocate resources better; however, a dearth of comprehensive review literature exists to assess the objective performance standards of these predictive models using quantitative evaluations. This study aims to conduct a meta-analysis to assess the diagnostic accuracy of AI in predicting ED disposition, encompassing admission, critical care, and mortality.

METHODS

Multiple databases, including Scopus, Springer, ScienceDirect, PubMed, Wiley, Sage, and Google Scholar, were searched until December 31, 2023, to gather relevant literature. Risk of bias was assessed using the Prediction Model Risk of Bias Assessment Tool. Pooled estimates of sensitivity, specificity, and area under the receiver operating characteristic curve (AUROC) were calculated to evaluate AI's predictive performance. Sub-group analyses were performed to explore covariates affecting AI predictive model performance.

RESULTS

The study included 88 articles possessed with 117 AI models, among which 39, 45, and 33 models predicted admission, critical care, and mortality, respectively. The reported statistics for sensitivity, specificity, and AUROC represent pooled summary measures derived from the component studies included in this meta-analysis. AI's summary sensitivity, specificity, and AUROC for predicting admission were 0.81 (95% Confidence Interval [CI] 0.74-0.86), 0.87 (95% CI 0.81-0.91), and 0.87 (95% CI 0.84-0.93), respectively. For critical care, the values were 0.86 (95% CI 0.79-0.91), 0.89 (95% CI 0.83-0.93), and 0.93 (95% CI 0.89-0.95), respectively, and for mortality, they were 0.85 (95% CI 0.80-0.89), 0.94 (95% CI 0.90-0.96), and 0.93 (95% CI 0.89-0.96), respectively. Emergent sample characteristics and AI techniques showed evidence of significant covariates influencing the heterogeneity of AI predictive models for ED disposition.

CONCLUSIONS

The meta-analysis indicates promising performance of AI in predicting ED disposition, with certain potential for improvement, especially in sensitivity. Future research could explore advanced AI techniques such as ensemble learning and cross-validation with hyper-parameter tuning to enhance predictive model efficacy.

TRIAL REGISTRATION

This systematic review was not registered with PROSPERO or any other similar registry because the review was completed prior to the opportunity for registration, and PROSPERO currently does not accept registrations for reviews that are already completed. We are committed to transparency and have adhered to best practices in systematic review methodology throughout this study.

Machine Learning for Predicting Stillbirth: A Systematic Review

Stillbirth is a major global issue, with over 5 million cases each year. The multifactorial nature of stillbirth makes it difficult to predict. Artificial intelligence (AI) and machine learning (ML) have the potential to enhance clinical decision-making and enable precise assessments. This study reviewed the literature on predictive ML models for stillbirth highlighting input characteristics, performance metrics, and validation. The PubMed, Cochrane, and Web of Science databases were searched for studies using AI to develop predictive models for stillbirth. Findings were analyzed qualitatively using narrative synthesis and graphics. Risk of bias and the applicability of the studies were assessed using PROBAST. Model design and performance were discussed. Eight studies involving 14,840,654 women with gestational ages ranging from 20 weeks to full term were included in the qualitative analysis. Most studies used neural networks, random forests, and logistic regression algorithms. The number of predictive features varied from 14 to 53. Only 50% of studies validated the models. Cross-validation was commonly employed, and only 25% of studies performed external validation. All studies reported area under the curve as a performance metric (range 0.54-0.9), and five studies reported sensitivity (range, 60- 90%) and specificity (range, 64 - 93.3%). A stacked ensemble model that analyzed 53 features performed better than other models (AUC = 0.9; sensitivity and specificity > 85%). Available ML models can attain a considerable degree of accuracy for prediction of stillbirth; however, these models require further development before they can be applied in a clinical setting.

Development and validation of a machine learning model to predict hemostatic intervention in patients with acute upper gastrointestinal bleeding

BACKGROUND

Acute upper gastrointestinal bleeding (UGIB) is common in clinical practice and has a wide range of severity. Along with medical therapy, endoscopic intervention is the mainstay treatment for hemostasis in high-risk rebleeding lesions. Predicting the need for endoscopic intervention would be beneficial in resource-limited areas for selective referral to an endoscopic center. The proposed risk stratification scores had limited accuracy. We developed a machine learning model to predict the need for endoscopic intervention in patients with acute UGIB.

METHODS

A prospectively collected database of UGIB patients from 2011 to 2020 was retrospectively reviewed. Patients older than 18 years diagnosed with UGIB who underwent endoscopy were included. Data comprised demographic characteristics, clinical presentation, and laboratory parameters. The cleaned data was used for model development and validation in Python. We conducted 80%-20% split sample training and test sets. The training set was used for supervised learning of 15 models using a stratified 5-fold cross-validation process. The model with the highest AUROC was then internally validated with the test set to evaluate performance.

RESULTS

Of 1389 patients, 615 (44.3%) of the cohorts received the endoscopic intervention (293 variceal- and 336 nonvariceal-bleeding interventions). Eighteen features, including demographic characteristics, clinical presentation, and laboratory parameters, were selected as input for 15 machine learning models. The result revealed that the linear discriminant analysis model could achieve the highest AUROC of 0.74 to predict endoscopic intervention. The model was validated with the test set, in which the AUROC was increased from 0.74 to 0.81. Finally, the model was deployed as a web application by Streamlit.

CONCLUSIONS

Our machine learning model can identify patients with acute UGIB who need endoscopic intervention with good performance. This may help primary care physicians prioritize patients who need referrals and optimize resource allocation in resource-limited areas. Further development and identification of more specific features might improve prediction performance.

TRIAL REGISTRATION

None (Retrospective cohort study) PATIENT & PUBLIC INVOLVEMENT: None.

New machine-learning models outperform conventional risk assessment tools in Gastrointestinal bleeding

Rapid and accurate identification of high-risk acute gastrointestinal bleeding (GIB) patients is essential. We developed two machine-learning (ML) models to calculate the risk of in-hospital mortality in patients admitted due to overt GIB. We analyzed the prospective, multicenter Hungarian GIB Registry's data. The predictive performance of XGBoost and CatBoost machine-learning algorithms with the Glasgow-Blatchford (GBS), pre-endoscopic Rockall and ABC scores were compared. We evaluated our models using five-fold cross-validation, and performance was measured by area under receiver operating characteristic curve (AUC) analysis with 95% confidence intervals (CI). Overall, we included 1,021 patients in the analysis. In-hospital death occurred in 108 cases. The XGBoost and the CatBoost model identified patients who died with an AUC of 0.84 (CI:0.76-0.90; 0.77-0.90; respectively) in the internal validation set, whereas the GBS and pre-endoscopic Rockall clinical scoring system's performance was significantly lower, AUC values of 0.68 (CI:0.62-0.74) and 0.62 (CI:0.56-0.67), respectively. ABC score had an AUC of 0.77 (CI:0.71-0.83). The XGBoost model had a specificity of 0.96 (CI:0.92-0.98) at a sensitivity of 0.25 (CI:0.10-0.43) compared with the CatBoost model, which had a specificity of 0.74 (CI:0.66-0.83) at a sensitivity of 0.78 (CI:0.57-0.95). XGBoost and the CatBoost models evaluate the mortality risk of acute GI bleeding better, than the conventional risk assessment tools.

Major Bleeding in the Emergency Department: A Practical Guide for Optimal Management

Major bleeding is a life-threatening condition with high morbidity and mortality. Trauma, gastrointestinal bleeding, haemoptysis, intracranial haemorrhage or other causes of bleeding represent major concerns in the Emergency Department (ED), especially when complicated by haemodynamic instability. Severity and source of bleeding, comorbidities, and prior use of anticoagulants are pivotal factors affecting both the clinical status and the patients' differential response to haemorrhage. Thus, risk stratification is fundamental in the initial assessment of patients with bleeding. Aggressive resuscitation is the principal step for achieving haemodynamic stabilization of the patient, which will further allow appropriate interventions to be made for the definite control of bleeding. Overall management of major bleeding in the ED should follow a holistic individualized approach which includes haemodynamic stabilization, repletion of volume and blood loss, and reversal of coagulopathy and identification of the source of bleeding. The aim of the present practical guide is to provide an update on recent epidemiological data about the most common etiologies of bleeding and summarize the latest evidence regarding the bundles of care for the management of patients with major bleeding of traumatic or non-traumatic etiology in the ED.

Hemoglobin decrease predicts untoward outcomes better than severity of anemia

Patients with gastrointestinal bleeding (GIB) exhibit varying tolerances to acute blood loss. We aimed to investigate the effect of relative Hb decrease (ΔHb%) on GIB outcomes. Participants enrolled in the Hungarian GIB Registry between 2019 and 2022 were analyzed. The primary outcome, defined as a composite endpoint, included in-hospital bleeding-related mortality and the need for urgent intervention. Four groups were created based on the lowest Hb measured during hospitalization (nadirHb), along with four subgroups categorized by ΔHb%. Regardless of the nadirHb, participants with higher ΔHb% had a higher probability of reaching the composite endpoint. A 30-40% ΔHb% decrease to a nadirHb of 80-90 g/L resulted in a similar likelihood of reaching the primary endpoint as a 0-10% ΔHb% to 70-80 g/L or 60-70 g/L, respectively (10% vs. 12%, p = 1.00; 10% vs. 10%, p = 1.00). Our results showed that a higher Hb decrease in GIB is associated with an increased untoward outcome rate even when the lowest hemoglobin exceeds the recommended transfusion thresholds. New randomized controlled trials investigating transfusion thresholds should consider ΔHb% as a potential key variable and risk factor.

Validation of an Electronic Health Record-Based Machine Learning Model Compared With Clinical Risk Scores for Gastrointestinal Bleeding

BACKGROUND & AIMS

Guidelines recommend use of risk stratification scores for patients presenting with gastrointestinal bleeding (GIB) to identify very-low-risk patients eligible for discharge from emergency departments. Machine learning models may outperform existing scores and can be integrated within the electronic health record (EHR) to provide real-time risk assessment without manual data entry. We present the first EHR-based machine learning model for GIB.

METHODS

The training cohort comprised 2546 patients and internal validation of 850 patients presenting with overt GIB (ie, hematemesis, melena, and hematochezia) to emergency departments of 2 hospitals from 2014 to 2019. External validation was performed on 926 patients presenting to a different hospital with the same EHR from 2014 to 2019. The primary outcome was a composite of red blood cell transfusion, hemostatic intervention (ie, endoscopic, interventional radiologic, or surgical), and 30-day all-cause mortality. We used structured data fields in the EHR, available within 4 hours of presentation, and compared the performance of machine learning models with current guideline-recommended risk scores, Glasgow-Blatchford Score, and Oakland Score. Primary analysis was area under the receiver operating characteristic curve. Secondary analysis was specificity at 99% sensitivity to assess the proportion of patients correctly identified as very low risk.

RESULTS

The machine learning model outperformed the Glasgow-Blatchford Score (area under the receiver operating characteristic curve, 0.92 vs 0.89; P < .001) and Oakland Score (area under the receiver operating characteristic curve, 0.92 vs 0.89; P < .001). At the very-low-risk threshold of 99% sensitivity, the machine learning model identified more very-low-risk patients: 37.9% vs 18.5% for Glasgow-Blatchford Score and 11.7% for Oakland Score (P < .001 for both comparisons).

CONCLUSIONS

An EHR-based machine learning model performs better than currently recommended clinical risk scores and identifies more very-low-risk patients eligible for discharge from the emergency department.

A systematic review and meta-analysis of artificial intelligence versus clinicians for skin cancer diagnosis

Scientific research of artificial intelligence (AI) in dermatology has increased exponentially. The objective of this study was to perform a systematic review and meta-analysis to evaluate the performance of AI algorithms for skin cancer classification in comparison to clinicians with different levels of expertise. Based on PRISMA guidelines, 3 electronic databases (PubMed, Embase, and Cochrane Library) were screened for relevant articles up to August 2022. The quality of the studies was assessed using QUADAS-2. A meta-analysis of sensitivity and specificity was performed for the accuracy of AI and clinicians. Fifty-three studies were included in the systematic review, and 19 met the inclusion criteria for the meta-analysis. Considering all studies and all subgroups of clinicians, we found a sensitivity (Sn) and specificity (Sp) of 87.0% and 77.1% for AI algorithms, respectively, and a Sn of 79.78% and Sp of 73.6% for all clinicians (overall); differences were statistically significant for both Sn and Sp. The difference between AI performance (Sn 92.5%, Sp 66.5%) vs. generalists (Sn 64.6%, Sp 72.8%), was greater, when compared with expert clinicians. Performance between AI algorithms (Sn 86.3%, Sp 78.4%) vs expert dermatologists (Sn 84.2%, Sp 74.4%) was clinically comparable. Limitations of AI algorithms in clinical practice should be considered, and future studies should focus on real-world settings, and towards AI-assistance.

Diagnostic Performance of Artificial Intelligence in Detection of Primary Malignant Bone Tumors: a Meta-Analysis

We aim to conduct a meta-analysis on studies that evaluated the diagnostic performance of artificial intelligence (AI) algorithms in the detection of primary bone tumors, distinguishing them from other bone lesions, and comparing them with clinician assessment. A systematic search was conducted using a combination of keywords related to bone tumors and AI. After extracting contingency tables from all included studies, we performed a meta-analysis using random-effects model to determine the pooled sensitivity and specificity, accompanied by their respective 95% confidence intervals (CI). Quality assessment was evaluated using a modified version of Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) and Prediction Model Study Risk of Bias Assessment Tool (PROBAST). The pooled sensitivities for AI algorithms and clinicians on internal validation test sets for detecting bone neoplasms were 84% (95% CI: 79.88) and 76% (95% CI: 64.85), and pooled specificities were 86% (95% CI: 81.90) and 64% (95% CI: 55.72), respectively. At external validation, the pooled sensitivity and specificity for AI algorithms were 84% (95% CI: 75.90) and 91% (95% CI: 83.96), respectively. The same numbers for clinicians were 85% (95% CI: 73.92) and 94% (95% CI: 89.97), respectively. The sensitivity and specificity for clinicians with AI assistance were 95% (95% CI: 86.98) and 57% (95% CI: 48.66). Caution is needed when interpreting findings due to potential limitations. Further research is needed to bridge this gap in scientific understanding and promote effective implementation for medical practice advancement.

Radiomics in Musculoskeletal Tumors

Sarcomas are heterogeneous rare tumors predominantly affecting the musculoskeletal (MSK) system. Due to significant variations in their natural history and variable response to conventional treatments, the discovery of novel diagnostic and prognostic biomarkers to guide therapeutic decision-making is an active and ongoing field of research. As new cellular, molecular, and metabolic biomarkers continue to be discovered, quantitative radiologic imaging is becoming increasingly important in sarcoma management. Radiomics offers the potential for discovering novel imaging diagnostic and predictive biomarkers using standard-of-care medical imaging. In this review, we detail the core concepts of radiomics and the application of radiomics to date in MSK sarcoma research. Also described are specific challenges related to radiomic studies, as well as viewpoints on clinical adoption and future perspectives in the field.

Development and validation of machine learning models to predict the need for haemostatic therapy in acute upper gastrointestinal bleeding

Background

Acute upper gastrointestinal bleeding (AUGIB) is a major cause of morbidity and mortality. This presentation however is not universally high risk as only 20-30% of bleeds require urgent haemostatic therapy. Nevertheless, the current standard of care is for all patients admitted to an inpatient bed to undergo endoscopy within 24 h for risk stratification which is invasive, costly and difficult to achieve in routine clinical practice.

Objectives

To develop novel non-endoscopic machine learning models for AUGIB to predict the need for haemostatic therapy by endoscopic, radiological or surgical intervention.

Design

A retrospective cohort study.

Method

We analysed data from patients admitted with AUGIB to hospitals from 2015 to 2020 (n = 970). Machine learning models were internally validated to predict the need for haemostatic therapy. The performance of the models was compared to the Glasgow-Blatchford score (GBS) using the area under receiver operating characteristic (AUROC) curves.

Results

The random forest classifier [AUROC 0.84 (0.80-0.87)] had the best performance and was superior to the GBS [AUROC 0.75 (0.72-0.78), p < 0.001] in predicting the need for haemostatic therapy in patients with AUGIB. A GBS cut-off of ⩾12 was associated with an accuracy, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of 0.74, 0.49, 0.81, 0.41 and 0.85, respectively. The Random Forrest model performed better with an accuracy, sensitivity, specificity, PPV and NPV of 0.82, 0.54, 0.90, 0.60 and 0.88, respectively.

Conclusion

We developed and validated a machine learning algorithm with high accuracy and specificity in predicting the need for haemostatic therapy in AUGIB. This could be used to risk stratify high-risk patients to urgent endoscopy.

Scoring systems for risk stratification in upper and lower gastrointestinal bleeding

Several scoring systems have been developed for both upper and lower GI bleeding to predict the bleeding severity and discriminate between low-risk patients, who may be suitable for outpatient management, and those who would likely need hospital-based interventions and are at high risk for adverse outcomes. Risk scores created to identify low-risk patients (namely the Glasgow Blatchford Score and the Oakland score) showed very good discriminative performances and their implementation has proven to be effective in reducing hospital admissions and healthcare burden. Conversely, the performances of risk scores in identifying specific adverse events to define high-risk patients are less accurate, and whether their integration into routine clinical practice has a tangible impact on patient management remains unproven. This review describes the existing risk score systems for GI bleeding, emphasizes key research findings, elucidates the circumstances in which their utilization can be beneficial, examines their constraints when considering routine clinical application, and discuss future development.

Performance of advanced machine learning algorithms overlogistic regression in predicting hospital readmissions: A meta-analysis

Objectives

Machine learning algorithms are being increasingly used for predicting hospital readmissions. This meta-analysis evaluated the performance of logistic regression (LR) and machine learning (ML) models for the prediction of 30-day hospital readmission among patients in the US.

Methods

Electronic databases (i.e., Medline, PubMed, and Embase) were searched from January 2015 to December 2019. Only studies in the English language were included. Two reviewers performed studies screening, quality appraisal, and data collection. The quality of the studies was assessed using the Quality in Prognosis Studies (QUIPS) tool. Model performance was evaluated using the Area Under the Curve (AUC). A random-effects meta-analysis was performed using STATA 16.

Results

Nine studies were included based on the selection criteria. The most common ML techniques were tree-based methods such as boosting and random forest. Most of the studies had a low risk of bias (8/9). The AUC was greater with ML to predict 30-day all-cause hospital readmission compared with LR [Mean Difference (MD): 0.03; 95% Confidence Interval (CI) 0.01-0.05]. Subgroup analyses found that deep-learning methods had a better performance compared with LR (MD 0.06; 95% CI, 0.04-0.09), followed by neural networks (MD: 0.03; 95% CI, 0.03-0.03), while the AUCs of the tree-based (MD: 0.02; 95% CI -0.00-0.04) and kernel-based (MD: 0.02; 95% CI 0.02 (-0.13-0.16) methods were no different compared to LR. More than half of the studies evaluated heart failure-related rehospitalization (N = 5). For the readmission prediction among heart failure patients, ML performed better compared with LR, with a mean difference in AUC of 0.04 (95% CI, 0.01-0.07). The leave-one-out sensitivity analysis confirmed the robustness of the findings.

Conclusion

Multiple ML methods were used to predict 30-day all-cause hospital readmission. Performance varied across the ML methods, with deep-learning methods showing the best performance over the LR.

Machine learning to predict major bleeding during anticoagulation for venous thromboembolism: possibilities and limitations

Predictive tools for major bleeding (MB) using machine learning (ML) might be advantageous over traditional methods. We used data from the Registro Informatizado de Enfermedad TromboEmbólica (RIETE) to develop ML algorithms to identify patients with venous thromboembolism (VTE) at increased risk of MB during the first 3 months of anticoagulation. A total of 55 baseline variables were used as predictors. New data prospectively collected from the RIETE were used for further validation. The RIETE and VTE-BLEED scores were used for comparisons. External validation was performed with the COMMAND-VTE database. Learning was carried out with data from 49 587 patients, of whom 873 (1.8%) had MB. The best performing ML method was XGBoost. In the prospective validation cohort the sensitivity, specificity, positive predictive value and F1 score were: 33.2%, 93%, 10%, and 15.4% respectively. F1 value for the RIETE and VTE-BLEED scores were 8.6% and 6.4% respectively. In the external validation cohort the metrics were 10.3%, 87.6%, 3.5% and 5.2% respectively. In that cohort, the F1 value for the RIETE score was 17.3% and for the VTE-BLEED score 9.75%. The performance of the XGBoost algorithm was better than that from the RIETE and VTE-BLEED scores only in the prospective validation cohort, but not in the external validation cohort.

Prediction of acute organophosphate poisoning severity using machine learning techniques

Poisoning with organophosphate compounds is a significant public health risk, especially in developing countries. Considering the importance of early and accurate prediction of organophosphate poisoning prognosis, the aim of this study was to develop a machine learning-based prediction model to predict the severity of organophosphate poisoning. The data of patients with organophosphate poisoning were retrospectively extracted and split into training and test sets in a ratio of 70:30. The feature selection was done by least absolute shrinkage and selection operator method. Selected features were fed into five machine learning techniques, including Histogram Boosting Gradient, eXtreme Gradient Boosting, K-Nearest Neighborhood, Support Vector Machine (SVM) (kernel = linear), and Random Forest. The Scikit-learn library in Python programming language was used to implement the models. Finally, the performance of developed models was measured using ten-fold cross-validation methods and some evaluation criteria with 95 % confidence intervals. A total of 1237 patients were used to train and test the machine learning models. According to the criteria determining severe organophosphate poisoning, 732 patients were assigned to group 1 (patients with mild to moderate poisoning) and 505 patients were assigned to group 2 (patients with severe poisoning). With an AUC value of 0.907 (95 % CI 0.89-0.92), the model developed using XGBoost outperformed other models. Feature importance evaluation found that venous blood gas-pH, white blood cells, and plasma cholinesterase activity were the top three variables that contribute the most to the prediction performance of the prognosis in patients with organophosphate poisoning. XGBoost model yield an accuracy of 90.1 % (95 % CI 0.891-0.918), specificity of 91.4 % (95 % CI 0.90-0.92), a sensitivity of 89.5 % (95 % CI 0.87-0.91), F-measure of 91.2 % (95 % CI 0.90-0.921), and Kappa statistic of 91.2 % (95 % CI 0.90-0.92). The machine learning-based prediction models can accurately predict the severity of organophosphate poisoning. Based on feature selection techniques, the most important predictors of organophosphate poisoning were VBG-pH, white blood cell count, plasma cholinesterase activity, VBG-BE, and age. The best algorithm with the highest predictive performance was the XGBoost classifier.

Predicting mortality in patients with nonvariceal upper gastrointestinal bleeding using machine-learning

BACKGROUND

Non-endoscopic risk scores, Glasgow Blatchford (GBS) and admission Rockall (Rock), are limited by poor specificity. The aim of this study was to develop an Artificial Neural Network (ANN) for the non-endoscopic triage of nonvariceal upper gastrointestinal bleeding (NVUGIB), with mortality as a primary outcome.

METHODS

Four machine learning algorithms, namely, Linear Discriminant Analysis (LDA), Quadratic Discriminant Analysis (QDA), logistic regression (LR), K-Nearest Neighbor (K-NN), were performed with GBS, Rock, Beylor Bleeding score (BBS), AIM65, and T-score.

RESULTS

A total of 1,096 NVUGIB hospitalized in the Gastroenterology Department of the County Clinical Emergency Hospital of Craiova, Romania, randomly divided into training and testing groups, were included retrospectively in our study. The machine learning models were more accurate at identifying patients who met the endpoint of mortality than any of the existing risk scores. AIM65 was the most important score in the detection of whether a NVUGIB would die or not, whereas BBS had no influence on this. Also, the greater AIM65 and GBS, and the lower Rock and T-score, the higher mortality will be.

CONCLUSION

The best accuracy was obtained by the hyperparameter-tuned K-NN classifier (98%), giving the highest precision and recall on the training and testing datasets among all developed models, showing that machine learning can accurately predict mortality in patients with NVUGIB.

Machine learning and discriminant function analysis in the formulation of generic models for sex prediction using patella measurements

Sex prediction from bone measurements that display sexual dimorphism is one of the most important aspects of forensic anthropology. Some bones like the skull and pelvis display distinct morphological traits that are based on shape. These morphological traits which are sexually dimorphic across different population groups have been shown to provide an acceptably high degree of accuracy in the prediction of sex. A sample of 100 patella of Mixed Ancestry South Africans (MASA) was collected from the Dart collection. Six parameters: maximum height (maxh), maximum breadth (maxw), maximum thickness (maxt), the height of articular facet (haf), lateral articular facet breadth (lafb), and medial articular facet breath (mafb) were used in this study. Stepwise and direct discriminant function analyses were performed for measurements that exhibited significant differences between male and female mean measurements, and the "leave-one-out" approach was used for validation. Moreover, we have used eight classical machine learning techniques along with feature ranking techniques to identify the best feature combinations for sex prediction. A stacking machine learning technique was trained and validated to classify the sex of the subject. Here, we have used the top performing three ML classifiers as base learners and the predictions of these models were used as inputs to different machine learning classifiers as meta learners to make the final decision. The measurements of the patella of South Africans are sexually dimorphic and this observation is consistent with previous studies on the patella of different countries. The range of average accuracies obtained for pooled multivariate discriminant function equations is 81.9-84.2%, while the stacking ML technique provides 90.8% accuracy which compares well with those presented for previous studies in other parts of the world. In conclusion, the models proposed in this study from measurements of the patella of different population groups in South Africa are useful resent with reasonably high average accuracies.

Clinical applications of machine learning in predicting 3D shapes of the human body: a systematic review

BACKGROUND

Predicting morphological changes to anatomical structures from 3D shapes such as blood vessels or appearance of the face is a growing interest to clinicians. Machine learning (ML) has had great success driving predictions in 2D, however, methods suitable for 3D shapes are unclear and the use cases unknown.

OBJECTIVE AND METHODS

This systematic review aims to identify the clinical implementation of 3D shape prediction and ML workflows. Ovid-MEDLINE, Embase, Scopus and Web of Science were searched until 28th March 2022.

RESULTS

13,754 articles were identified, with 12 studies meeting final inclusion criteria. These studies involved prediction of the face, head, aorta, forearm, and breast, with most aiming to visualize shape changes after surgical interventions. ML algorithms identified were regressions (67%), artificial neural networks (25%), and principal component analysis (8%). Meta-analysis was not feasible due to the heterogeneity of the outcomes.

CONCLUSION

3D shape prediction is a nascent but growing area of research in medicine. This review revealed the feasibility of predicting 3D shapes using ML clinically, which could play an important role for clinician-patient visualization and communication. However, all studies were early phase and there were inconsistent language and reporting. Future work could develop guidelines for publication and promote open sharing of source code.

Artificial intelligence and its impact on the domains of universal health coverage, health emergencies and health promotion: An overview of systematic reviews

BACKGROUND

Artificial intelligence is fueling a new revolution in medicine and in the healthcare sector. Despite the growing evidence on the benefits of artificial intelligence there are several aspects that limit the measure of its impact in people's health. It is necessary to assess the current status on the application of AI towards the improvement of people's health in the domains defined by WHO's Thirteenth General Programme of Work (GPW13) and the European Programme of Work (EPW), to inform about trends, gaps, opportunities, and challenges.

OBJECTIVE

To perform a systematic overview of systematic reviews on the application of artificial intelligence in the people's health domains as defined in the GPW13 and provide a comprehensive and updated map on the application specialties of artificial intelligence in terms of methodologies, algorithms, data sources, outcomes, predictors, performance, and methodological quality.

METHODS

A systematic search in MEDLINE, EMBASE, Cochrane and IEEEXplore was conducted between January 2015 and June 2021 to collect systematic reviews using a combination of keywords related to the domains of universal health coverage, health emergencies protection, and better health and wellbeing as defined by the WHO's PGW13 and EPW. Eligibility criteria was based on methodological quality and the inclusion of practical implementation of artificial intelligence. Records were classified and labeled using ICD-11 categories into the domains of the GPW13. Descriptors related to the area of implementation, type of modeling, data entities, outcomes and implementation on care delivery were extracted using a structured form and methodological aspects of the included reviews studies was assessed using the AMSTAR checklist.

RESULTS

The search strategy resulted in the screening of 815 systematic reviews from which 203 were assessed for eligibility and 129 were included in the review. The most predominant domain for artificial intelligence applications was Universal Health Coverage (N = 98) followed by Health Emergencies (N = 16) and Better Health and Wellbeing (N = 15). Neoplasms area on Universal Health Coverage was the disease area featuring most of the applications (21.7 %, N = 28). The reviews featured analytics primarily over both public and private data sources (67.44 %, N = 87). The most used type of data was medical imaging (31.8 %, N = 41) and predictors based on regions of interest and clinical data. The most prominent subdomain of Artificial Intelligence was Machine Learning (43.4 %, N = 56), in which Support Vector Machine method was predominant (20.9 %, N = 27). Regarding the purpose, the application of Artificial Intelligence I is focused on the prediction of the diseases (36.4 %, N = 47). With respect to the validation, more than a half of the reviews (54.3 %, N = 70) did not report a validation procedure and, whenever available, the main performance indicator was the accuracy (28.7 %, N = 37). According to the methodological quality assessment, a third of the reviews (34.9 %, N = 45) implemented methods for analysis the risk of bias and the overall AMSTAR score below was 5 (4.01 ± 1.93) on all the included systematic reviews.

CONCLUSION

Artificial intelligence is being used for disease modelling, diagnose, classification and prediction in the three domains of GPW13. However, the evidence is often limited to laboratory and the level of adoption is largely unbalanced between ICD-11 categoriesand diseases. Data availability is a determinant factor on the developmental stage of artificial intelligence applications. Most of the reviewed studies show a poor methodological quality and are at high risk of bias, which limits the reproducibility of the results and the reliability of translating these applications to real clinical scenarios. The analyzed papers show results only in laboratory and testing scenarios and not in clinical trials nor case studies, limiting the supporting evidence to transfer artificial intelligence to actual care delivery.

Machine Learning-Based Mortality Prediction of Patients at Risk During Hospital Admission

OBJECTIVES

The ability to predict in-hospital mortality from data available at hospital admission would identify patients at risk and thereby assist hospital-wide patient safety initiatives. Our aim was to use modern machine learning tools to predict in-hospital mortality from standardized data sets available at hospital admission.

METHODS

This was a retrospective, observational study in 3 adult tertiary care hospitals in Western Australia between January 2008 and June 2017. Primary outcome measures were the area under the curve for the receiver operating characteristics curve, the F1 score, and the average precision of the 4 machine learning algorithms used: logistic regression, neural networks, random forests, and gradient boosting trees.

RESULTS

Using our 4 predictive models, in-hospital mortality could be predicted satisfactorily (areas under the curve for neural networks, logistic regression, random forests, and gradient boosting trees: 0.932, 0.936, 0.935, and 0.935, respectively), with moderate F1 scores: 0.378, 0.367, 0.380, and 0.380, respectively. Average precision values were 0.312, 0.321, 0.334, and 0.323, respectively. It remains unknown whether additional features might improve our models; however, this would result in additional efforts for data acquisition in daily clinical practice.

CONCLUSIONS

This study demonstrates that using only a limited, standardized data set in-hospital mortality can be predicted satisfactorily at the time point of hospital admission. More parameters describing patient's health are likely needed to improve our model.

Risk of bias of prognostic models developed using machine learning: a systematic review in oncology

BACKGROUND

Prognostic models are used widely in the oncology domain to guide medical decision-making. Little is known about the risk of bias of prognostic models developed using machine learning and the barriers to their clinical uptake in the oncology domain.

METHODS

We conducted a systematic review and searched MEDLINE and EMBASE databases for oncology-related studies developing a prognostic model using machine learning methods published between 01/01/2019 and 05/09/2019. The primary outcome was risk of bias, judged using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). We described risk of bias overall and for each domain, by development and validation analyses separately.

RESULTS

We included 62 publications (48 development-only; 14 development with validation). 152 models were developed across all publications and 37 models were validated. 84% (95% CI: 77 to 89) of developed models and 51% (95% CI: 35 to 67) of validated models were at overall high risk of bias. Bias introduced in the analysis was the largest contributor to the overall risk of bias judgement for model development and validation. 123 (81%, 95% CI: 73.8 to 86.4) developed models and 19 (51%, 95% CI: 35.1 to 67.3) validated models were at high risk of bias due to their analysis, mostly due to shortcomings in the analysis including insufficient sample size and split-sample internal validation.

CONCLUSIONS

The quality of machine learning based prognostic models in the oncology domain is poor and most models have a high risk of bias, contraindicating their use in clinical practice. Adherence to better standards is urgently needed, with a focus on sample size estimation and analysis methods, to improve the quality of these models.

Methodological conduct of prognostic prediction models developed using machine learning in oncology: a systematic review

BACKGROUND

Describe and evaluate the methodological conduct of prognostic prediction models developed using machine learning methods in oncology.

METHODS

We conducted a systematic review in MEDLINE and Embase between 01/01/2019 and 05/09/2019, for studies developing a prognostic prediction model using machine learning methods in oncology. We used the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement, Prediction model Risk Of Bias ASsessment Tool (PROBAST) and CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) to assess the methodological conduct of included publications. Results were summarised by modelling type: regression-, non-regression-based and ensemble machine learning models.

RESULTS

Sixty-two publications met inclusion criteria developing 152 models across all publications. Forty-two models were regression-based, 71 were non-regression-based and 39 were ensemble models. A median of 647 individuals (IQR: 203 to 4059) and 195 events (IQR: 38 to 1269) were used for model development, and 553 individuals (IQR: 69 to 3069) and 50 events (IQR: 17.5 to 326.5) for model validation. A higher number of events per predictor was used for developing regression-based models (median: 8, IQR: 7.1 to 23.5), compared to alternative machine learning (median: 3.4, IQR: 1.1 to 19.1) and ensemble models (median: 1.7, IQR: 1.1 to 6). Sample size was rarely justified (n = 5/62; 8%). Some or all continuous predictors were categorised before modelling in 24 studies (39%). 46% (n = 24/62) of models reporting predictor selection before modelling used univariable analyses, and common method across all modelling types. Ten out of 24 models for time-to-event outcomes accounted for censoring (42%). A split sample approach was the most popular method for internal validation (n = 25/62, 40%). Calibration was reported in 11 studies. Less than half of models were reported or made available.

CONCLUSIONS

The methodological conduct of machine learning based clinical prediction models is poor. Guidance is urgently needed, with increased awareness and education of minimum prediction modelling standards. Particular focus is needed on sample size estimation, development and validation analysis methods, and ensuring the model is available for independent validation, to improve quality of machine learning based clinical prediction models.

Can minimal clinically important differences in patient reported outcome measures be predicted by machine learning in patients with total knee or hip arthroplasty? A systematic review

OBJECTIVES

To systematically review studies using machine learning (ML) algorithms to predict whether patients undergoing total knee or total hip arthroplasty achieve an improvement as high or higher than the minimal clinically important differences (MCID) in patient reported outcome measures (PROMs) (classification problem).

METHODS

Studies were eligible to be included in the review if they collected PROMs both pre- and postintervention, reported the method of MCID calculation and applied ML. ML was defined as a family of models which automatically learn from data when selecting features, identifying nonlinear relations or interactions. Predictive performance must have been assessed using common metrics. Studies were searched on MEDLINE, PubMed Central, Web of Science Core Collection, Google Scholar and Cochrane Library. Study selection and risk of bias assessment (ROB) was conducted by two independent researchers.

RESULTS

517 studies were eligible for title and abstract screening. After screening title and abstract, 18 studies qualified for full-text screening. Finally, six studies were included. The most commonly applied ML algorithms were random forest and gradient boosting. Overall, eleven different ML algorithms have been applied in all papers. All studies reported at least fair predictive performance, with two reporting excellent performance. Sample size varied widely across studies, with 587 to 34,110 individuals observed. PROMs also varied widely across studies, with sixteen applied to TKA and six applied to THA. There was no single PROM utilized commonly in all studies. All studies calculated MCIDs for PROMs based on anchor-based or distribution-based methods or referred to literature which did so. Five studies reported variable importance for their models. Two studies were at high risk of bias.

DISCUSSION

No ML model was identified to perform best at the problem stated, nor can any PROM said to be best predictable. Reporting standards must be improved to reduce risk of bias and improve comparability to other studies.

Nursing Value Analysis and Risk Assessment of Acute Gastrointestinal Bleeding Using Multiagent Reinforcement Learning Algorithm

Gastrointestinal bleeding (GIB) indicates an issue in the digestive system. Blood can be found in feces or vomiting; however, it is not always visible, even if it makes the stool appear darkish or muddy. The bleeding can range in harshness from light to severe and can be dangerous. It is advised that nursing value analysis and risk assessment of patients with GIB is essential, but existing risk assessment techniques function inconsistently. Machine learning (ML) has the potential to increase risk evaluation. For evaluating risk in patients with GIB, scoring techniques are ineffective; a machine learning method would help. As a result, we present а unique machine learning-based nursing value analysis and risk assessment framework in this research to construct a model to evaluate the risk of hospital-based interventions or mortality in individuals with GIB and make a comparison to that of other rating systems. Initially, the dataset is collected, and preprocessing is done. Feature extraction is done using local binary patterns (LBP). Classification is performed using a fuzzy support vector machine (FSVM) classifier. For risk assessment and nursing value analysis, machine learning-based prediction using a multiagent reinforcement algorithm is employed. For improving the performance of the proposed system, we use spider monkey optimization (SMO) algorithm. The performance metrics like classification accuracy, area under the receiver-operating characteristic curve (AUROC), area under the curve (AUC), sensitivity, specificity, and precision are analyzed and compared with the traditional approaches. In individuals with GIB, the suggested technique had a good-excellent prognostic efficacy, and it outperformed other traditional models.

A New Dawn for the Use of Artificial Intelligence in Gastroenterology, Hepatology and Pancreatology

Artificial intelligence (AI) is rapidly becoming an essential tool in the medical field as well as in daily life. Recent developments in deep learning, a subfield of AI, have brought remarkable advances in image recognition, which facilitates improvement in the early detection of cancer by endoscopy, ultrasonography, and computed tomography. In addition, AI-assisted big data analysis represents a great step forward for precision medicine. This review provides an overview of AI technology, particularly for gastroenterology, hepatology, and pancreatology, to help clinicians utilize AI in the near future.

Current Status and Future Perspective of Artificial Intelligence in the Management of Peptic Ulcer Bleeding: A Review of Recent Literature

With the decreasing incidence of peptic ulcer bleeding (PUB) over the past two decades, the clinician experience of managing patients with PUB has also declined, especially for young endoscopists. A patient with PUB management requires collaborative care involving the emergency department, gastroenterologist, radiologist, and surgeon, from initial assessment to hospital discharge. The application of artificial intelligence (AI) methods has remarkably improved people's lives. In particular, AI systems have shown great potential in many areas of gastroenterology to increase human performance. Colonoscopy polyp detection or diagnosis by an AI system was recently introduced for commercial use to improve endoscopist performance. Although PUB is a longstanding health problem, these newly introduced AI technologies may soon impact endoscopists' clinical practice by improving the quality of care for these patients. To update the current status of AI application in PUB, we reviewed recent relevant literature and provided future perspectives that are required to integrate such AI tools into real-world practice.

A Machine Learning-Based Algorithm for the Prediction of Intensive Care Unit Delirium (PRIDE): Retrospective Study

Background

Delirium frequently occurs among patients admitted to the intensive care unit (ICU). There is limited evidence to support interventions to treat or resolve delirium in patients who have already developed delirium. Therefore, the early recognition and prevention of delirium are important in the management of critically ill patients.

Objective

This study aims to develop and validate a delirium prediction model within 24 hours of admission to the ICU using electronic health record data. The algorithm was named the Prediction of ICU Delirium (PRIDE).

Methods

This is a retrospective cohort study performed at a tertiary referral hospital with 120 ICU beds. We only included patients who were 18 years or older at the time of admission and who stayed in the medical or surgical ICU. Patients were excluded if they lacked a Confusion Assessment Method for the ICU record from the day of ICU admission or if they had a positive Confusion Assessment Method for the ICU record at the time of ICU admission. The algorithm to predict delirium was developed using patient data from the first 2 years of the study period and validated using patient data from the last 6 months. Random forest (RF), Extreme Gradient Boosting (XGBoost), deep neural network (DNN), and logistic regression (LR) were used. The algorithms were externally validated using MIMIC-III data, and the algorithm with the largest area under the receiver operating characteristics (AUROC) curve in the external data set was named the PRIDE algorithm.

Results

A total of 37,543 cases were collected. After patient exclusion, 12,409 remained as our study population, of which 3816 (30.8%) patients experienced delirium incidents during the study period. Based on the exclusion criteria, out of the 96,016 ICU admission cases in the MIMIC-III data set, 2061 cases were included, and 272 (13.2%) delirium incidents occurred. The average AUROCs and 95% CIs for internal validation were 0.916 (95% CI 0.916-0.916) for RF, 0.919 (95% CI 0.919-0.919) for XGBoost, 0.881 (95% CI 0.878-0.884) for DNN, and 0.875 (95% CI 0.875-0.875) for LR. Regarding the external validation, the best AUROC were 0.721 (95% CI 0.72-0.721) for RF, 0.697 (95% CI 0.695-0.699) for XGBoost, 0.655 (95% CI 0.654-0.657) for DNN, and 0.631 (95% CI 0.631-0.631) for LR. The Brier score of the RF model is 0.168, indicating that it is well-calibrated.

Conclusions

A machine learning approach based on electronic health record data can be used to predict delirium within 24 hours of ICU admission. RF, XGBoost, DNN, and LR models were used, and they effectively predicted delirium. However, with the potential to advise ICU physicians and prevent ICU delirium, prospective studies are required to verify the algorithm’s performance.

Role of artificial intelligence in multidisciplinary imaging diagnosis of gastrointestinal diseases

The use of artificial intelligence-based tools is regarded as a promising approach to increase clinical efficiency in diagnostic imaging, improve the interpretability of results, and support decision-making for the detection and prevention of diseases. Radiology, endoscopy and pathology images are suitable for deep-learning analysis, potentially changing the way care is delivered in gastroenterology. The aim of this review is to examine the key aspects of different neural network architectures used for the evaluation of gastrointestinal conditions, by discussing how different models behave in critical tasks, such as lesion detection or characterization (i.e. the distinction between benign and malignant lesions of the esophagus, the stomach and the colon). To this end, we provide an overview on recent achievements and future prospects in deep learning methods applied to the analysis of radiology, endoscopy and histologic whole-slide images of the gastrointestinal tract.

Early identification of patients with acute gastrointestinal bleeding using natural language processing and decision rules

BACKGROUND AND AIM

Guidelines recommend risk stratification scores in patients presenting with gastrointestinal bleeding (GIB), but such scores are uncommonly employed in practice. Automation and deployment of risk stratification scores in real time within electronic health records (EHRs) would overcome a major impediment. This requires an automated mechanism to accurately identify ("phenotype") patients with GIB at the time of presentation. The goal is to identify patients with acute GIB by developing and evaluating EHR-based phenotyping algorithms for emergency department (ED) patients.

METHODS

We specified criteria using structured data elements to create rules for identifying patients and also developed multiple natural language processing (NLP)-based approaches for automated phenotyping of patients, tested them with tenfold cross-validation for 10 iterations (n = 7144) and external validation (n = 2988) and compared them with a standard method to identify patient conditions, the Systematized Nomenclature of Medicine. The gold standard for GIB diagnosis was the independent dual manual review of medical records. The primary outcome was the positive predictive value.

RESULTS

A decision rule using GIB-specific terms from ED triage and ED review-of-systems assessment performed better than the Systematized Nomenclature of Medicine on internal validation and external validation (positive predictive value = 85% confidence interval:83%-87% vs 69% confidence interval:66%-72%; P < 0.001). The syntax-based NLP algorithm and Bidirectional Encoder Representation from Transformers neural network-based NLP algorithm had similar performance to the structured-data fields decision rule.

CONCLUSIONS

An automated decision rule employing GIB-specific triage and review-of-systems terms can be used to trigger EHR-based deployment of risk stratification models to guide clinical decision making in real time for patients with acute GIB presenting to the ED.

Application of machine learning in predicting hospital readmissions: a scoping review of the literature

BACKGROUND

Advances in machine learning (ML) provide great opportunities in the prediction of hospital readmission. This review synthesizes the literature on ML methods and their performance for predicting hospital readmission in the US.

METHODS

This review was performed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Extension for Scoping Reviews (PRISMA-ScR) Statement. The extraction of items was also guided by the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS). Electronic databases PUBMED, MEDLINE, and EMBASE were systematically searched from January 1, 2015, through December 10, 2019. The articles were imported into COVIDENCE online software for title/abstract screening and full-text eligibility. Observational studies using ML techniques for hospital readmissions among US patients were eligible for inclusion. Articles without a full text available in the English language were excluded. A qualitative synthesis included study characteristics, ML algorithms utilized, and model validation, and quantitative analysis assessed model performance. Model performances in terms of Area Under the Curve (AUC) were analyzed using R software. Quality in Prognosis Studies (QUIPS) tool was used to assess the quality of the reviewed studies.

RESULTS

Of 522 citations reviewed, 43 studies met the inclusion criteria. A majority of the studies used electronic health records (24, 56%), followed by population-based data sources (15, 35%) and administrative claims data (4, 9%). The most common algorithms were tree-based methods (23, 53%), neural network (NN) (14, 33%), regularized logistic regression (12, 28%), and support vector machine (SVM) (10, 23%). Most of these studies (37, 85%) were of high quality. A majority of these studies (28, 65%) reported ML algorithms with an AUC above 0.70. There was a range of variability within AUC reported by these studies with a median of 0.68 (IQR: 0.64-0.76; range: 0.50-0.90).

CONCLUSIONS

The ML algorithms involving tree-based methods, NN, regularized logistic regression, and SVM are commonly used to predict hospital readmission in the US. Further research is needed to compare the performance of ML algorithms for hospital readmission prediction.

Comparative Effectiveness of Machine Learning Approaches for Predicting Gastrointestinal Bleeds in Patients Receiving Antithrombotic Treatment

IMPORTANCE

Anticipating the risk of gastrointestinal bleeding (GIB) when initiating antithrombotic treatment (oral antiplatelets or anticoagulants) is limited by existing risk prediction models. Machine learning algorithms may result in superior predictive models to aid in clinical decision-making.

OBJECTIVE

To compare the performance of 3 machine learning approaches with the commonly used HAS-BLED (hypertension, abnormal kidney and liver function, stroke, bleeding, labile international normalized ratio, older age, and drug or alcohol use) risk score in predicting antithrombotic-related GIB.

DESIGN, SETTING, AND PARTICIPANTS

This retrospective cross-sectional study used data from the OptumLabs Data Warehouse, which contains medical and pharmacy claims on privately insured patients and Medicare Advantage enrollees in the US. The study cohort included patients 18 years or older with a history of atrial fibrillation, ischemic heart disease, or venous thromboembolism who were prescribed oral anticoagulant and/or thienopyridine antiplatelet agents between January 1, 2016, and December 31, 2019.

EXPOSURES

A cohort of patients prescribed oral anticoagulant and thienopyridine antiplatelet agents was divided into development and validation cohorts based on date of index prescription. The development cohort was used to train 3 machine learning models to predict GIB at 6 and 12 months: regularized Cox proportional hazards regression (RegCox), random survival forests (RSF), and extreme gradient boosting (XGBoost).

MAIN OUTCOMES AND MEASURES

The performance of the models for predicting GIB in the validation cohort, evaluated using the area under the receiver operating characteristic curve (AUC), sensitivity, specificity, positive predictive value, and prediction density plots. Relative importance scores were used to identify the variables that were most influential in the top-performing machine learning model.

RESULTS

In the entire study cohort of 306 463 patients, 166 177 (54.2%) were male, 193 648 (63.2%) were White, the mean (SD) age was 69.0 (12.6) years, and 12 322 (4.0%) had experienced a GIB. In the validation data set, the HAS-BLED model had an AUC of 0.60 for predicting GIB at 6 months and 0.59 at 12 months. The RegCox model performed the best in the validation set, with an AUC of 0.67 at 6 months and 0.66 at 12 months. XGBoost was similar, with AUCs of 0.67 at 6 months and 0.66 at 12 months, whereas for RSF, AUCs were 0.62 at 6 months and 0.60 at 12 months. The variables with the highest importance scores in the RegCox model were prior GI bleed (importance score, 0.72); atrial fibrillation, ischemic heart disease, and venous thromboembolism combined (importance score, 0.38); and use of gastroprotective agents (importance score, 0.32).

CONCLUSIONS AND RELEVANCE

In this cross-sectional study, the machine learning models examined showed similar performance in identifying patients at high risk for GIB after being prescribed antithrombotic agents. Two models (RegCox and XGBoost) performed modestly better than the HAS-BLED score. A prospective evaluation of the RegCox model compared with HAS-BLED may provide a better understanding of the clinical impact of improved performance.

Neural network predicts need for red blood cell transfusion for patients with acute gastrointestinal bleeding admitted to the intensive care unit

Acute gastrointestinal bleeding is the most common gastrointestinal cause for hospitalization. For high-risk patients requiring intensive care unit stay, predicting transfusion needs during the first 24 h using dynamic risk assessment may improve resuscitation with red blood cell transfusion in admitted patients with severe acute gastrointestinal bleeding. A patient cohort admitted for acute gastrointestinal bleeding (N = 2,524) was identified from the Medical Information Mart for Intensive Care III (MIMIC-III) critical care database and separated into training (N = 2,032) and internal validation (N = 492) sets. The external validation patient cohort was identified from the eICU collaborative database of patients admitted for acute gastrointestinal bleeding presenting to large urban hospitals (N = 1,526). 62 demographic, clinical, and laboratory test features were consolidated into 4-h time intervals over the first 24 h from admission. The outcome measure was the transfusion of red blood cells during each 4-h time interval. A long short-term memory (LSTM) model, a type of Recurrent Neural Network, was compared to a regression-based models on time-updated data. The LSTM model performed better than discrete time regression-based models for both internal validation (AUROC 0.81 vs 0.75 vs 0.75; P < 0.001) and external validation (AUROC 0.65 vs 0.56 vs 0.56; P < 0.001). A LSTM model can be used to predict the need for transfusion of packed red blood cells over the first 24 h from admission to help personalize the care of high-risk patients with acute gastrointestinal bleeding.

Machine Learning Compared With Conventional Statistical Models for Predicting Myocardial Infarction Readmission and Mortality: A Systematic Review

BACKGROUND

Machine learning (ML) methods are increasingly used in addition to conventional statistical modelling (CSM) for predicting readmission and mortality in patients with myocardial infarction (MI). However, the two approaches have not been systematically compared across studies of prognosis in patients with MI.

METHODS

Following PRISMA guidelines, we systematically reviewed the literature via Medline, EPub, Cochrane Central, Embase, Inspec, ACM Digital Library, and Web of Science. Eligible studies included primary research articles published from January 2000 to March 2020, comparing ML and CSM for prognostication after MI.

RESULTS

Of 7,348 articles, 112 underwent full-text review, with the final set composed of 24 articles representing 374,365 patients. ML methods included artificial neural networks (n = 12 studies), random forests (n = 11), decision trees (n = 8), support vector machines (n = 8), and Bayesian techniques (n = 7). CSM included logistic regression (n = 19 studies), existing CSM-derived risk scores (n = 12), and Cox regression (n = 2). Thirteen of 19 studies examining mortality reported higher C-indexes with the use of ML compared with CSM. One study examined readmissions at 2 different time points, with C-indexes that were higher for ML than CSM. Across all studies, a total of 29 comparisons were performed, but the majority (n = 26, 90%) found small (< 0.05) absolute differences in the C-index between ML and CSM. With the use of a modified CHARMS checklist, sources of bias were identifiable in the majority of studies, and only 2 were externally validated.

CONCLUSION

Although ML algorithms tended to have higher C-indexes than CSM for predicting death or readmission after MI, these studies exhibited threats to internal validity and were often unvalidated. Further comparisons are needed, with adherence to clinical quality standards for prognosis research. (Trial registration: PROSPERO CRD42019134896).

Prediction of Neurologically Intact Survival in Cardiac Arrest Patients without Pre-Hospital Return of Spontaneous Circulation: Machine Learning Approach

Current multimodal approaches for the prognostication of out-of-hospital cardiac arrest (OHCA) are based mainly on the prediction of poor neurological outcomes; however, it is challenging to identify patients expected to have a favorable outcome, especially before the return of spontaneous circulation (ROSC). We developed and validated a machine learning-based system to predict good outcome in OHCA patients before ROSC. This prospective, multicenter, registry-based study analyzed non-traumatic OHCA data collected between October 2015 and June 2017. We used information available before ROSC as predictor variables, and the primary outcome was neurologically intact survival at discharge, defined as cerebral performance category 1 or 2. The developed models’ robustness were evaluated and compared with various score metrics to confirm their performance. The model using a voting classifier had the best performance in predicting good neurological outcome (area under the curve = 0.926). We confirmed that the six top-weighted variables predicting neurological outcomes, such as several duration variables after the instant of OHCA and several electrocardiogram variables in the voting classifier model, showed significant differences between the two neurological outcome groups. These findings demonstrate the potential utility of a machine learning model to predict good neurological outcome of OHCA patients before ROSC.

Advancing care for acute gastrointestinal bleeding using artificial intelligence

The future of gastrointestinal bleeding will include the integration of machine learning algorithms to enhance clinician risk assessment and decision making. Machine learning algorithms have shown promise in outperforming existing clinical risk scores for both upper and lower gastrointestinal bleeding but have not been validated in any prospective clinical trials. The adoption of electronic health records provides an exciting opportunity to deploy risk prediction tools in real time and also to expand the data available to train predictive models. Machine learning algorithms can be used to identify patients with acute gastrointestinal bleeding using data extracted from the electronic health record. This can lead to an automated process to find patients with symptoms of acute gastrointestinal bleeding so that risk prediction tools can be then triggered to consistently provide decision support to the physician. Neural network models can be used to provide continuous risk predictions for patients who are at higher risk, which can be used to guide triage of patients to appropriate levels of care. Finally, the future will likely include neural network-based analysis of endoscopic stigmata of bleeding to help guide best practices for hemostasis during the endoscopic procedure. Machine learning will enhance the delivery of care at every level for patients with acute gastrointestinal bleeding through identifying very low risk patients for outpatient management, triaging high risk patients for higher levels of care, and guiding optimal intervention during endoscopy.

Machine Learning Versus Usual Care for Diagnostic and Prognostic Prediction in the Emergency Department: A Systematic Review

OBJECTIVE

Having shown promise in other medical fields, we sought to determine whether machine learning (ML) models perform better than usual care in diagnostic and prognostic prediction for emergency department (ED) patients.

METHODS

In this systematic review, we searched MEDLINE, Embase, Central, and CINAHL from inception to October 17, 2019. We included studies comparing diagnostic and prognostic prediction of ED patients by ML models to usual care methods (triage-based scores, clinical prediction tools, clinician judgment) using predictor variables readily available to ED clinicians. We extracted commonly reported performance metrics of model discrimination and classification. We used the PROBAST tool for risk of bias assessment (PROSPERO registration: CRD42020158129).

RESULTS

The search yielded 1,656 unique records, of which 23 studies involving 16,274,647 patients were included. In all seven diagnostic studies, ML models outperformed usual care in all performance metrics. In six studies assessing in-hospital mortality, the best-performing ML models had better discrimination (area under the receiver operating characteristic curve [AUROC] =0.74-0.94) than any clinical decision tool (AUROC =0.68-0.81). In four studies assessing hospitalization, ML models had better discrimination (AUROC =0.80-0.83) than triage-based scores (AUROC =0.68-0.82). Clinical heterogeneity precluded meta-analysis. Most studies had high risk of bias due to lack of external validation, low event rates, and insufficient reporting of calibration.

CONCLUSIONS

Our review suggests that ML may have better prediction performance than usual care for ED patients with a variety of clinical presentations and outcomes. However, prediction model reporting guidelines should be followed to provide clinically applicable data. Interventional trials are needed to assess the impact of ML models on patient-centered outcomes.

Machine Learning Models for Predicting Neonatal Mortality: A Systematic Review

INTRODUCTION

Approximately 7,000 newborns die every day, accounting for almost half of child deaths under 5 years of age. Deciphering which neonates are at increased risk for mortality can have an important global impact. As such, integrating high computational technology (e.g., artificial intelligence [AI]) may help identify the early and potentially modifiable predictors of neonatal mortality. Therefore, the objective of this study was to collate, critically appraise, and analyze neonatal prediction studies that included AI.

METHODS

A literature search was performed in PubMed, Cochrane, OVID, and Google Scholar. We included studies that used AI (e.g., machine learning (ML) and deep learning) to formulate prediction models for neonatal death. We excluded small studies (n < 500 individuals) and studies using only antenatal factors to predict mortality. Two independent investigators screened all articles for inclusion. The data collection consisted of study design, number of models, features used per model, feature importance, internal and/or external validation, and calibration analysis. Our primary outcome was the average area under the receiving characteristic curve (AUC) or sensitivity and specificity for all models included in each study.

RESULTS

Of 434 articles, 11 studies were included. The total number of participants was 1.26 M with gestational ages ranging from 22 weeks to term. Number of features ranged from 3 to 66 with timing of prediction as early as 5 min of life to a maximum of 7 days of age. The average number of models per study was 4, with neural network, random forest, and logistic regression comprising the most used models (58.3%). Five studies (45.5%) reported calibration plots and 2 (18.2%) conducted external validation. Eight studies reported results by AUC and 5 studies reported the sensitivity and specificity. The AUC varied from 58.3% to 97.0%. The mean sensitivities ranged from 63% to 80% and specificities from 78% to 99%. The best overall model was linear discriminant analysis, but it also had a high number of features (n = 17).

DISCUSSION/CONCLUSION

ML models can accurately predict death in neonates. This analysis demonstrates the most commonly used predictors and metrics for AI prediction models for neonatal mortality. Future studies should focus on external validation, calibration, as well as deployment of applications that can be readily accessible to health-care providers.

Machine learning vs. conventional statistical models for predicting heart failure readmission and mortality

Aims

This study aimed to review the performance of machine learning (ML) methods compared with conventional statistical models (CSMs) for predicting readmission and mortality in patients with heart failure (HF) and to present an approach to formally evaluate the quality of studies using ML algorithms for prediction modelling.

Methods and results

Following Preferred Reporting Items for Systematic Reviews and Meta‐Analyses guidelines, we performed a systematic literature search using MEDLINE, EPUB, Cochrane CENTRAL, EMBASE, INSPEC, ACM Library, and Web of Science. Eligible studies included primary research articles published between January 2000 and July 2020 comparing ML and CSMs in mortality and readmission prognosis of initially hospitalized HF patients. Data were extracted and analysed by two independent reviewers. A modified CHARMS checklist was developed in consultation with ML and biostatistics experts for quality assessment and was utilized to evaluate studies for risk of bias. Of 4322 articles identified and screened by two independent reviewers, 172 were deemed eligible for a full‐text review. The final set comprised 20 articles and 686 842 patients. ML methods included random forests (n = 11), decision trees (n = 5), regression trees (n = 3), support vector machines (n = 9), neural networks (n = 12), and Bayesian techniques (n = 3). CSMs included logistic regression (n = 16), Cox regression (n = 3), or Poisson regression (n = 3). In 15 studies, readmission was examined at multiple time points ranging from 30 to 180 day readmission, with the majority of studies (n = 12) presenting prediction models for 30 day readmission outcomes. Of a total of 21 time‐point comparisons, ML‐derived c‐indices were higher than CSM‐derived c‐indices in 16 of the 21 comparisons. In seven studies, mortality was examined at 9 time points ranging from in‐hospital mortality to 1 year survival; of these nine, seven reported higher c‐indices using ML. Two of these seven studies reported survival analyses utilizing random survival forests in their ML prediction models. Both reported higher c‐indices when using ML compared with CSMs. A limitation of studies using ML techniques was that the majority were not externally validated, and calibration was rarely assessed. In the only study that was externally validated in a separate dataset, ML was superior to CSMs (c‐indices 0.913 vs. 0.835).

Conclusions

ML algorithms had better discrimination than CSMs in most studies aiming to predict risk of readmission and mortality in HF patients. Based on our review, there is a need for external validation of ML‐based studies of prediction modelling. We suggest that ML‐based studies should also be evaluated using clinical quality standards for prognosis research. Registration: PROSPERO CRD42020134867

Proceedings from the First Global Artificial Intelligence in Gastroenterology and Endoscopy Summit

BACKGROUND AND AIMS

Artificial intelligence (AI), specifically deep learning, offers the potential to enhance the field of GI endoscopy in areas ranging from lesion detection and classification to quality metrics and documentation. Progress in this field will be measured by whether AI implementation can lead to improved patient outcomes and more efficient clinical workflow for GI endoscopists. The aims of this article are to report the findings of a multidisciplinary group of experts focusing on issues in AI research and applications related to gastroenterology and endoscopy, to review the current status of the field, and to produce recommendations for investigators developing and studying new AI technologies for gastroenterology.

METHODS

A multidisciplinary meeting was held on September 28, 2019, bringing together academic, industry, and regulatory experts in diverse fields including gastroenterology, computer and imaging sciences, machine learning, computer vision, U.S. Food and Drug Administration, and the National Institutes of Health. Recent and ongoing studies in gastroenterology and current technology in AI were presented and discussed, key gaps in knowledge were identified, and recommendations were made for research that would have the highest impact in making advances and implementation in the field of AI to gastroenterology.

RESULTS

There was a consensus that AI will transform the field of gastroenterology, particularly endoscopy and image interpretation. Powered by advanced machine learning algorithms, the use of computer vision in endoscopy has the potential to result in better prediction and treatment outcomes for patients with gastroenterology disorders and cancer. Large libraries of endoscopic images, "EndoNet," will be important to facilitate development and application of AI systems. The regulatory environment for implementation of AI systems is evolving, but common outcomes such as colon polyp detection have been highlighted as potential clinical trial endpoints. Other threshold outcomes will be important, as well as clarity on iterative improvement of clinical systems.

CONCLUSIONS

Gastroenterology is a prime candidate for early adoption of AI. AI is rapidly moving from an experimental phase to a clinical implementation phase in gastroenterology. It is anticipated that the implementation of AI in gastroenterology over the next decade will have a significant and positive impact on patient care and clinical workflows. Ongoing collaboration among gastroenterologists, industry experts, and regulatory agencies will be important to ensure that progress is rapid and clinically meaningful. However, several constraints and areas will benefit from further exploration, including potential clinical applications, implementation, structure and governance, role of gastroenterologists, and potential impact of AI in gastroenterology.

Is Artificial Intelligence Better Than Human Clinicians in Predicting Patient Outcomes?

In contrast with medical imaging diagnostics powered by artificial intelligence (AI), in which deep learning has led to breakthroughs in recent years, patient outcome prediction poses an inherently challenging problem because it focuses on events that have not yet occurred. Interestingly, the performance of machine learning-based patient outcome prediction models has rarely been compared with that of human clinicians in the literature. Human intuition and insight may be sources of underused predictive information that AI will not be able to identify in electronic data. Both human and AI predictions should be investigated together with the aim of achieving a human-AI symbiosis that synergistically and complementarily combines AI with the predictive abilities of clinicians.

Prediction of Adverse Events in Stable Non-Variceal Gastrointestinal Bleeding Using Machine Learning

Clinical risk-scoring systems are important for identifying patients with upper gastrointestinal bleeding (UGIB) who are at a high risk of hemodynamic instability. We developed an algorithm that predicts adverse events in patients with initially stable non-variceal UGIB using machine learning (ML). Using prospective observational registry, 1439 out of 3363 consecutive patients were enrolled. Primary outcomes included adverse events such as mortality, hypotension, and rebleeding within 7 days. Four machine learning algorithms, namely, logistic regression with regularization (LR), random forest classifier (RF), gradient boosting classifier (GB), and voting classifier (VC), were compared with the Glasgow-Blatchford score (GBS) and Rockall scores. The RF model showed the highest accuracies and significant improvement over conventional methods for predicting mortality (area under the curve: RF 0.917 vs. GBS 0.710), but the performance of the VC model was best in hypotension (VC 0.757 vs. GBS 0.668) and rebleeding within 7 days (VC 0.733 vs. GBS 0.694). Clinically significant variables including blood urea nitrogen, albumin, hemoglobin, platelet, prothrombin time, age, and lactate were identified by the global feature importance analysis. These results suggest that ML models will be useful early predictive tools for identifying high-risk patients with initially stable non-variceal UGIB admitted at an emergency department.

Machine Learning Prognostic Models for Gastrointestinal Bleeding Using Electronic Health Record Data

Risk assessment tools for patients with gastrointestinal bleeding may be used for determining level of care and informing management decisions. Development of models that use data from electronic health records is an important step for future deployment of such tools in clinical practice. Furthermore, machine learning tools have the potential to outperform standard clinical risk assessment tools. The authors developed a new machine learning tool for the outcome of in-hospital mortality and suggested it outperforms the intensive care unit prognostic tool, APACHE IVa. Limitations include lack of generalizability beyond intensive care unit patients, inability to use early in the hospital course, and lack of external validation.

Machine learning health-related applications in low-income and middle-income countries: a scoping review protocol

INTRODUCTION

Machine learning (ML) has been used in bio-medical research, and recently in clinical and public health research. However, much of the available evidence comes from high-income countries, where different health profiles challenge the application of this research to low/middle-income countries (LMICs). It is largely unknown what ML applications are available for LMICs that can support and advance clinical medicine and public health. We aim to address this gap by conducting a scoping review of health-related ML applications in LMICs.

METHODS AND ANALYSIS

This scoping review will follow the methodology proposed by Levac et al. The search strategy is informed by recent systematic reviews of ML health-related applications. We will search Embase, Medline and Global Health (through Ovid), Cochrane and Google Scholar; we will present the date of our searches in the final review. Titles and abstracts will be screened by two reviewers independently; selected reports will be studied by two reviewers independently. Reports will be included if they are primary research where data have been analysed, ML techniques have been used on data from LMICs and they aimed to improve health-related outcomes. We will synthesise the information following evidence mapping recommendations.

ETHICS AND DISSEMINATION

The review will provide a comprehensive list of health-related ML applications in LMICs. The results will be disseminated through scientific publications. We also plan to launch a website where ML models can be hosted so that researchers, policymakers and the general public can readily access them.

Validation of a Machine Learning Model That Outperforms Clinical Risk Scoring Systems for Upper Gastrointestinal Bleeding

BACKGROUND & AIMS

Scoring systems are suboptimal for determining risk in patients with upper gastrointestinal bleeding (UGIB); these might be improved by a machine learning model. We used machine learning to develop a model to calculate the risk of hospital-based intervention or death in patients with UGIB and compared its performance with other scoring systems.

METHODS

We analyzed data collected from consecutive unselected patients with UGIB from medical centers in 4 countries (the United States, Scotland, England, and Denmark; n = 1958) from March 2014 through March 2015. We used the data to derive and internally validate a gradient-boosting machine learning model to identify patients who met a composite endpoint of hospital-based intervention (transfusion or hemostatic intervention) or death within 30 days. We compared the performance of the machine learning prediction model with validated pre-endoscopic clinical risk scoring systems (the Glasgow-Blatchford score [GBS], admission Rockall score, and AIMS65). We externally validated the machine learning model using data from 2 Asia-Pacific sites (Singapore and New Zealand; n = 399). Performance was measured by area under receiver operating characteristic curve (AUC) analysis.

RESULTS

The machine learning model identified patients who met the composite endpoint with an AUC of 0.91 in the internal validation set; the clinical scoring systems identified patients who met the composite endpoint with AUC values of 0.88 for the GBS (P = .001), 0.73 for Rockall score (P < .001), and 0.78 for AIMS65 score (P < .001). In the external validation cohort, the machine learning model identified patients who met the composite endpoint with an AUC of 0.90, the GBS with an AUC of 0.87 (P = .004), the Rockall score with an AUC of 0.66 (P < .001), and the AIMS65 with an AUC of 0.64 (P < .001). At cutoff scores at which the machine learning model and GBS identified patients who met the composite endpoint with 100% sensitivity, the specificity values were 26% with the machine learning model versus 12% with GBS (P < .001).

CONCLUSIONS

We developed a machine learning model that identifies patients with UGIB who met a composite endpoint of hospital-based intervention or death within 30 days with a greater AUC and higher levels of specificity, at 100% sensitivity, than validated clinical risk scoring systems. This model could increase identification of low-risk patients who can be safely discharged from the emergency department for outpatient management.

Clinical utility of pre-endoscopy risk scores in upper gastrointestinal bleeding

Introduction: Acute upper-gastrointestinal bleeding (AUGIB) is a common medical emergency, with an incidence of 103-172 per 100,000 in the United Kingdom (UK) and mortality of 2% to 10%. Early and accurate prediction of the severity of an AUGIB episode may help guide management, including in or outpatient management, level of care required, and timing of endoscopy. This article aims to address the clinical utility of the various pre-endoscopic risk assessment tools used in AUGIB.Areas covered: The authors undertook a literature review of the current evidence on the pre-endoscopic risk assessment scores. Additional the authors discuss the recently published novel risk assessment scores.Expert opinion: The evidence shows that GBS is the most clinically useful risk assessment score in correctly identifying very low-risk patients suitable for outpatient management. At present, research is ongoing to assess machine learning in the assessment of patients presenting with AUGIB. More research is needed but it shows promise for the future.

Validation of a Machine Learning Model That Outperforms Clinical Risk Scoring Systems for Upper Gastrointestinal Bleeding

BACKGROUND & AIMS

Scoring systems are suboptimal for determining risk in patients with upper gastrointestinal bleeding (UGIB); these might be improved by a machine learning model. We used machine learning to develop a model to calculate the risk of hospital-based intervention or death in patients with UGIB and compared its performance with other scoring systems.

METHODS

We analyzed data collected from consecutive unselected patients with UGIB from medical centers in 4 countries (the United States, Scotland, England, and Denmark; n = 1958) from March 2014 through March 2015. We used the data to derive and internally validate a gradient-boosting machine learning model to identify patients who met a composite endpoint of hospital-based intervention (transfusion or hemostatic intervention) or death within 30 days. We compared the performance of the machine learning prediction model with validated pre-endoscopic clinical risk scoring systems (the Glasgow-Blatchford score [GBS], admission Rockall score, and AIMS65). We externally validated the machine learning model using data from 2 Asia-Pacific sites (Singapore and New Zealand; n = 399). Performance was measured by area under receiver operating characteristic curve (AUC) analysis.

RESULTS

The machine learning model identified patients who met the composite endpoint with an AUC of 0.91 in the internal validation set; the clinical scoring systems identified patients who met the composite endpoint with AUC values of 0.88 for the GBS (P = .001), 0.73 for Rockall score (P < .001), and 0.78 for AIMS65 score (P < .001). In the external validation cohort, the machine learning model identified patients who met the composite endpoint with an AUC of 0.90, the GBS with an AUC of 0.87 (P = .004), the Rockall score with an AUC of 0.66 (P < .001), and the AIMS65 with an AUC of 0.64 (P < .001). At cutoff scores at which the machine learning model and GBS identified patients who met the composite endpoint with 100% sensitivity, the specificity values were 26% with the machine learning model versus 12% with GBS (P < .001).

CONCLUSIONS

We developed a machine learning model that identifies patients with UGIB who met a composite endpoint of hospital-based intervention or death within 30 days with a greater AUC and higher levels of specificity, at 100% sensitivity, than validated clinical risk scoring systems. This model could increase identification of low-risk patients who can be safely discharged from the emergency department for outpatient management.