Prediction of adverse cardiac events in emergency department patients with chest pain using machine learning for variable selection

Performance of machine learning models in predicting difficult laryngoscopy in the emergency department: a single-centre retrospective study comparing with conventional regression method

BACKGROUND

Emergency endotracheal intubation is a critical skill for managing airway emergencies in the emergency department (ED). Accurate prediction of difficult laryngoscopy is essential for improving first-attempt success, minimizing complications, optimizing resource utilization, and enhancing patient outcomes. Traditional methods, such as the LEMON criteria, have limited predictive accuracy. Machine learning (ML) offers advanced predictive capabilities by analyzing large datasets and identifying complex variable interactions. This study aimed to develop and validate the performance of ML models for predicting difficult laryngoscopy in the ED, comparing it with a conventional regression model.

METHODS

A retrospective cohort study was conducted on 4,370 adult patients who underwent intubation in the ED at Thammasat University Hospital. Difficult laryngoscopy was defined as a Cormack-Lehane grade III or IV. Patients were divided into development (training, 70%) and validation (testing, 30%) cohorts. Predictors of difficult laryngoscopy were identified using multivariable stepwise backward elimination logistic regression and were used to develop ML models, including Logistic Regression, Decision Tree, Random Forest, and XGBoost. Model performance was evaluated using the area under the receiver operating characteristic curve (AuROC), accuracy, precision, recall, and F1-score. Validation was performed on the validation cohort to confirm model accuracy.

RESULTS

Nine significant predictors were identified: male sex, trauma, absence of neuromuscular blocking agents, large incisors, large tongue, limited mouth opening, short thyrohyoid distance, obstructed airway, and poor neck mobility. The Random Forest model demonstrated the highest predictive performance, with an AuROC of 0.82 (95% CI: 0.78-0.85), accuracy of 0.89, recall of 0.89, and F1-score of 0.87, outperforming conventional regression (AuROC 0.76, 95% CI: 0.73-0.78) and other ML models. DeLong's test confirmed a statistically significant difference in AuROC between the two models (p = 0.002). The Decision Tree showed limited performance due to overfitting, while XGBoost demonstrated strong precision. No significant differences were found when comparing the two models with conventional regression (p = 0.498 and 0.496, respectively).

CONCLUSION

The Random Forest model provides the most robust prediction of difficult laryngoscopy, outperforming both conventional and other ML methods. While ML models improve predictive accuracy, logistic regression remains a practical option in resource-limited settings. Integrating ML into clinical workflows could enhance decision-making, resource allocation, and patient safety in emergency airway management. Future research should prioritize external validation and real-world implementation.

Emergence of Artificial Intelligence and Machine Learning Models in Sudden Cardiac Arrest: A Comprehensive Review of Predictive Performance and Clinical Decision Support

Sudden cardiac death/sudden cardiac arrest (SCD/SCA) is an increasingly prevalent cause of mortality globally, particularly in individuals with preexisting cardiac conditions. The ambiguous premortem warnings and the restricted interventional window related to SCD account for the complexity of the condition. Current reports suggest SCD to be accountable for 20% of all deaths hence accurately predicting SCD risk is an imminent concern. Traditional approaches for predicting SCA, particularly "track-and-trigger" warning systems have demonstrated considerable inadequacies, including low sensitivity, false alarms, decreased diagnostic liability, reliance on clinician involvement, and human errors. Artificial intelligence (AI) and machine learning (ML) models have demonstrated near-perfect accuracy in predicting SCA risk, allowing clinicians to intervene timely. Given the constraints of current diagnostics, exploring the benefits of AI and ML models in enhancing outcomes for SCA/SCD is imperative. This review article aims to investigate the efficacy of AI and ML models in predicting and managing SCD, particularly targeting accuracy in prediction.

Data analytics approach for short- and long-term mortality prediction following acute non-ST-elevation myocardial infarction (NSTEMI) and Unstable Angina (UA) in Asians

BACKGROUND

Traditional risk assessment tools often lack accuracy when predicting the short- and long-term mortality following a non-ST-segment elevation myocardial infarction (NSTEMI) or Unstable Angina (UA) in specific population.

OBJECTIVE

To employ machine learning (ML) and stacked ensemble learning (EL) methods in predicting short- and long-term mortality in Asian patients diagnosed with NSTEMI/UA and to identify the associated features, subsequently evaluating these findings against established risk scores.

METHODS

We analyzed data from the National Cardiovascular Disease Database for Malaysia (2006-2019), representing a diverse NSTEMI/UA Asian cohort. Algorithm development utilized in-hospital records of 9,518 patients, 30-day data from 7,133 patients, and 1-year data from 7,031 patients. This study utilized 39 features, including demographic, cardiovascular risk, medication, and clinical features. In the development of the stacked EL model, four base learner algorithms were employed: eXtreme Gradient Boosting (XGB), Support Vector Machine (SVM), Naive Bayes (NB), and Random Forest (RF), with the Generalized Linear Model (GLM) serving as the meta learner. Significant features were chosen and ranked using ML feature importance with backward elimination. The predictive performance of the algorithms was assessed using the area under the curve (AUC) as a metric. Validation of the algorithms was conducted against the TIMI for NSTEMI/UA using a separate validation dataset, and the net reclassification index (NRI) was subsequently determined.

RESULTS

Using both complete and reduced features, the algorithm performance achieved an AUC ranging from 0.73 to 0.89. The top-performing ML algorithm consistently surpassed the TIMI risk score for in-hospital, 30-day, and 1-year predictions (with AUC values of 0.88, 0.88, and 0.81, respectively, all p < 0.001), while the TIMI scores registered significantly lower at 0.55, 0.54, and 0.61. This suggests the TIMI score tends to underestimate patient mortality risk. The net reclassification index (NRI) of the best ML algorithm for NSTEMI/UA patients across these periods yielded an NRI between 40-60% (p < 0.001) relative to the TIMI NSTEMI/UA risk score. Key features identified for both short- and long-term mortality included age, Killip class, heart rate, and Low-Molecular-Weight Heparin (LMWH) administration.

CONCLUSIONS

In a broad multi-ethnic population, ML approaches outperformed conventional TIMI scoring in classifying patients with NSTEMI and UA. ML allows for the precise identification of unique characteristics within individual Asian populations, improving the accuracy of mortality predictions. Continuous development, testing, and validation of these ML algorithms holds the promise of enhanced risk stratification, thereby revolutionizing future management strategies and patient outcomes.

Development and validation of a nomogram-assisted tool to predict potentially reversible cognitive frailty in Chinese community-living older adults

BACKGROUND

Cognitive frailty (CF) is a complex and heterogeneous clinical syndrome that indicates the onset of neurodegenerative processes and poor prognosis. In order to prevent the occurrence and development of CF in real world, we intended to develop and validate a simple and timely diagnostic instrument based on comprehensive geriatric assessment that will identify patients with potentially reversible CF (PRCF).

METHODS

750 community-dwelling individuals aged over 60 years were randomly allocated to either a training or validation set at a 4:1 ratio. We used the operator regression model offering the least absolute data dimension shrinkage and feature selection among candidate predictors. PRCF was defined as the presence of physical pre-frailty, frailty, and mild cognitive impairment (MCI) occurring simultaneously. Multivariate logistic regression was conducted to build a diagnostic tool to present data as a nomogram. The performance of the tool was assessed with respect to its calibration, discrimination, and clinical usefulness.

RESULTS

PRCF was observed in 326 patients (43%). Predictors in the tool were educational background, coronary heart disease, handgrip strength, gait speed, instrumental activity of daily living (IADL) disability, subjective cognitive decline (SCD) and five-times-sit-to-stand test. The diagnostic nomogram-assisted tool exhibited good calibration and discrimination with a C-index of 0.805 and a higher C-index of 0.845 in internal validation. The calibration plots demonstrated strong agreement in both the training and validation sets, while decision curve analysis confirmed the nomogram's efficacy in clinical practice.

CONCLUSIONS

This tool can effectively identify older adults at high risk for PRCF, enabling physicians to make informed clinical decisions and implement proper patient-centered individual interventions.

Artificial Intelligence in Ventricular Arrhythmias and Sudden Death

Sudden cardiac arrest due to lethal ventricular arrhythmias is a major cause of mortality worldwide and results in more years of potential life lost than any individual cancer. Most of these sudden cardiac arrest events occur unexpectedly in individuals who have not been identified as high-risk due to the inadequacy of current risk stratification tools. Artificial intelligence tools are increasingly being used to solve complex problems and are poised to help with this major unmet need in the field of clinical electrophysiology. By leveraging large and detailed datasets, artificial intelligence-based prediction models have the potential to enhance the risk stratification of lethal ventricular arrhythmias. This review presents a synthesis of the published literature and a discussion of future directions in this field.

Artificial Intelligence in Resuscitation: A Scoping Review

INTRODUCTION

Cardiac arrest is a significant cause of premature mortality and severe disability. Despite the death rate steadily decreasing over the previous decade, only 22% of survivors achieve good clinical status and only 25% of patients survive until their discharge from the hospital. The objective of this scoping review was to review relevant AI modalities and the main potential applications of AI in resuscitation.

METHODS

We conducted the literature search for related studies in PubMed, EMBASE, and Google Scholar. We included peer-reviewed publications and articles in the press, pooling and characterizing the data by their model types, goals, and benefits.

RESULTS

After identifying 268 original studies, we chose 59 original studies (reporting 1,817,419 patients) to include in the qualitative synthesis. AI-based methods appear to be superior to traditional methods in achieving high-level performance.

CONCLUSION

AI might be useful in predicting cardiac arrest, heart rhythm disorders, and post-cardiac arrest outcomes, as well as in the delivery of drone-delivered defibrillators and notification of dispatchers. AI-powered technologies could be valuable assistants to continuously track patient conditions. Healthcare professionals should assist in the research and development of AI-powered technologies as well as their implementation into clinical practice.

Performance of digital early warning score (NEWS2) in a cardiac specialist setting: retrospective cohort study

INTRODUCTION

Patients with cardiovascular diseases (CVD) are at significant risk of developing critical events. Early warning scores (EWS) are recommended for early recognition of deteriorating patients, yet their performance has been poorly studied in cardiac care settings. Standardisation and integrated National Early Warning Score 2 (NEWS2) in electronic health records (EHRs) are recommended yet have not been evaluated in specialist settings.

OBJECTIVE

To investigate the performance of digital NEWS2 in predicting critical events: death, intensive care unit (ICU) admission, cardiac arrest and medical emergencies.

METHODS

Retrospective cohort analysis.

STUDY COHORT

Individuals admitted with CVD diagnoses in 2020; including patients with COVID-19 due to conducting the study during the COVID-19 pandemic.

MEASURES

We tested the ability of NEWS2 in predicting the three critical outcomes from admission and within 24 hours before the event. NEWS2 was supplemented with age and cardiac rhythm and investigated. We used logistic regression analysis with the area under the receiver operating characteristic curve (AUC) to measure discrimination.

RESULTS

In 6143 patients admitted under cardiac specialties, NEWS2 showed moderate to low predictive accuracy of traditionally examined outcomes: death, ICU admission, cardiac arrest and medical emergency (AUC: 0.63, 0.56, 0.70 and 0.63, respectively). Supplemented NEWS2 with age showed no improvement while age and cardiac rhythm improved discrimination (AUC: 0.75, 0.84, 0.95 and 0.94, respectively). Improved performance was found of NEWS2 with age for COVID-19 cases (AUC: 0.96, 0.70, 0.87 and 0.88, respectively).

CONCLUSION

The performance of NEWS2 in patients with CVD is suboptimal, and fair for patients with CVD with COVID-19 to predict deterioration. Adjustment with variables that strongly correlate with critical cardiovascular outcomes, that is, cardiac rhythm, can improve the model. There is a need to define critical endpoints, engagement with clinical experts in development and further validation and implementation studies of EHR-integrated EWS in cardiac specialist settings.

Tree-Based Algorithms and Association Rule Mining for Predicting Patients’ Neurological Outcomes After First-Aid Treatment for an Out-of-Hospital Cardiac Arrest During COVID-19 Pandemic: Application of Data Mining

Objective

The authors performed several tree-based algorithms and an association rules mining as data mining tools to find useful determinants for neurological outcomes in out-of-hospital cardiac arrest (OHCA) patients as well as to assess the effect of the first-aid and basic characteristics in the EMS system.

Patients and Methods

This was a retrospective cohort study. The outcome was Cerebral Performance Categories grading on OHCA patients at hospital discharge. Decision tree-based models inclusive of C4.5 algorithm, classification and regression tree and random forest were built to determine an OHCA patient’s prognosis. Association rules mining was another data mining method which we used to find the combination of prognostic factors linked to the outcome.

Results

The total of 3520 patients were included in the final analysis. The mean age was 67.53 (±18.4) year-old and 63.4% were men. To overcome the imbalance outcome issue in machine learning, the random forest has a better predictive ability for OHCA patients in overall accuracy (91.19%), weighted precision (88.76%), weighted recall (91.20%) and F1 score (0.9) by oversampling adjustment. Under association rules mining, patients who had any witness on the spot when encountering OHCA or who had ever ROSC during first-aid would be highly correlated with good CPC prognosis.

Conclusion

The random forest has a better predictive ability for OHCA patients. This paper provides a role model applying several machine learning algorithms to the first-aid clinical assessment that will be promising combining with Artificial Intelligence for applying to emergency medical services.

Explainable AI (XAI) Applied in Machine Learning for Pain Modeling: A Review

Pain is a complex term that describes various sensations that create discomfort in various ways or types inside the human body. Generally, pain has consequences that range from mild to severe in different organs of the body and will depend on the way it is caused, which could be an injury, illness or medical procedures including testing, surgeries or therapies, etc. With recent advances in artificial-intelligence (AI) systems associated in biomedical and healthcare settings, the contiguity of physician, clinician and patient has shortened. AI, however, has more scope to interpret the pain associated in patients with various conditions by using any physiological or behavioral changes. Facial expressions are considered to give much information that relates with emotions and pain, so clinicians consider these changes with high importance for assessing pain. This has been achieved in recent times with different machine-learning and deep-learning models. To accentuate the future scope and importance of AI in medical field, this study reviews the explainable AI (XAI) as increased attention is given to an automatic assessment of pain. This review discusses how these approaches are applied for different pain types.

A Computer-Assisted System for Early Mortality Risk Prediction in Patients with Traumatic Brain Injury Using Artificial Intelligence Algorithms in Emergency Room Triage

Traumatic brain injury (TBI) remains a critical public health challenge. Although studies have found several prognostic factors for TBI, a useful early predictive tool for mortality has yet to be developed in the triage of the emergency room. This study aimed to use machine learning algorithms of artificial intelligence (AI) to develop predictive models for TBI patients in the emergency room triage. We retrospectively enrolled 18,249 adult TBI patients in the electronic medical records of three hospitals of Chi Mei Medical Group from January 2010 to December 2019, and undertook the 12 potentially predictive feature variables for predicting mortality during hospitalization. Six machine learning algorithms including logistical regression (LR) random forest (RF), support vector machines (SVM), LightGBM, XGBoost, and multilayer perceptron (MLP) were used to build the predictive model. The results showed that all six predictive models had high AUC from 0.851 to 0.925. Among these models, the LR-based model was the best model for mortality risk prediction with the highest AUC of 0.925; thus, we integrated the best model into the existed hospital information system for assisting clinical decision-making. These results revealed that the LR-based model was the best model to predict the mortality risk in patients with TBI in the emergency room. Since the developed prediction system can easily obtain the 12 feature variables during the initial triage, it can provide quick and early mortality prediction to clinicians for guiding deciding further treatment as well as helping explain the patient’s condition to family members.

Using machine learning for the personalised prediction of revision endoscopic sinus surgery

BACKGROUND

Revision endoscopic sinus surgery (ESS) is often considered for chronic rhinosinusitis (CRS) if maximal conservative treatment and baseline ESS prove insufficient. Emerging research outlines the risk factors of revision ESS. However, accurately predicting revision ESS at the individual level remains uncertain. This study aims to examine the prediction accuracy of revision ESS and to identify the effects of risk factors at the individual level.

METHODS

We collected demographic and clinical variables from the electronic health records of 767 surgical CRS patients ≥16 years of age. Revision ESS was performed on 111 (14.5%) patients. The prediction accuracy of revision ESS was examined by training and validating different machine learning models, while the effects of variables were analysed using the Shapley values and partial dependence plots.

RESULTS

The logistic regression, gradient boosting and random forest classifiers performed similarly in predicting revision ESS. Area under the receiving operating characteristic curve (AUROC) values were 0.744, 0.741 and 0.730, respectively, using data collected from the baseline visit until six months after baseline ESS. The length of time during which data were collected improved the prediction performance. For data collection times of 0, 3, 6 and 12 months after baseline ESS, AUROC values for the logistic regression were 0.682, 0.715, 0.744 and 0.784, respectively. The number of visits before or after baseline ESS, the number of days from the baseline visit to the baseline ESS, patient age, CRS with nasal polyps (CRSwNP), asthma, non-steroidal anti-inflammatory drug exacerbated respiratory disease and immunodeficiency or suspicion of it all associated with revision ESS. Patient age and number of visits before baseline ESS carried non-linear effects for predictions.

CONCLUSIONS

Intelligent data analysis identified important predictors of revision ESS at the individual level, such as the frequency of clinical visits, patient age, Type 2 high diseases and immunodeficiency or a suspicion of it.

Neonatal Adverse Events' Trigger Tool Setup With Random Forest

OBJECTIVE

This study aimed to develop a trigger tool for detection of neonatal adverse events (AEs) and to validate its effectiveness.

STUDY DESIGN

Random forest (RF) algorithm was used to build the predictive model by analyzing data from the medical records of 782 neonates in our previous study. Thirteen variables for each patient were used to predict neonatal AEs. Next, the critical variables were selected based on recursive elimination of variables to form the list of triggers. Then, a trigger tool with those triggers was established and tested by reviewing medical records. The positive predictive value of individual triggers and of the entire tool was evaluated.

RESULTS

Data from 782 neonates, including 297 patients with and 485 patients without AEs, were collected to build the original RF model. Then, the 6 most important variables, including diarrhea, antibiotic use, fever, death, skin damage, and suspected necrotizing enterocolitis, were selected to establish a neonate-focused trigger tool. The forest with the 6 variables predicted AEs with a sensitivity of 70.7%, a specificity of 92.0%, and an error rate of 16.1%. In a validation study of the trigger tool, 655 neonates with birth weights ≥1500 g were enrolled, and review of their medical records revealed 1709 triggers and 1172 unique AEs. The 3 most common AEs identified were skin damage, iatrogenic diarrhea, and environmental factor-related fever. The total positive predictive value of the trigger tool was 0.686.

CONCLUSIONS

The neonate-focused trigger tool developed using the RF algorithm efficiently and reliably identifies AEs among hospitalized neonates with birth weights ≥1500 g.

Artificial Intelligence in Predicting Cardiac Arrest: Scoping Review

BACKGROUND

Cardiac arrest is a life-threatening cessation of activity in the heart. Early prediction of cardiac arrest is important, as it allows for the necessary measures to be taken to prevent or intervene during the onset. Artificial intelligence (AI) technologies and big data have been increasingly used to enhance the ability to predict and prepare for the patients at risk.

OBJECTIVE

This study aims to explore the use of AI technology in predicting cardiac arrest as reported in the literature.

METHODS

A scoping review was conducted in line with the guidelines of the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) extension for scoping reviews. Scopus, ScienceDirect, Embase, the Institute of Electrical and Electronics Engineers, and Google Scholar were searched to identify relevant studies. Backward reference list checks of the included studies were also conducted. Study selection and data extraction were independently conducted by 2 reviewers. Data extracted from the included studies were synthesized narratively.

RESULTS

Out of 697 citations retrieved, 41 studies were included in the review, and 6 were added after backward citation checking. The included studies reported the use of AI in the prediction of cardiac arrest. Of the 47 studies, we were able to classify the approaches taken by the studies into 3 different categories: 26 (55%) studies predicted cardiac arrest by analyzing specific parameters or variables of the patients, whereas 16 (34%) studies developed an AI-based warning system. The remaining 11% (5/47) of studies focused on distinguishing patients at high risk of cardiac arrest from patients who were not at risk. Two studies focused on the pediatric population, and the rest focused on adults (45/47, 96%). Most of the studies used data sets with a size of <10,000 samples (32/47, 68%). Machine learning models were the most prominent branch of AI used in the prediction of cardiac arrest in the studies (38/47, 81%), and the most used algorithm was the neural network (23/47, 49%). K-fold cross-validation was the most used algorithm evaluation tool reported in the studies (24/47, 51%).

CONCLUSIONS

AI is extensively used to predict cardiac arrest in different patient settings. Technology is expected to play an integral role in improving cardiac medicine. There is a need for more reviews to learn the obstacles to the implementation of AI technologies in clinical settings. Moreover, research focusing on how to best provide clinicians with support to understand, adapt, and implement this technology in their practice is also necessary.

Retrospective validation of a machine learning clinical decision support tool for myocardial infarction risk stratification

Diagnosis and appropriate intervention for myocardial infarction (MI) are time-sensitive but rely on clinical measures that can be progressive and initially inconclusive, underscoring the need for an accurate and early predictor of MI to support diagnostic and clinical management decisions. The objective of this study was to develop a machine learning algorithm (MLA) to predict MI diagnosis based on electronic health record data (EHR) readily available during Emergency Department assessment. An MLA was developed using retrospective patient data. The MLA used patient data as they became available in the first 3 h of care to predict MI diagnosis (defined by International Classification of Diseases, 10th revision code) at any time during the encounter. The MLA obtained an area under the receiver operating characteristic curve of 0.87, sensitivity of 87% and specificity of 70%, outperforming the comparator scoring systems TIMI and GRACE on all metrics. An MLA can synthesize complex EHR data to serve as a clinically relevant risk stratification tool for MI.

Accuracy of Machine Learning Models to Predict In-hospital Cardiac Arrest: A Systematic Review

PURPOSE/AIMS

Despite advances in healthcare, the incidence of in-hospital cardiac arrest (IHCA) has continued to rise for the past decade. Identifying those patients at risk has proven challenging. Our objective was to conduct a systematic review of the literature to compare the IHCA predictive performance of machine learning (ML) models with the Modified Early Warning Score (MEWS).

DESIGN

The systematic review was conducted following the Preferred Reporting Items of Systematic Review and Meta-Analysis guidelines and registered on PROSPERO CRD42020182357.

METHOD

Data extraction was completed using the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies checklist. The risk of bias and applicability were evaluated using the Prediction model Risk of Bias Assessment Tool.

RESULTS

Nine articles were included in this review that developed or validated IHCA prediction models and compared them with the MEWS. The studies by Jang et al and Kim et al showed that their ML models outperformed MEWS to predict IHCA with good to excellent predictive performance.

CONCLUSIONS

The ML models presented in this systematic review demonstrate a novel approach to predicting IHCA. All included studies suggest that ML models had similar or better predictive performance compared with MEWS. However, there is substantial variability in performance measures and concerns for risk of bias.

Validação de um Algoritmo de Inteligência Artificial para a Predição Diagnóstica de Doença Coronariana: Comparação com um Modelo Estatístico Tradicional

Fundamento:

Objetivo:

Métodos:

Resultados:

Conclusão:

Predicting in-hospital mortality in adult non-traumatic emergency department patients: a retrospective comparison of the Modified Early Warning Score (MEWS) and machine learning approach

Background

A feasible and accurate risk prediction systems for emergency department (ED) patients is urgently required. The Modified Early Warning Score (MEWS) is a wide-used tool to predict clinical outcomes in ED. Literatures showed that machine learning (ML) had better predictability in specific patient population than traditional scoring system. By analyzing a large multicenter dataset, we aim to develop a ML model to predict in-hospital morality of the adult non traumatic ED patients for different time stages, and comparing performance with other ML models and MEWS.

Methods

A retrospective observational cohort study was conducted in five Taiwan EDs including two tertiary medical centers and three regional hospitals. All consecutively adult (>17 years old) non-traumatic patients admit to ED during a 9-year period (January first, 2008 to December 31th, 2016) were included. Exclusion criteria including patients with (1) out-of-hospital cardiac arrest and (2) discharge against medical advice and transferred to other hospital (3) missing collect variables. The primary outcome was in-hospital mortality and were categorized into 6, 24, 72, 168 hours mortality. MEWS was calculated by systolic blood pressure, pulse rate, respiratory rate, body temperature, and level of consciousness. An ensemble supervised stacking ML model was developed and compared to sensitive and unsensitive Xgboost, Random Forest, and Adaboost. We conducted a performance test and examine both the area under the receiver operating characteristic (AUROC) and the area under the precision and recall curve (AUPRC) as the comparative measures.

Result

After excluding 182,001 visits (7.46%), study group was consisted of 24,37,326 ED visits. The dataset was split into 67% training data and 33% test data for ML model development. There was no statistically difference found in the characteristics between two groups. For the prediction of 6, 24, 72, 168 hours in-hospital mortality, the AUROC of MEW and ML mode was 0.897, 0.865, 0.841, 0.816 and 0.939, 0.928, 0.913, 0.902 respectively. The stacking ML model outperform other ML model as well. For the prediction of in-hospital mortality over 48-hours, AUPRC performance of MEWS drop below 0.1, while the AUPRC of ML mode was 0.317 in 6 hours and 0.2150 in 168 hours. For each time frame, ML model achieved statistically significant higher AUROC and AUPRC than MEWS (all P < 0.001). Both models showed decreasing prediction ability as time elapse, but there was a trend that the gap of AUROC values between two model increases gradually (P < 0.001). Three MEWS thresholds (score >3, >4, and >5) were determined as baselines for comparison, ML mode consistently showed improved or equally performance in sensitivity, PPV, NPV, but not in specific.

Conclusion

Stacking ML methods improve predicted in-hospital mortality than MEWS in adult non-traumatic ED patients, especially in the prediction of delayed mortality.

Detecting Pathogen Exposure During the Non-symptomatic Incubation Period Using Physiological Data: Proof of Concept in Non-human Primates

Background and Objectives: Early warning of bacterial and viral infection, prior to the development of overt clinical symptoms, allows not only for improved patient care and outcomes but also enables faster implementation of public health measures (patient isolation and contact tracing). Our primary objectives in this effort are 3-fold. First, we seek to determine the upper limits of early warning detection through physiological measurements. Second, we investigate whether the detected physiological response is specific to the pathogen. Third, we explore the feasibility of extending early warning detection with wearable devices. Research Methods: For the first objective, we developed a supervised random forest algorithm to detect pathogen exposure in the asymptomatic period prior to overt symptoms (fever). We used high-resolution physiological telemetry data (aortic blood pressure, intrathoracic pressure, electrocardiograms, and core temperature) from non-human primate animal models exposed to two viral pathogens: Ebola and Marburg (N = 20). Second, to determine reusability across different pathogens, we evaluated our algorithm against three independent physiological datasets from non-human primate models (N = 13) exposed to three different pathogens: Lassa and Nipah viruses and Y. pestis. For the third objective, we evaluated performance degradation when the algorithm was restricted to features derived from electrocardiogram (ECG) waveforms to emulate data from a non-invasive wearable device. Results: First, our cross-validated random forest classifier provides a mean early warning of 51 ± 12 h, with an area under the receiver-operating characteristic curve (AUC) of 0.93 ± 0.01. Second, our algorithm achieved comparable performance when applied to datasets from different pathogen exposures - a mean early warning of 51 ± 14 h and AUC of 0.95 ± 0.01. Last, with a degraded feature set derived solely from ECG, we observed minimal degradation - a mean early warning of 46 ± 14 h and AUC of 0.91 ± 0.001. Conclusion: Under controlled experimental conditions, physiological measurements can provide over 2 days of early warning with high AUC. Deviations in physiological signals following exposure to a pathogen are due to the underlying host's immunological response and are not specific to the pathogen. Pre-symptomatic detection is strong even when features are limited to ECG-derivatives, suggesting that this approach may translate to non-invasive wearable devices.

A review on utilizing machine learning technology in the fields of electronic emergency triage and patient priority systems in telemedicine: Coherent taxonomy, motivations, open research challenges and recommendations for intelligent future work

BACKGROUND

With the remarkable increasing in the numbers of patients, the triaging and prioritizing patients into multi-emergency level is required to accommodate all the patients, save more lives, and manage the medical resources effectively. Triaging and prioritizing patients becomes particularly challenging especially for the patients who are far from hospital and use telemedicine system. To this end, the researchers exploiting the useful tool of machine learning to address this challenge. Hence, carrying out an intensive investigation and in-depth study in the field of using machine learning in E-triage and patient priority are essential and required.

OBJECTIVES

This research aims to (1) provide a literature review and an in-depth study on the roles of machine learning in the fields of electronic emergency triage (E-triage) and prioritize patients for fast healthcare services in telemedicine applications. (2) highlight the effectiveness of machine learning methods in terms of algorithms, medical input data, output results, and machine learning goals in remote healthcare telemedicine systems. (3) present the relationship between machine learning goals and the electronic triage processes specifically on the: triage levels, medical features for input, outcome results as outputs, and the relevant diseases. (4), the outcomes of our analyses are subjected to organize and propose a cross-over taxonomy between machine learning algorithms and telemedicine structure. (5) present lists of motivations, open research challenges and recommendations for future intelligent work for both academic and industrial sectors in telemedicine and remote healthcare applications.

METHODS

An intensive research is carried out by reviewing all articles related to the field of E-triage and remote priority systems that utilise machine learning algorithms and sensors. We have searched all related keywords to investigate the databases of Science Direct, IEEE Xplore, Web of Science, PubMed, and Medline for the articles, which have been published from January 2012 up to date.

RESULTS

A new crossover matching between machine learning methods and telemedicine taxonomy is proposed. The crossover-taxonomy is developed in this study to identify the relationship between machine learning algorithm and the equivalent telemedicine categories whereas the machine learning algorithm has been utilized. The impact of utilizing machine learning is composed in proposing the telemedicine architecture based on synchronous (real-time/ online) and asynchronous (store-and-forward / offline) structure. In addition to that, list of machine learning algorithms, list of the performance metrics, list of inputs data and outputs results are presented. Moreover, open research challenges, the benefits of utilizing machine learning and the recommendations for new research opportunities that need to be addressed for the synergistic integration of multidisciplinary works are organized and presented accordingly.

DISCUSSION

The state-of-the-art studies on the E-triage and priority systems that utilise machine learning algorithms in telemedicine architecture are discussed. This approach allows the researchers to understand the modernisation of healthcare systems and the efficient use of artificial intelligence and machine learning. In particular, the growing worldwide population and various chronic diseases such as heart chronic diseases, blood pressure and diabetes, require smart health monitoring systems in E-triage and priority systems, in which machine learning algorithms could be greatly beneficial.

CONCLUSIONS

Although research directions on E-triage and priority systems that use machine learning algorithms in telemedicine vary, they are equally essential and should be considered. Hence, we provide a comprehensive review to emphasise the advantages of the existing research in multidisciplinary works of artificial intelligence, machine learning and healthcare services.

Applications of machine learning to undifferentiated chest pain in the emergency department: A systematic review

BACKGROUND

Chest pain is amongst the most common reason for presentation to the emergency department (ED). There are many causes of chest pain, and it is important for the emergency physician to quickly and accurately diagnose life threatening causes such as acute myocardial infarction (AMI). Multiple clinical decision tools have been developed to assist clinicians in risk stratifying patients with chest. There is growing recognition that machine learning (ML) will have a significant impact on the practice of medicine in the near future and may assist with diagnosis and risk stratification. This systematic review aims to evaluate how ML has been applied to adults presenting to the ED with undifferentiated chest pain and assess if ML models show improved performance when compared to physicians or current risk stratification techniques.

METHODS AND FINDINGS

We conducted a systematic review of journal articles that applied a ML technique to an adult patient presenting to an emergency department with undifferentiated chest pain. Multiple databases were searched from inception through to November 2020. In total, 3361 articles were screened, and 23 articles were included. We did not conduct a metanalysis due to a high level of heterogeneity between studies in both their methods, and reporting. The most common primary outcomes assessed were diagnosis of acute myocardial infarction (AMI) (12 studies), and prognosis of major adverse cardiovascular event (MACE) (6 studies). There were 14 retrospective studies and 5 prospective studies. Four studies reported the development of a machine learning model retrospectively then tested it prospectively. The most common machine learning methods used were artificial neural networks (14 studies), random forest (6 studies), support vector machine (5 studies), and gradient boosting (2 studies). Multiple studies achieved high accuracy in both the diagnosis of AMI in the ED setting, and in predicting mortality and composite outcomes over various timeframes. ML outperformed existing risk stratification scores in all cases, and physicians in three out of four cases. The majority of studies were single centre, retrospective, and without prospective or external validation. There were only 3 studies that were considered low risk of bias and had low applicability concerns. Two studies reported integrating the ML model into clinical practice.

CONCLUSIONS

Research on applications of ML for undifferentiated chest pain in the ED has been ongoing for decades. ML has been reported to outperform emergency physicians and current risk stratification tools to diagnose AMI and predict MACE but has rarely been integrated into practice. Many studies assessing the use of ML in undifferentiated chest pain in the ED have a high risk of bias. It is important that future studies make use of recently developed standardised ML reporting guidelines, register their protocols, and share their datasets and code. Future work is required to assess the impact of ML model implementation on clinical decision making, patient orientated outcomes, and patient and physician acceptability.

TRIAL REGISTRATION

International Prospective Register of Systematic Reviews registration number: CRD42020184977.

Development and Assessment of an Interpretable Machine Learning Triage Tool for Estimating Mortality After Emergency Admissions

IMPORTANCE

Triage in the emergency department (ED) is a complex clinical judgment based on the tacit understanding of the patient's likelihood of survival, availability of medical resources, and local practices. Although a scoring tool could be valuable in risk stratification, currently available scores have demonstrated limitations.

OBJECTIVES

To develop an interpretable machine learning tool based on a parsimonious list of variables available at ED triage; provide a simple, early, and accurate estimate of patients' risk of death; and evaluate the tool's predictive accuracy compared with several established clinical scores.

DESIGN, SETTING, AND PARTICIPANTS

This single-site, retrospective cohort study assessed all ED patients between January 1, 2009, and December 31, 2016, who were subsequently admitted to a tertiary hospital in Singapore. The Score for Emergency Risk Prediction (SERP) tool was derived using a machine learning framework. To estimate mortality outcomes after emergency admissions, SERP was compared with several triage systems, including Patient Acuity Category Scale, Modified Early Warning Score, National Early Warning Score, Cardiac Arrest Risk Triage, Rapid Acute Physiology Score, and Rapid Emergency Medicine Score. The initial analyses were completed in October 2020, and additional analyses were conducted in May 2021.

MAIN OUTCOMES AND MEASURES

Three SERP scores, namely SERP-2d, SERP-7d, and SERP-30d, were developed using the primary outcomes of interest of 2-, 7-, and 30-day mortality, respectively. Secondary outcomes included 3-day mortality and inpatient mortality. The SERP's predictive power was measured using the area under the curve in the receiver operating characteristic analysis.

RESULTS

The study included 224 666 ED episodes in the model training cohort (mean [SD] patient age, 63.60 [16.90] years; 113 426 [50.5%] female), 56 167 episodes in the validation cohort (mean [SD] patient age, 63.58 [16.87] years; 28 427 [50.6%] female), and 42 676 episodes in the testing cohort (mean [SD] patient age, 64.85 [16.80] years; 21 556 [50.5%] female). The mortality rates in the training cohort were 0.8% at 2 days, 2.2% at 7 days, and 5.9% at 30 days. In the testing cohort, the areas under the curve of SERP-30d were 0.821 (95% CI, 0.796-0.847) for 2-day mortality, 0.826 (95% CI, 0.811-0.841) for 7-day mortality, and 0.823 (95% CI, 0.814-0.832) for 30-day mortality and outperformed several benchmark scores.

CONCLUSIONS AND RELEVANCE

In this retrospective cohort study, SERP had better prediction performance than existing triage scores while maintaining easy implementation and ease of ascertainment in the ED. It has the potential to be widely applied and validated in different circumstances and health care settings.

Predicting outcomes in older ED patients with influenza in real time using a big data-driven and machine learning approach to the hospital information system

BACKGROUND

Predicting outcomes in older patients with influenza in the emergency department (ED) by machine learning (ML) has never been implemented. Therefore, we conducted this study to clarify the clinical utility of implementing ML.

METHODS

We recruited 5508 older ED patients (≥65 years old) in three hospitals between 2009 and 2018. Patients were randomized into a 70%/30% split for model training and testing. Using 10 clinical variables from their electronic health records, a prediction model using the synthetic minority oversampling technique preprocessing algorithm was constructed to predict five outcomes.

RESULTS

The best areas under the curves of predicting outcomes were: random forest model for hospitalization (0.840), pneumonia (0.765), and sepsis or septic shock (0.857), XGBoost for intensive care unit admission (0.902), and logistic regression for in-hospital mortality (0.889) in the testing data. The predictive model was further applied in the hospital information system to assist physicians' decisions in real time.

CONCLUSIONS

ML is a promising way to assist physicians in predicting outcomes in older ED patients with influenza in real time. Evaluations of the effectiveness and impact are needed in the future.

Utilizing machine learning dimensionality reduction for risk stratification of chest pain patients in the emergency department

BACKGROUND

Chest pain is among the most common presenting complaints in the emergency department (ED). Swift and accurate risk stratification of chest pain patients in the ED may improve patient outcomes and reduce unnecessary costs. Traditional logistic regression with stepwise variable selection has been used to build risk prediction models for ED chest pain patients. In this study, we aimed to investigate if machine learning dimensionality reduction methods can improve performance in deriving risk stratification models.

METHODS

A retrospective analysis was conducted on the data of patients > 20 years old who presented to the ED of Singapore General Hospital with chest pain between September 2010 and July 2015. Variables used included demographics, medical history, laboratory findings, heart rate variability (HRV), and heart rate n-variability (HRnV) parameters calculated from five to six-minute electrocardiograms (ECGs). The primary outcome was 30-day major adverse cardiac events (MACE), which included death, acute myocardial infarction, and revascularization within 30 days of ED presentation. We used eight machine learning dimensionality reduction methods and logistic regression to create different prediction models. We further excluded cardiac troponin from candidate variables and derived a separate set of models to evaluate the performance of models without using laboratory tests. Receiver operating characteristic (ROC) and calibration analysis was used to compare model performance.

RESULTS

Seven hundred ninety-five patients were included in the analysis, of which 247 (31%) met the primary outcome of 30-day MACE. Patients with MACE were older and more likely to be male. All eight dimensionality reduction methods achieved comparable performance with the traditional stepwise variable selection; The multidimensional scaling algorithm performed the best with an area under the curve of 0.901. All prediction models generated in this study outperformed several existing clinical scores in ROC analysis.

CONCLUSIONS

Dimensionality reduction models showed marginal value in improving the prediction of 30-day MACE for ED chest pain patients. Moreover, they are black box models, making them difficult to explain and interpret in clinical practice.

Machine Learning-Based Early Warning Systems for Clinical Deterioration: Systematic Scoping Review

BACKGROUND

Timely identification of patients at a high risk of clinical deterioration is key to prioritizing care, allocating resources effectively, and preventing adverse outcomes. Vital signs-based, aggregate-weighted early warning systems are commonly used to predict the risk of outcomes related to cardiorespiratory instability and sepsis, which are strong predictors of poor outcomes and mortality. Machine learning models, which can incorporate trends and capture relationships among parameters that aggregate-weighted models cannot, have recently been showing promising results.

OBJECTIVE

This study aimed to identify, summarize, and evaluate the available research, current state of utility, and challenges with machine learning-based early warning systems using vital signs to predict the risk of physiological deterioration in acutely ill patients, across acute and ambulatory care settings.

METHODS

PubMed, CINAHL, Cochrane Library, Web of Science, Embase, and Google Scholar were searched for peer-reviewed, original studies with keywords related to "vital signs," "clinical deterioration," and "machine learning." Included studies used patient vital signs along with demographics and described a machine learning model for predicting an outcome in acute and ambulatory care settings. Data were extracted following PRISMA, TRIPOD, and Cochrane Collaboration guidelines.

RESULTS

We identified 24 peer-reviewed studies from 417 articles for inclusion; 23 studies were retrospective, while 1 was prospective in nature. Care settings included general wards, intensive care units, emergency departments, step-down units, medical assessment units, postanesthetic wards, and home care. Machine learning models including logistic regression, tree-based methods, kernel-based methods, and neural networks were most commonly used to predict the risk of deterioration. The area under the curve for models ranged from 0.57 to 0.97.

CONCLUSIONS

In studies that compared performance, reported results suggest that machine learning-based early warning systems can achieve greater accuracy than aggregate-weighted early warning systems but several areas for further research were identified. While these models have the potential to provide clinical decision support, there is a need for standardized outcome measures to allow for rigorous evaluation of performance across models. Further research needs to address the interpretability of model outputs by clinicians, clinical efficacy of these systems through prospective study design, and their potential impact in different clinical settings.

Machine Learning Versus Usual Care for Diagnostic and Prognostic Prediction in the Emergency Department: A Systematic Review

OBJECTIVE

Having shown promise in other medical fields, we sought to determine whether machine learning (ML) models perform better than usual care in diagnostic and prognostic prediction for emergency department (ED) patients.

METHODS

In this systematic review, we searched MEDLINE, Embase, Central, and CINAHL from inception to October 17, 2019. We included studies comparing diagnostic and prognostic prediction of ED patients by ML models to usual care methods (triage-based scores, clinical prediction tools, clinician judgment) using predictor variables readily available to ED clinicians. We extracted commonly reported performance metrics of model discrimination and classification. We used the PROBAST tool for risk of bias assessment (PROSPERO registration: CRD42020158129).

RESULTS

The search yielded 1,656 unique records, of which 23 studies involving 16,274,647 patients were included. In all seven diagnostic studies, ML models outperformed usual care in all performance metrics. In six studies assessing in-hospital mortality, the best-performing ML models had better discrimination (area under the receiver operating characteristic curve [AUROC] =0.74-0.94) than any clinical decision tool (AUROC =0.68-0.81). In four studies assessing hospitalization, ML models had better discrimination (AUROC =0.80-0.83) than triage-based scores (AUROC =0.68-0.82). Clinical heterogeneity precluded meta-analysis. Most studies had high risk of bias due to lack of external validation, low event rates, and insufficient reporting of calibration.

CONCLUSIONS

Our review suggests that ML may have better prediction performance than usual care for ED patients with a variety of clinical presentations and outcomes. However, prediction model reporting guidelines should be followed to provide clinically applicable data. Interventional trials are needed to assess the impact of ML models on patient-centered outcomes.

AutoScore: A Machine Learning-Based Automatic Clinical Score Generator and Its Application to Mortality Prediction Using Electronic Health Records

BACKGROUND

Risk scores can be useful in clinical risk stratification and accurate allocations of medical resources, helping health providers improve patient care. Point-based scores are more understandable and explainable than other complex models and are now widely used in clinical decision making. However, the development of the risk scoring model is nontrivial and has not yet been systematically presented, with few studies investigating methods of clinical score generation using electronic health records.

OBJECTIVE

This study aims to propose AutoScore, a machine learning-based automatic clinical score generator consisting of 6 modules for developing interpretable point-based scores. Future users can employ the AutoScore framework to create clinical scores effortlessly in various clinical applications.

METHODS

We proposed the AutoScore framework comprising 6 modules that included variable ranking, variable transformation, score derivation, model selection, score fine-tuning, and model evaluation. To demonstrate the performance of AutoScore, we used data from the Beth Israel Deaconess Medical Center to build a scoring model for mortality prediction and then compared the data with other baseline models using the receiver operating characteristic analysis. A software package in R 3.5.3 (R Foundation) was also developed to demonstrate the implementation of AutoScore.

RESULTS

Implemented on the data set with 44,918 individual admission episodes of intensive care, the AutoScore-created scoring models performed comparably well as other standard methods (ie, logistic regression, stepwise regression, least absolute shrinkage and selection operator, and random forest) in terms of predictive accuracy and model calibration but required fewer predictors and presented high interpretability and accessibility. The nine-variable, AutoScore-created, point-based scoring model achieved an area under the curve (AUC) of 0.780 (95% CI 0.764-0.798), whereas the model of logistic regression with 24 variables had an AUC of 0.778 (95% CI 0.760-0.795). Moreover, the AutoScore framework also drives the clinical research continuum and automation with its integration of all necessary modules.

CONCLUSIONS

We developed an easy-to-use, machine learning-based automatic clinical score generator, AutoScore; systematically presented its structure; and demonstrated its superiority (predictive performance and interpretability) over other conventional methods using a benchmark database. AutoScore will emerge as a potential scoring tool in various medical applications.

Prediction of incident myocardial infarction using machine learning applied to harmonized electronic health record data

Background

With cardiovascular disease increasing, substantial research has focused on the development of prediction tools. We compare deep learning and machine learning models to a baseline logistic regression using only ‘known’ risk factors in predicting incident myocardial infarction (MI) from harmonized EHR data.

Methods

Large-scale case-control study with outcome of 6-month incident MI, conducted using the top 800, from an initial 52 k procedures, diagnoses, and medications within the UCHealth system, harmonized to the Observational Medical Outcomes Partnership common data model, performed on 2.27 million patients. We compared several over- and under- sampling techniques to address the imbalance in the dataset. We compared regularized logistics regression, random forest, boosted gradient machines, and shallow and deep neural networks. A baseline model for comparison was a logistic regression using a limited set of ‘known’ risk factors for MI. Hyper-parameters were identified using 10-fold cross-validation.

Results

Twenty thousand Five hundred and ninety-one patients were diagnosed with MI compared with 2.25 million who did not. A deep neural network with random undersampling provided superior classification compared with other methods. However, the benefit of the deep neural network was only moderate, showing an F1 Score of 0.092 and AUC of 0.835, compared to a logistic regression model using only ‘known’ risk factors. Calibration for all models was poor despite adequate discrimination, due to overfitting from low frequency of the event of interest.

Conclusions

Our study suggests that DNN may not offer substantial benefit when trained on harmonized data, compared to traditional methods using established risk factors for MI.

Real-time AI prediction for major adverse cardiac events in emergency department patients with chest pain

Background

A big-data-driven and artificial intelligence (AI) with machine learning (ML) approach has never been integrated with the hospital information system (HIS) for predicting major adverse cardiac events (MACE) in patients with chest pain in the emergency department (ED). Therefore, we conducted the present study to clarify it.

Methods

In total, 85,254 ED patients with chest pain in three hospitals between 2009 and 2018 were identified. We randomized the patients into a 70%/30% split for ML model training and testing. We used 14 clinical variables from their electronic health records to construct a random forest model with the synthetic minority oversampling technique preprocessing algorithm to predict acute myocardial infarction (AMI) < 1 month and all-cause mortality < 1 month. Comparisons of the predictive accuracies among random forest, logistic regression, support-vector clustering (SVC), and K-nearest neighbor (KNN) models were also performed.

Results

Predicting MACE using the random forest model produced areas under the curves (AUC) of 0.915 for AMI < 1 month and 0.999 for all-cause mortality < 1 month. The random forest model had better predictive accuracy than logistic regression, SVC, and KNN. We further integrated the AI prediction model with the HIS to assist physicians with decision-making in real time. Validation of the AI prediction model by new patients showed AUCs of 0.907 for AMI < 1 month and 0.888 for all-cause mortality < 1 month.

Conclusions

An AI real-time prediction model is a promising method for assisting physicians in predicting MACE in ED patients with chest pain. Further studies to evaluate the impact on clinical practice are warranted.

Prediction of breakthrough pain during labour neuraxial analgesia: comparison of machine learning and multivariable regression approaches

INTRODUCTION

Risk-prediction models for breakthrough pain facilitate interventions to forestall inadequate labour analgesia, but limited work has used machine learning to identify predictive factors. We compared the performance of machine learning and regression techniques in identifying parturients at increased risk of breakthrough pain during labour epidural analgesia.

METHODS

A single-centre retrospective study involved parturients receiving patient-controlled epidural analgesia. The primary outcome was breakthrough pain. We randomly selected 80% of the cohort (training cohort) to develop three prediction models using random forest, XGBoost, and logistic regression, followed by validation against the remaining 20% of the cohort (validation cohort). Area-under-the-receiver operating characteristic curve (AUC), sensitivity, specificity, and positive and negative predictive values (PPV and NPV) were used to assess model performance.

RESULTS

Data from 20 716 parturients were analysed. The incidence of breakthrough pain was 14.2%. Of 31 candidate variables, random forest, XGBoost and logistic regression models included 30, 23, and 15 variables, respectively. Unintended venous puncture, post-neuraxial analgesia highest pain score, number of dinoprostone suppositories, neuraxial technique, number of neuraxial attempts, depth to epidural space, body mass index, pre-neuraxial analgesia oxytocin infusion rate, maternal age, pre-neuraxial analgesia cervical dilation, anaesthesiologist rank, and multiparity, were identified in all three models. All three models performed similarly, with AUC 0.763-0.772, sensitivity 67.0-69.4%, specificity 70.9-76.2%, PPV 28.3-31.8%, and NPV 93.3-93.5%.

CONCLUSIONS

Machine learning did not improve the prediction of breakthrough pain compared with multivariable regression. Larger population-wide studies are needed to improve predictive ability.

Heart rate n-variability (HRnV) and its application to risk stratification of chest pain patients in the emergency department

Background

Chest pain is one of the most common complaints among patients presenting to the emergency department (ED). Causes of chest pain can be benign or life threatening, making accurate risk stratification a critical issue in the ED. In addition to the use of established clinical scores, prior studies have attempted to create predictive models with heart rate variability (HRV). In this study, we proposed heart rate n-variability (HRnV), an alternative representation of beat-to-beat variation in electrocardiogram (ECG), and investigated its association with major adverse cardiac events (MACE) in ED patients with chest pain.

Methods

We conducted a retrospective analysis of data collected from the ED of a tertiary hospital in Singapore between September 2010 and July 2015. Patients > 20 years old who presented to the ED with chief complaint of chest pain were conveniently recruited. Five to six-minute single-lead ECGs, demographics, medical history, troponin, and other required variables were collected. We developed the HRnV-Calc software to calculate HRnV parameters. The primary outcome was 30-day MACE, which included all-cause death, acute myocardial infarction, and revascularization. Univariable and multivariable logistic regression analyses were conducted to investigate the association between individual risk factors and the outcome. Receiver operating characteristic (ROC) analysis was performed to compare the HRnV model (based on leave-one-out cross-validation) against other clinical scores in predicting 30-day MACE.

Results

A total of 795 patients were included in the analysis, of which 247 (31%) had MACE within 30 days. The MACE group was older, with a higher proportion being male patients. Twenty-one conventional HRV and 115 HRnV parameters were calculated. In univariable analysis, eleven HRV and 48 HRnV parameters were significantly associated with 30-day MACE. The multivariable stepwise logistic regression identified 16 predictors that were strongly associated with MACE outcome; these predictors consisted of one HRV, seven HRnV parameters, troponin, ST segment changes, and several other factors. The HRnV model outperformed several clinical scores in the ROC analysis.

Conclusions

The novel HRnV representation demonstrated its value of augmenting HRV and traditional risk factors in designing a robust risk stratification tool for patients with chest pain in the ED.

El Kassabi H, Ismail H, Nujum Navaz A. ECG Monitoring Systems: Review, Architecture, Processes, and Key Challenges

Health monitoring and its related technologies is an attractive research area. The electrocardiogram (ECG) has always been a popular measurement scheme to assess and diagnose cardiovascular diseases (CVDs). The number of ECG monitoring systems in the literature is expanding exponentially. Hence, it is very hard for researchers and healthcare experts to choose, compare, and evaluate systems that serve their needs and fulfill the monitoring requirements. This accentuates the need for a verified reference guiding the design, classification, and analysis of ECG monitoring systems, serving both researchers and professionals in the field. In this paper, we propose a comprehensive, expert-verified taxonomy of ECG monitoring systems and conduct an extensive, systematic review of the literature. This provides evidence-based support for critically understanding ECG monitoring systems' components, contexts, features, and challenges. Hence, a generic architectural model for ECG monitoring systems is proposed, an extensive analysis of ECG monitoring systems' value chain is conducted, and a thorough review of the relevant literature, classified against the experts' taxonomy, is presented, highlighting challenges and current trends. Finally, we identify key challenges and emphasize the importance of smart monitoring systems that leverage new technologies, including deep learning, artificial intelligence (AI), Big Data and Internet of Things (IoT), to provide efficient, cost-aware, and fully connected monitoring systems.

Developing and validating models for predicting nursing home admission using only RAI-HC instrument data

OBJECTIVE

In recent years research has identified important predictors for nursing home admission (NHA). However, as far as we know, the previous risk models use complex variable sets from many sources and the output is a single risk value. The objective of this study was to develop an NHA risk model with a variable set from single data source and richer output information.

METHODS

In this study, we developed a model selecting variables only from the RAI-HC (Resident Assessment Instrument - Home Care) system. Furthermore, we used principal component analysis and K-means clustering to target proper interventions for high-risk clients.

RESULTS

The performance of the model was close to the complex previous model (recall [Formula: see text] vs. [Formula: see text] and specificity [Formula: see text] vs. [Formula: see text]). For the risk clients, three intervention clusters (deficiency in physical functionality, deficiency in cognitive functionality and depression and mood disorders) were found.

CONCLUSION

The NHA risk model and intervention clusters are important because they enable the identification of proper interventions for the right clients. The fact that the model with RAI-HC data alone was accurate enough simplifies the integration of the NHA risk model into practice because it uses data from one system and the algorithm can be integrated easily into the source system.

Prediction of good neurological recovery after out-of-hospital cardiac arrest: A machine learning analysis

BACKGROUND

This study aimed to train, validate and compare predictive models that use machine learning analysis for good neurological recovery in OHCA patients.

METHODS

Adult OHCA patients who had a presumed cardiac etiology and a sustained return of spontaneous circulation between 2013 and 2016 were analyzed; 80% of the individuals were analyzed for training and 20% were analyzed for validation. We developed using six machine learning algorithms: logistic regression (LR), extreme gradient boosting (XGB), support vector machine, random forest, elastic net (EN), and neural network. Variables that could be obtained within 24 hours of the emergency department visit were used. The area under the receiver operation curve (AUROC) was calculated to assess the discrimination. Calibration was assessed by the Hosmer-Lemeshow test. Reclassification was assessed by using the continuous net reclassification index (NRI).

RESULTS

A total of 19,860 OHCA patients were included in the analysis. Of the 15,888 patients in the training group, 2228 (14.0%) had a good neurological recovery; of the 3972 patients in the validation group, 577 (14.5%) had a good neurological recovery. The LR, XGB, and EN models showed the highest discrimination powers (AUROC (95% CI)) of 0.949 (0.941-0.957) for all), and all three models were well calibrated (Hosmer-Lemeshow test: p >0.05). The XGB model reclassified patients according to their true risk better than the LR model (NRI: 0.110), but the EN model reclassified patients worse than the LR model (NRI: -1.239).

CONCLUSION

The best performing machine learning algorithm was the XGB and LR algorithm.

Advanced diagnostic imaging utilization during emergency department visits in the United States: A predictive modeling study for emergency department triage

Background

Emergency department (ED) crowding is associated with negative health outcomes, patient dissatisfaction, and longer length of stay (LOS). The addition of advanced diagnostic imaging (ADI), namely CT, ultrasound (U/S), and MRI to ED encounter work up is a predictor of longer length of stay. Earlier and improved prediction of patients’ need for advanced imaging may improve overall ED efficiency. The aim of the study was to detect the association between ADI utilization and the structured and unstructured information immediately available during ED triage, and to develop and validate models to predict utilization of ADI during an ED encounter.

Methods

We used the United States National Hospital Ambulatory Medical Care Survey data from 2009 to 2014 to examine which sociodemographic and clinical factors immediately available at ED triage were associated with the utilization of CT, U/S, MRI, and multiple ADI during a patient’s ED stay. We used natural language processing (NLP) topic modeling to incorporate free-text reason for visit data available at time of ED triage in addition to other structured patient data to predict the use of ADI using multivariable logistic regression models.

Results

Among the 139,150 adult ED visits from a national probability sample of hospitals across the U.S, 21.9% resulted in ADI use, including 16.8% who had a CT, 3.6% who had an ultrasound, 0.4% who had an MRI, and 1.2% of the population who had multiple types of ADI. The c-statistic of the predictive models was greater than or equal to 0.78 for all imaging outcomes, and the addition of text-based reason for visit information improved the accuracy of all predictive models.

Conclusions

Patient information immediately available during ED triage can accurately predict the eventual use of advanced diagnostic imaging during an ED visit. Such models have the potential to be incorporated into the ED triage workflow in order to more rapidly identify patients who may require advanced imaging during their ED stay and assist with medical decision-making.

Developing neural network models for early detection of cardiac arrest in emergency department

BACKGROUND

Automated surveillance for cardiac arrests would be useful in overcrowded emergency departments. The purpose of this study is to develop and test artificial neural network (ANN) classifiers for early detection of patients at risk of cardiac arrest in emergency departments.

METHODS

This is a single-center electronic health record (EHR)-based study. The primary outcome was the development of cardiac arrest within 24 h after prediction. Three ANN models were trained: multilayer perceptron (MLP), long-short-term memory (LSTM), and hybrid. These were compared to other classifiers including the modified early warning score (MEWS), logistic regression, and random forest. We used AUROC, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for the comparison.

RESULTS

During the study period, there were a total of 374,605 ED visits and 2,910,321 patient status updates. The ANN models (MLP, LSTM, and hybrid) achieved higher AUROC (AUROC: 0.929, 0.933, and 0.936; 95% confidential interval: 0.926-0.932, 0.930-0.936, and 0.933-0.939, respectively) compared to the non-ANN models, and the hybrid model exhibited the best performance. The ANN classifiers displayed higher performance in most of the test characteristics when the threshold levels of the classifiers were fixed to display the same positive result as those at the three MEWS thresholds (score ≥ 3, ≥4, and ≥5), and when compared with each other.

CONCLUSIONS

The ANN improves upon MEWS and conventional machine learning algorithms for the prediction of cardiac arrests in emergency departments. The hybrid ANN model utilizing both baseline and sequence information achieved the best performance.

An exhaustive review and analysis on applications of statistical forecasting in hospital emergency departments

Emergency departments (EDs) provide medical treatment for a broad spectrum of illnesses and injuries to patients who arrive at all hours of the day. The quality and efficient delivery of health care in EDs are associated with a number of factors, such as patient overall length of stay (LOS) and admission, prompt ambulance diversion, quick and accurate triage, nurse and physician assessment, diagnostic and laboratory services, consultations and treatment. One of the most important ways to plan the healthcare delivery efficiently is to make forecasts of ED processes. The aim this study is thus to provide an exhaustive review for ED stakeholders interested in applying forecasting methods to their ED processes. A categorisation, analysis and interpretation of 102 papers is performed for review. This exhaustive review provides an insight for researchers and practitioners about forecasting in EDs in terms of showing current state and potential areas for future attempts.

Validation of deep-learning-based triage and acuity score using a large national dataset

AIM

Triage is important in identifying high-risk patients amongst many less urgent patients as emergency department (ED) overcrowding has become a national crisis recently. This study aims to validate that a Deep-learning-based Triage and Acuity Score (DTAS) identifies high-risk patients more accurately than existing triage and acuity scores using a large national dataset.

METHODS

We conducted a retrospective observational cohort study using data from the Korean National Emergency Department Information System (NEDIS), which collected data on visits in real time from 151 EDs. The NEDIS data was split into derivation data (January 2014-June 2016) and validation data (July-December 2016). We also used data from the Sejong General Hospital (SGH) for external validation (January-December 2017). We predicted in-hospital mortality, critical care, and hospitalization using initial information of ED patients (age, sex, chief complaint, time from symptom onset to ED visit, arrival mode, trauma, initial vital signs and mental status as predictor variables).

RESULTS

A total of 11,656,559 patients were included in this study. The primary outcome was in-hospital mortality. The Area Under the Receiver Operating Characteristic curve (AUROC) and Area Under the Precision and Recall Curve (AUPRC) of DTAS were 0.935 and 0.264. It significantly outperformed Korean triage and acuity score (AUROC:0.785, AUPRC:0.192), modified early warning score (AUROC:0.810, AUPRC:0.116), logistic regression (AUROC:0.903, AUPRC:0.209), and random forest (AUROC:0.910, AUPRC:0.179).

CONCLUSION

Deep-learning-based Triage and Acuity Score predicted in-hospital mortality, critical care, and hospitalization more accurately than existing triages and acuity, and it was validated using a large, multicenter dataset.

Root traits of European Vicia faba cultivars-Using machine learning to explore adaptations to agroclimatic conditions

Faba bean (Vicia faba L.) is an important source of protein, but breeding for increased yield stability and stress tolerance is hampered by the scarcity of phenotyping information. Because comparisons of cultivars adapted to different agroclimatic zones improve our understanding of stress tolerance mechanisms, the root architecture and morphology of 16 European faba bean cultivars were studied at maturity. Different machine learning (ML) approaches were tested in their usefulness to analyse trait variations between cultivars. A supervised, that is, hypothesis-driven, ML approach revealed that cultivars from Portugal feature greater and coarser but less frequent lateral roots at the top of the taproot, potentially enhancing water uptake from deeper soil horizons. Unsupervised clustering revealed that trait differences between northern and southern cultivars are not predominant but that two cultivar groups, independently from major and minor types, differ largely in overall root system size. Methodological guidelines on how to use powerful ML methods such as random forest models for enhancing the phenotypical exploration of plants are given.

Evaluation of the efficacy of the National Early Warning Score in predicting in-hospital mortality via the risk stratification

PURPOSE

To investigate the efficacy of the National Early Warning Score (NEWS) in predicting in-hospital mortality.

MATERIALS AND METHODS

This was a retrospective observational study and the electronic medical records of the patients were reviewed based on NEWS at the time of admission.

RESULTS

The performance of NEWS was effective in predicting hospital mortality (area under the curve: 0.765; 95% confidence interval: 0.659-0.846). Based on the Kaplan Meier survival curves, the survival time of patients who are at high risk according to NEWS was significantly shorter than that of patients who are at low risk (p < 0.001). Results of the multivariate Cox proportional hazards regression analysis showed that the hazard ratios of patients who are at medium and high risk based on NEWS were 2.6 and 4.7, respectively (p < 0.001). In addition, our study showed that the combination model that used other factors, such as age and diagnosis, was more effective than NEWS alone in predicting hospital mortality (NEWS: 0.765; combination model: 0.861; p < 0.005).

CONCLUSIONS

NEWS is a simple and useful bedside tool for predicting in-hospital mortality. In addition, the rapid response team must consider other clinical factors as well as screening tools to improve clinical outcomes.

Development and validation of classifiers and variable subsets for predicting nursing home admission

Background

In previous years a substantial number of studies have identified statistically important predictors of nursing home admission (NHA). However, as far as we know, the analyses have been done at the population-level. No prior research has analysed the prediction accuracy of a NHA model for individuals.

Methods

This study is an analysis of 3056 longer-term home care customers in the city of Tampere, Finland. Data were collected from the records of social and health service usage and RAI-HC (Resident Assessment Instrument - Home Care) assessment system during January 2011 and September 2015. The aim was to find out the most efficient variable subsets to predict NHA for individuals and validate the accuracy. The variable subsets of predicting NHA were searched by sequential forward selection (SFS) method, a variable ranking metric and the classifiers of logistic regression (LR), support vector machine (SVM) and Gaussian naive Bayes (GNB). The validation of the results was guaranteed using randomly balanced data sets and cross-validation. The primary performance metrics for the classifiers were the prediction accuracy and AUC (average area under the curve).

Results

The LR and GNB classifiers achieved 78% accuracy for predicting NHA. The most important variables were RAI MAPLE (Method for Assigning Priority Levels), functional impairment (RAI IADL, Activities of Daily Living), cognitive impairment (RAI CPS, Cognitive Performance Scale), memory disorders (diagnoses G30-G32 and F00-F03) and the use of community-based health-service and prior hospital use (emergency visits and periods of care).

Conclusion

The accuracy of the classifier for individuals was high enough to convince the officials of the city of Tampere to integrate the predictive model based on the findings of this study as a part of home care information system. Further work need to be done to evaluate variables that are modifiable and responsive to interventions.

Phenotyping: Using Machine Learning for Improved Pairwise Genotype Classification Based on Root Traits

Phenotyping local crop cultivars is becoming more and more important, as they are an important genetic source for breeding - especially in regard to inherent root system architectures. Machine learning algorithms are promising tools to assist in the analysis of complex data sets; novel approaches are need to apply them on root phenotyping data of mature plants. A greenhouse experiment was conducted in large, sand-filled columns to differentiate 16 European Pisum sativum cultivars based on 36 manually derived root traits. Through combining random forest and support vector machine models, machine learning algorithms were successfully used for unbiased identification of most distinguishing root traits and subsequent pairwise cultivar differentiation. Up to 86% of pea cultivar pairs could be distinguished based on top five important root traits (Timp5) - Timp5 differed widely between cultivar pairs. Selecting top important root traits (Timp) provided a significant improved classification compared to using all available traits or randomly selected trait sets. The most frequent Timp of mature pea cultivars was total surface area of lateral roots originating from tap root segments at 0-5 cm depth. The high classification rate implies that culturing did not lead to a major loss of variability in root system architecture in the studied pea cultivars. Our results illustrate the potential of machine learning approaches for unbiased (root) trait selection and cultivar classification based on rather small, complex phenotypic data sets derived from pot experiments. Powerful statistical approaches are essential to make use of the increasing amount of (root) phenotyping information, integrating the complex trait sets describing crop cultivars.