Predicting hospital admission for older emergency department patients: Insights from machine learning

A meta-analysis of the diagnostic test accuracy of artificial intelligence predicting emergency department dispositions

BACKGROUND

The rapid advancement of Artificial Intelligence (AI) has led to its widespread application across various domains, showing encouraging outcomes. Many studies have utilized AI to forecast emergency department (ED) disposition, aiming to forecast patient outcomes earlier and to allocate resources better; however, a dearth of comprehensive review literature exists to assess the objective performance standards of these predictive models using quantitative evaluations. This study aims to conduct a meta-analysis to assess the diagnostic accuracy of AI in predicting ED disposition, encompassing admission, critical care, and mortality.

METHODS

Multiple databases, including Scopus, Springer, ScienceDirect, PubMed, Wiley, Sage, and Google Scholar, were searched until December 31, 2023, to gather relevant literature. Risk of bias was assessed using the Prediction Model Risk of Bias Assessment Tool. Pooled estimates of sensitivity, specificity, and area under the receiver operating characteristic curve (AUROC) were calculated to evaluate AI's predictive performance. Sub-group analyses were performed to explore covariates affecting AI predictive model performance.

RESULTS

The study included 88 articles possessed with 117 AI models, among which 39, 45, and 33 models predicted admission, critical care, and mortality, respectively. The reported statistics for sensitivity, specificity, and AUROC represent pooled summary measures derived from the component studies included in this meta-analysis. AI's summary sensitivity, specificity, and AUROC for predicting admission were 0.81 (95% Confidence Interval [CI] 0.74-0.86), 0.87 (95% CI 0.81-0.91), and 0.87 (95% CI 0.84-0.93), respectively. For critical care, the values were 0.86 (95% CI 0.79-0.91), 0.89 (95% CI 0.83-0.93), and 0.93 (95% CI 0.89-0.95), respectively, and for mortality, they were 0.85 (95% CI 0.80-0.89), 0.94 (95% CI 0.90-0.96), and 0.93 (95% CI 0.89-0.96), respectively. Emergent sample characteristics and AI techniques showed evidence of significant covariates influencing the heterogeneity of AI predictive models for ED disposition.

CONCLUSIONS

The meta-analysis indicates promising performance of AI in predicting ED disposition, with certain potential for improvement, especially in sensitivity. Future research could explore advanced AI techniques such as ensemble learning and cross-validation with hyper-parameter tuning to enhance predictive model efficacy.

TRIAL REGISTRATION

This systematic review was not registered with PROSPERO or any other similar registry because the review was completed prior to the opportunity for registration, and PROSPERO currently does not accept registrations for reviews that are already completed. We are committed to transparency and have adhered to best practices in systematic review methodology throughout this study.

Effect of Home Care on Physical Function in Post-Intensive Care Unit Patients: A Meta-Analysis

BACKGROUND

A decline in physical function is commonly observed after patients transition to their homes following hospital admission; this is especially true for patients requiring mechanical ventilation in an intensive care unit (ICU).

OBJECTIVE

This meta-analysis examines characteristics and effects of home-based or outpatient+home-based interventions used to improve physical function post-discharge in patients who received mechanical ventilation in an ICU.

METHODS

PRISMA guidelines were utilized. The literature search was conducted with the assistance of a medical librarian. Study inclusion criteria were post-ICU adult patients receiving mechanical ventilation who then had home-based or outpatient+home-based care to improve physical function after discharge. Effect size (Hedges' g) was calculated with random effects models.

RESULTS

Our search yielded 11 studies that met the inclusion criteria. The majority were randomized controlled trials, with 1 quasi-experimental study. All studies included physical therapists, and 2 included nurses. The 11 studies reported results for 39 physical function measurements. The overall pooled intervention effect across the 4 studies that utilized the 6-minute walk test was 0.32 (95% confidence intervals [CI]: 0.05 to 0.58), for the 3 studies that utilized the Timed Up and Go test it was 1.38 (95% CI: -0.09 to 2.84), and for the 8 studies that used the SF-36 Physical Function subscale, it was 0.31 (95% CI: 0.09 to 0.52).

CONCLUSIONS

This review's findings show that patients may improve their physical function after participating in specific intervention programs that are home-based alone or outpatient+home-based care. However, the effect sizes are small, so it may be useful to explore how to maximize the gains in physical function.

Using machine learning and natural language processing in triage for prediction of clinical disposition in the emergency department

BACKGROUND

Accurate triage is required for efficient allocation of resources and to decrease patients' length of stay. Triage decisions are often subjective and vary by provider, leading to patients being over-triaged or under-triaged. This study developed machine learning models that incorporated natural language processing (NLP) to predict patient disposition. The models were assessed by comparing their performance with the judgements of emergency physicians (EPs).

METHOD

This retrospective study obtained data from patients visiting EDs between January 2018 and December 2019. Internal validation data came from China Medical University Hospital (CMUH), while external validation data were obtained from Asia University Hospital (AUH). Nontrauma patients aged ≥ 20 years were included. The models were trained using structured data and unstructured data (free-text notes) processed by NLP. The primary outcome was death in the ED or admission to the intensive care unit, and the secondary outcome was either admission to a general ward or transferal to another hospital. Six machine learning models (CatBoost, Light Gradient Boosting Machine, Logistic Regression, Random Forest, Extremely Randomized Trees, and Gradient Boosting) and one Logistic Regression derived from triage level were developed and evaluated using EPs' predictions as reference.

RESULT

A total of 17,2101 and 41,883 patients were enrolled from CMUH and AUH, respectively. EPs achieved F1 core of 0.361 and 0.498 for the primary and secondary outcomes, respectively. All machine learning models achieved higher F1 scores compared to EPs and Logistic Regression derived from triage level. Random Forest was selected for further evaluation and fine-tuning, because of its robust calibration and predictive performance. In internal validation, it achieved Brier scores of 0.072 and 0.089 for the primary and secondary outcomes, respectively, and 0.076 and 0.095 in external validation. Further analysis revealed that incorporating unstructured data significantly enhanced the model's performance. Threshold adjustments were applied to improve clinical applicability, aiming to balance the trade-off between sensitivity and positive predictive value.

CONCLUSION

This study developed and validated machine learning models that integrate structured and unstructured triage data to predict patient dispositions, distinguishing between general ward and critical conditions like ICU admissions and ED deaths. Integrating both structured and unstructured data significantly improved model performance.

Setting the balance of care for older adults at risk of hospitalization and delayed discharge: A mixed-methods research protocol

INTRODUCTION

Delayed hospital discharge is a persistent care quality issue experienced across health systems worldwide and remains a priority area to be addressed in Canada. Often associated with a decrease in services while waiting to leave the hospital, delayed discharge from hospital can lead to increased frailty, physical and cognitive decline, and caregiver burnout. Optimizing availability of and timely access to community-based health and social care are avenues that could reduce initial admissions to the hospital and length of hospital stay, and facilitate hospital discharges.

METHODS

This research will explore the ways in which community resources could be leveraged to potentially avoid hospitalization and delayed hospital discharge for older adults using sequential mixed-methods including co-design. To better understand the characteristics and needs of older adults, the research team will first identify sub-populations of older adults (65 years old or older) at risk of hospitalization and delayed discharge using comprehensive, longitudinal administrative health data. From these health data, risk profiles and personas will be created and then shared with key partners (e.g., older adults, caregivers, healthcare providers, healthcare decision-makers), who will be engaged to identify, leverage, and create targeted care solutions. The barriers and facilitators to the implementation of these care solutions will then be explored.

DISCUSSION

Delayed hospital discharge has been a critical care quality issue across Canada for decades. The current research will provide health system leaders with an approach to better allocate services to older adults in order to avoid delayed hospital discharge and identify gaps in health and social care resources based on the characteristics, needs, and preferences of older adults, their caregivers, and providers.

A study of "left against medical advice" emergency department patients: an optimized explainable artificial intelligence framework

The issue of left against medical advice (LAMA) patients is common in today's emergency departments (EDs). This issue represents a medico-legal risk and may result in potential readmission, mortality, or revenue loss. Thus, understanding the factors that cause patients to "leave against medical advice" is vital to mitigate and potentially eliminate these adverse outcomes. This paper proposes a framework for studying the factors that affect LAMA in EDs. The framework integrates machine learning, metaheuristic optimization, and model interpretation techniques. Metaheuristic optimization is used for hyperparameter optimization-one of the main challenges of machine learning model development. Adaptive tabu simulated annealing (ATSA) metaheuristic algorithm is utilized for optimizing the parameters of extreme gradient boosting (XGB). The optimized XGB models are used to predict the LAMA outcomes for patients under treatment in ED. The designed algorithms are trained and tested using four data groups which are created using feature selection. The model with the best predictive performance is then interpreted using the SHaply Additive exPlanations (SHAP) method. The results show that best model has an area under the curve (AUC) and sensitivity of 76% and 82%, respectively. The best model was explained using SHAP method.

Integrating Deep Learning with Electronic Health Records for Early Glaucoma Detection: A Multi-Dimensional Machine Learning Approach

BACKGROUND

Recent advancements in deep learning have significantly impacted ophthalmology, especially in glaucoma, a leading cause of irreversible blindness worldwide. In this study, we developed a reliable predictive model for glaucoma detection using deep learning models based on clinical data, social and behavior risk factor, and demographic data from 1652 participants, split evenly between 826 control subjects and 826 glaucoma patients.

METHODS

We extracted structural data from control and glaucoma patients' electronic health records (EHR). Three distinct machine learning classifiers, the Random Forest and Gradient Boosting algorithms, as well as the Sequential model from the Keras library of TensorFlow, were employed to conduct predictive analyses across our dataset. Key performance metrics such as accuracy, F1 score, precision, recall, and the area under the receiver operating characteristics curve (AUC) were computed to both train and optimize these models.

RESULTS

The Random Forest model achieved an accuracy of 67.5%, with a ROC AUC of 0.67, outperforming the Gradient Boosting and Sequential models, which registered accuracies of 66.3% and 64.5%, respectively. Our results highlighted key predictive factors such as intraocular pressure, family history, and body mass index, substantiating their roles in glaucoma risk assessment.

CONCLUSIONS

This study demonstrates the potential of utilizing readily available clinical, lifestyle, and demographic data from EHRs for glaucoma detection through deep learning models. While our model, using EHR data alone, has a lower accuracy compared to those incorporating imaging data, it still offers a promising avenue for early glaucoma risk assessment in primary care settings. The observed disparities in model performance and feature significance show the importance of tailoring detection strategies to individual patient characteristics, potentially leading to more effective and personalized glaucoma screening and intervention.

Developmental Prediction of Poststroke Patients in Activities of Daily Living by Using Tree-Structured Parzen Estimator-Optimized Stacking Ensemble Approaches

Poststroke injuries limit the daily activities of patients and cause considerable inconvenience. Therefore, predicting the activities of daily living (ADL) results of patients with stroke before hospital discharge can assist clinical workers in formulating more personalized and effective strategies for therapeutic intervention, and prepare hospital discharge plans that suit the patients needs. This study used the leave-one-out cross-validation procedure to evaluate the performance of the machine learning models. In addition, testing methods were used to identify the optimal weak learners, which were then combined to form a stacking model. Subsequently, a hyperparameter optimization algorithm was used to optimize the model hyperparameters. Finally, optimization algorithms were used to analyze each feature, and features of high importance were identified by limiting the number of features to be included in the machine learning models. After various features were fed into the learning models to predict the Barthel index (BI) at discharge, the results indicated that random forest (RF), adaptive boosting (AdaBoost), and multilayer perceptron (MLP) produced suitable results. The most critical prediction factor of this study was the BI at admission. Machine learning models can be used to assist clinical workers in predicting the ADL of patients with stroke at hospital discharge.

Evaluating methods for risk prediction of Covid-19 mortality in nursing home residents before and after vaccine availability: a retrospective cohort study

BACKGROUND

SARS-CoV-2 vaccines are effective in reducing hospitalization, COVID-19 symptoms, and COVID-19 mortality for nursing home (NH) residents. We sought to compare the accuracy of various machine learning models, examine changes to model performance, and identify resident characteristics that have the strongest associations with 30-day COVID-19 mortality, before and after vaccine availability.

METHODS

We conducted a population-based retrospective cohort study analyzing data from all NH facilities across Ontario, Canada. We included all residents diagnosed with SARS-CoV-2 and living in NHs between March 2020 and July 2021. We employed five machine learning algorithms to predict COVID-19 mortality, including logistic regression, LASSO regression, classification and regression trees (CART), random forests, and gradient boosted trees. The discriminative performance of the models was evaluated using the area under the receiver operating characteristic curve (AUC) for each model using 10-fold cross-validation. Model calibration was determined through evaluation of calibration slopes. Variable importance was calculated by repeatedly and randomly permutating the values of each predictor in the dataset and re-evaluating the model's performance.

RESULTS

A total of 14,977 NH residents and 20 resident characteristics were included in the model. The cross-validated AUCs were similar across algorithms and ranged from 0.64 to 0.67. Gradient boosted trees and logistic regression had an AUC of 0.67 pre- and post-vaccine availability. CART had the lowest discrimination ability with an AUC of 0.64 pre-vaccine availability, and 0.65 post-vaccine availability. The most influential resident characteristics, irrespective of vaccine availability, included advanced age (≥ 75 years), health instability, functional and cognitive status, sex (male), and polypharmacy.

CONCLUSIONS

The predictive accuracy and discrimination exhibited by all five examined machine learning algorithms were similar. Both logistic regression and gradient boosted trees exhibit comparable performance and display slight superiority over other machine learning algorithms. We observed consistent model performance both before and after vaccine availability. The influence of resident characteristics on COVID-19 mortality remained consistent across time periods, suggesting that changes to pre-vaccination screening practices for high-risk individuals are effective in the post-vaccination era.

The Association Between Frailty and a Nurse-Identified Need for Comprehensive Geriatric Assessment Referral from the Emergency Department

BACKGROUND

Emergency nurses commonly conduct geriatric assessments in the emergency department (ED). However, little is known about what geriatric syndromes or clinical presentations prompt a nurse to document an identified need for comprehensive geriatric assessment (CGA).

OBJECTIVES

To examine the association between geriatric syndromes, like frailty, and a nurse-identified need for a CGA following emergency care.

METHODS

We conducted a secondary analysis of a multi-province Canadian cohort from the InterRAI Multinational Cohort Study. We collected data at ED registration from patients 75 years of age and older (n = 2,274) from eight ED sites across Canada between November 2009 and April 2012. Geriatric syndromes were assessed by trained emergency nurses using the interRAI ED Contact Assessment; and we retrospectively calculated the ED frailty index. We employed binary logistic regression to determine the adjusted associations between geriatric syndromes and a nurse-identified need for a CGA.

RESULTS

Approximately one-quarter (28%) of older adults were identified to need a CGA following emergency care. A 0.1 unit increase in the ED frailty index increased the likelihood of a nurse identify a need for CGA (RD: 6.6; 95% CI = 5.5-7.9). Most geriatric syndromes increased the probability of a nurse documenting the need for a CGA.

CONCLUSION

When assessed by emergency nurses, the identified need for CGA is strongly linked to the presence of geriatric syndromes, including frailty. We provide face validity for the continued use of emergency nurses for screening and assessing older ED patients.

Prehospital prediction of hospital admission for emergent acuity patients transported by paramedics: A population-based cohort study using machine learning

INTRODUCTION

The closest emergency department (ED) may not always be the optimal hospital for certain stable high acuity patients if further distanced ED's can provide specialized care or are less overcrowded. Machine learning (ML) predictions may support paramedic decision-making to transport a subgroup of emergent patients to a more suitable, albeit more distanced, ED if hospital admission is unlikely. We examined whether characteristics known to paramedics in the prehospital setting were predictive of hospital admission in emergent acuity patients.

MATERIALS AND METHODS

We conducted a population-level cohort study using four ML algorithms to analyze ED visits of the National Ambulatory Care Reporting System from January 1, 2018 to December 31, 2019 in Ontario, Canada. We included all adult patients (≥18 years) transported to the ED by paramedics with an emergent Canadian Triage Acuity Scale score. We included eight characteristic classes as model predictors that are recorded at ED triage. All ML algorithms were trained and assessed using 10-fold cross-validation to predict hospital admission from the ED. Predictive model performance was determined using the area under curve (AUC) with 95% confidence intervals and probabilistic accuracy using the Brier Scaled score. Variable importance scores were computed to determine the top 10 predictors of hospital admission.

RESULTS

All machine learning algorithms demonstrated acceptable accuracy in predicting hospital admission (AUC 0.77-0.78, Brier Scaled 0.22-0.24). The characteristics most predictive of admission were age between 65 to 105 years, referral source from a residential care facility, presenting with a respiratory complaint, and receiving home care.

DISCUSSION

Hospital admission was accurately predicted based on patient characteristics known prehospital to paramedics prior to arrival. Our results support consideration of policy modification to permit certain emergent acuity patients to be transported to a further distanced ED. Additionally, this study demonstrates the utility of ML in paramedic and prehospital research.

Using Machine Learning to Predict Hospital Disposition With Geriatric Emergency Department Innovation Intervention

STUDY OBJECTIVE

The Geriatric Emergency Department Innovations (GEDI) program is a nurse-based geriatric assessment and care coordination program that reduces preventable admissions for older adults. Unfortunately, only 5% of older adults receive GEDI care because of resource limitations. The objective of this study was to predict the likelihood of hospitalization accurately and consistently with and without GEDI care using machine learning models to better target patients for the GEDI program.

METHODS

We performed a cross-sectional observational study of emergency department (ED) patients between 2010 and 2018. Using propensity-score matching, GEDI patients were matched to other older adult patients. Multiple models, including random forest, were used to predict hospital admission. Multiple second-layer models, including random forest, were then used to predict whether GEDI assessment would change predicted hospital admission. Final model performance was reported as the area under the curve using receiver operating characteristic models.

RESULTS

We included 128,050 patients aged over 65 years. The random forest ED disposition model had an area under the curve of 0.774 (95% confidence interval [CI] 0.741 to 0.806). In the random forest GEDI change-in-disposition model, 24,876 (97.3%) ED visits were predicted to have no change in disposition with GEDI assessment, and 695 (2.7%) ED visits were predicted to have a change in disposition with GEDI assessment.

CONCLUSION

Our machine learning models could predict who will likely be discharged with GEDI assessment with good accuracy and thus select a cohort appropriate for GEDI care. In addition, future implementation through integration into the electronic health record may assist in selecting patients to be prioritized for GEDI care.

Examining the utility and accuracy of the interRAI Emergency Department Screener in identifying high-risk older emergency department patients: A Canadian multiprovince prospective cohort study

Objectives

We set out to determine the accuracy of the interRAI Emergency Department (ED) Screener in predicting the need for detailed geriatric assessment in the ED. Our secondary objective was to determine the discriminative ability of the interRAI ED Screener for predicting the odds of discharge home and extended ED length of stay (>24 hours).

Methods

We conducted a multiprovince prospective cohort study in Canada. The need for detailed geriatric assessment was determined using the interRAI ED Screener and the interRAI ED Contact Assessment as the reference standard. A score of ≥5 was used to classify high-risk patients. Assessments were conducted by emergency and research nurses. We calculated the sensitivity, positive predictive value, and false discovery rate of the interRAI ED Screener. We employed logistic regression to predict ED outcomes while adjusting for age, sex, academic status, and the province of care.

Results

A total of 5629 older ED patients across 11 ED sites were evaluated using the interRAI ED Screener and 1061 were evaluated with the interRAI ED Contact Assessment. Approximately one-third of patients were discharged home or experienced an extended ED length of stay. The interRAI ED Screener had a sensitivity of 93%, a positive predictive value of 82%, and a false discovery rate of 18%. The interRAI ED Screener predicted discharge home and extended ED length of stay with fair accuracy.

Conclusion

The interRAI ED Screener is able to accurately and rapidly identify individuals with medical complexity. The interRAI ED Screener predicts patient-important health outcomes in older ED patients, highlighting its value for vulnerability screening.

Predicting hospital admission from emergency department triage data for patients presenting with fall-related fractures

PURPOSE

Predict in advance the need for hospitalization of adult patients for fall-related fractures based on information available at the time of triage to help decision-making at the emergency department (ED).

METHODS

We developed machine learning models using routinely collected triage data at a regional hospital chain in Pennsylvania to predict admission to an inpatient unit. We considered all patients presenting to the ED for fall-related fractures. Patients who were 18 years or younger, who left the ED against medical advice, left the ED waiting room without being seen by a provider, and left the ED after initial diagnostics were excluded from the analysis. We compared models obtained using triage data (pre-model) with models developed using additional data obtained after physicians' diagnoses (post-model).

RESULTS

Our results show good discriminatory power on predicting hospital admissions. Neural network models performed the best (AUC: pre-model = 0.938 [CI 0.920-0.956], post-model = 0.983 [0.974-0.992]). The logistic regression analysis provides additional insights into the data and the relationships between the variables.

CONCLUSIONS

Using limited data available at the time of triage, we developed four machine learning models aimed at predicting hospitalization for patients presenting to the ED for fall-related fractures. All the four models were robust and performed well. Neural network method, however, performed the best for both pre- and post-models. Simple, parsimonious machine learning models can provide high accuracy for predicting hospital admission.

Benchmarking emergency department prediction models with machine learning and public electronic health records

The demand for emergency department (ED) services is increasing across the globe, particularly during the current COVID-19 pandemic. Clinical triage and risk assessment have become increasingly challenging due to the shortage of medical resources and the strain on hospital infrastructure caused by the pandemic. As a result of the widespread use of electronic health records (EHRs), we now have access to a vast amount of clinical data, which allows us to develop prediction models and decision support systems to address these challenges. To date, there is no widely accepted clinical prediction benchmark related to the ED based on large-scale public EHRs. An open-source benchmark data platform would streamline research workflows by eliminating cumbersome data preprocessing, and facilitate comparisons among different studies and methodologies. Based on the Medical Information Mart for Intensive Care IV Emergency Department (MIMIC-IV-ED) database, we created a benchmark dataset and proposed three clinical prediction benchmarks. This study provides future researchers with insights, suggestions, and protocols for managing data and developing predictive tools for emergency care.

Imbalanced prediction of emergency department admission using natural language processing and deep neural network

The emergency department (ED) plays a very significant role in the hospital. Owing to the rising number of ED visits, medical service points, and ED market, overcrowding of EDs has become serious worldwide. Overcrowding has long been recognized as a vital issue that increases the risk to patients and negative emotions of medical personnel and impacts hospital cost management. For the past years, many researchers have been applying artificial intelligence to reduce crowding situations in the ED. Nevertheless, the datasets in ED hospital admission are naturally inherent with the high-class imbalance in the real world. Previous studies have not considered the imbalance of the datasets, particularly addressing the imbalance. This study purposes to develop a natural language processing model of a deep neural network with an attention mechanism to solve the imbalanced problem in ED admission. The proposed framework is used for predicting hospital admission so that the hospitals can arrange beds early and solve the problem of congestion in the ED. Furthermore, the study compares a variety of methods and obtains the best composition that has the best performance for forecasting hospitalization in ED. The study used the data from a specific hospital in Taiwan as an empirical study. The experimental result demonstrates that almost all imbalanced methods can improve the model's performance. In addition, the natural language processing model of Bi-directional Long Short-Term Memory with attention mechanism has the best results in all-natural language processing methods.

Predicting Patient Wait Times by Using Highly Deidentified Data in Mental Health Care: Enhanced Machine Learning Approach

BACKGROUND

Wait times impact patient satisfaction, treatment effectiveness, and the efficiency of care that the patients receive. Wait time prediction in mental health is a complex task and is affected by the difficulty in predicting the required number of treatment sessions for outpatients, high no-show rates, and the possibility of using group treatment sessions. The task of wait time analysis becomes even more challenging if the input data has low utility, which happens when the data is highly deidentified by removing both direct and quasi identifiers.

OBJECTIVE

The first aim of this study was to develop machine learning models to predict the wait time from referral to the first appointment for psychiatric outpatients by using real-time data. The second aim was to enhance the performance of these predictive models by utilizing the system's knowledge while the input data were highly deidentified. The third aim was to identify the factors that drove long wait times, and the fourth aim was to build these models such that they were practical and easy-to-implement (and therefore, attractive to care providers).

METHODS

We analyzed retrospective highly deidentified administrative data from 8 outpatient clinics at Ontario Shores Centre for Mental Health Sciences in Canada by using 6 machine learning methods to predict the first appointment wait time for new outpatients. We used the system's knowledge to mitigate the low utility of our data. The data included 4187 patients who received care through 30,342 appointments.

RESULTS

The average wait time varied widely between different types of mental health clinics. For more than half of the clinics, the average wait time was longer than 3 months. The number of scheduled appointments and the rate of no-shows varied widely among clinics. Despite these variations, the random forest method provided the minimum root mean square error values for 4 of the 8 clinics, and the second minimum root mean square error for the other 4 clinics. Utilizing the system's knowledge increased the utility of our highly deidentified data and improved the predictive power of the models.

CONCLUSIONS

The random forest method, enhanced with the system's knowledge, provided reliable wait time predictions for new outpatients, regardless of low utility of the highly deidentified input data and the high variation in wait times across different clinics and patient types. The priority system was identified as a factor that contributed to long wait times, and a fast-track system was suggested as a potential solution.

Influence of artificial intelligence on the work design of emergency department clinicians a systematic literature review

Objective

This systematic literature review aims to demonstrate how Artificial Intelligence (AI) is currently used in emergency departments (ED) and how it alters the work design of ED clinicians. AI is still new and unknown to many healthcare professionals in emergency care, leading to unfamiliarity with its capabilities.

Method

Various criteria were used to establish the suitability of the articles to answer the research question. This study was based on 34 selected peer-reviewed papers on the use of Artificial Intelligence (AI) in the Emergency Department (ED), published in the last five years. Drawing on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, all articles were scanned, read full-text, and analyzed afterward.

Results

The majority of the AI applications consisted of AI-based tools to aid with clinical decisions and to relieve overcrowded EDs of their burden. AI support was mostly offered during triage, the moment that sets the patient trajectory. There is ample evidence that AI-based applications could improve the clinical decision-making process.

Conclusion

The use of AI in EDs is still in its nascent stages. Many studies focus on the question of whether AI has clinical utility, such as decision support, improving resource allocation, reducing diagnostic errors, and promoting proactivity. Some studies suggest that AI-based tools essentially have the ability to outperform human skills. However, it is evident from the literature that current technology does not have the aims or power to do so. Nevertheless, AI-based tools can impact clinician work design in the ED by providing support with clinical decisions, which could ultimately help alleviate a portion of the increasing clinical burden.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-08070-7.

Leveraging Large-Scale Electronic Health Records and Interpretable Machine Learning for Clinical Decision Making at the Emergency Department: Protocol for System Development and Validation

Background

There is a growing demand globally for emergency department (ED) services. An increase in ED visits has resulted in overcrowding and longer waiting times. The triage process plays a crucial role in assessing and stratifying patients’ risks and ensuring that the critically ill promptly receive appropriate priority and emergency treatment. A substantial amount of research has been conducted on the use of machine learning tools to construct triage and risk prediction models; however, the black box nature of these models has limited their clinical application and interpretation.

Objective

In this study, we plan to develop an innovative, dynamic, and interpretable System for Emergency Risk Triage (SERT) for risk stratification in the ED by leveraging large-scale electronic health records (EHRs) and machine learning.

Methods

To achieve this objective, we will conduct a retrospective, single-center study based on a large, longitudinal data set obtained from the EHRs of the largest tertiary hospital in Singapore. Study outcomes include adverse events experienced by patients, such as the need for an intensive care unit and inpatient death. With preidentified candidate variables drawn from expert opinions and relevant literature, we will apply an interpretable machine learning–based AutoScore to develop 3 SERT scores. These 3 scores can be used at different times in the ED, that is, on arrival, during ED stay, and at admission. Furthermore, we will compare our novel SERT scores with established clinical scores and previously described black box machine learning models as baselines. Receiver operating characteristic analysis will be conducted on the testing cohorts for performance evaluation.

Results

The study is currently being conducted. The extracted data indicate approximately 1.8 million ED visits by over 810,000 unique patients. Modelling results are expected to be published in 2022.

Conclusions

The SERT scoring system proposed in this study will be unique and innovative because of its dynamic nature and modelling transparency. If successfully validated, our proposed solution will establish a standard for data processing and modelling by taking advantage of large-scale EHRs and interpretable machine learning tools.

International Registered Report Identifier (IRRID)

DERR1-10.2196/34201

Machine learning in patient flow: a review

This work is a review of the ways in which machine learning has been used in order to plan, improve or aid the problem of moving patients through healthcare services. We decompose the patient flow problem into four subcategories: prediction of demand on a healthcare institution, prediction of the demand and resource required to transfer patients from the emergency department to the hospital, prediction of potential resource required for the treatment and movement of inpatients and prediction of length-of-stay and discharge timing. We argue that there are benefits to both approaches of considering the healthcare institution as a whole as well as the patient by patient case and that ideally a combination of these would be best for improving patient flow through hospitals. We also argue that it is essential for there to be a shared dataset that will allow researchers to benchmark their algorithms on and thereby allow future researchers to build on that which has already been done. We conclude that machine learning for the improvement of patient flow is still a young field with very few papers tailor-making machine learning methods for the problem being considered. Future works should consider the need to transfer algorithms trained on a dataset to multiple hospitals and allowing for dynamic algorithms which will allow real-time decision-making to help clinical staff on the shop floor.