An exhaustive review and analysis on applications of statistical forecasting in hospital emergency departments

Probabilistic prediction of arrivals and hospitalizations in emergency departments in Île-de-France

BACKGROUND

Forecasts of future demand is foundational for effective resource allocation in emergency departments (EDs). As ED demand is inherently variable, it is important for forecasts to characterize the range of possible future demand. However, extant research focuses primarily on producing point forecasts using a wide variety of prediction algorithms. In this study, our objective is to generate point and interval predictions that accurately characterize the variability in ED demand using ensemble methods that combine predictions from multiple base algorithms based on their empirical performance.

METHODS

Data consisted in daily arrivals and subsequent hospitalizations at 72 emergency departments in Île-de-France from 2014-2018. Additional explanatory variables were collected including public and school holidays, meteorological variables, and public health trends. One-day ahead point and 80% interval predictions of arrivals and hospitalizations were produced by predicting the 10%, 50%, and 90% quantiles of the forecast distribution. Quantile prediction algorithms included methods such as ARIMAX, variations of random forests, and generalized additive models. Ensemble predictions were then formed using Exponentially Weighted Averaging, Bernstein Online Aggregation, and Super Learning. Prediction intervals were post-processed using Adaptive Conformal Inference techniques. Point predictions were evaluated by their Mean Absolute Error (MAE) and Mean Absolute Percentage Error (MAPE), and 80% interval predictions by their empirical coverage and mean interval width.

RESULTS

For point forecasts, ensemble methods achieved lower average MAE and MAPE than any of the base algorithms. All of the base algorithms and ensemble methods yielded prediction intervals with near optimal empirical coverage after conformalization. For hospitalizations, the shortest mean interval widths were achieved by the ensemble methods.

CONCLUSIONS

Ensemble methods yield joint point and prediction intervals that adapt to individual EDs and achieve better performance than individual algorithms. Conformal inference techniques improve the performance of the prediction intervals.

Forecasting Mortality Associated Emergency Department Crowding with LightGBM and Time Series Data

Emergency department (ED) crowding is a global public health issue that has been repeatedly associated with increased mortality. Predicting future service demand would enable preventative measures aiming to eliminate crowding along with its detrimental effects. Recent findings in our ED indicate that occupancy ratios exceeding 90% are associated with increased 10-day mortality. In this paper, we aim to predict these crisis periods using retrospective time series data such as weather, availability of hospital beds, calendar variables and occupancy statistics from a large Nordic ED with a LightGBM model. We predict mortality associated crowding for the whole ED and individually for its different operational sections. We demonstrate that afternoon crowding can be predicted at 11 a.m. with an AUC of 0.82 (95% CI 0.78-0.86) and at 8 a.m. with an AUC up to 0.79 (95% CI 0.75-0.83). Consequently we show that forecasting mortality-associated crowding using time series data is feasible.

Enhanced forecasting of emergency department patient arrivals using feature engineering approach and machine learning

BACKGROUND

Emergency department (ED) overcrowding is an important problem in many countries. Accurate predictions of ED patient arrivals can help management to better allocate staff and medical resources. In this study, we investigate the use of calendar and meteorological predictors, as well as feature-engineered variables, to predict daily patient arrivals using datasets from eleven different EDs across three countries.

METHODS

Six machine learning (ML) algorithms were tested on forecasting horizons of 7 and 45 days. Three of them - Light Gradient Boosting Machine (LightGBM), Support Vector Machine with Radial Basis Function (SVM-RBF), and Neural Network Autoregression (NNAR) - were never before reported for predicting ED patient arrivals. Algorithms' hyperparameters were tuned through a grid-search with cross-validation. Prediction performance was assessed using fivefold cross-validation and four performance metrics.

RESULTS

The eXtreme Gradient Boosting (XGBoost) was the best-performing model on both prediction horizons, also outperforming results reported in past studies on ED arrival prediction. XGBoost and NNAR achieved the best performance in nine out of the eleven analyzed datasets, with MAPE values ranging from 5.03% to 14.1%. Feature engineering (FE) improved the performance of the ML algorithms.

CONCLUSION

Accuracy in predicting ED arrivals, achieved through the FE approach, is key for managing human and material resources, as well as reducing patient waiting times and lengths of stay.

Development of an approach to forecast future takeaway outlet growth around schools and population exposure to takeaways in England

BACKGROUND

Neighbourhood exposure to takeaways can contribute negatively to diet and diet-related health outcomes. Urban planners within local authorities (LAs) in England can modify takeaway exposure through denying planning permission to new outlets in management zones around schools. LAs sometimes refer to these as takeaway "exclusion zones". Understanding the long-term impacts of this intervention on the takeaway retail environment and health, an important policy question, requires methods to forecast future takeaway growth and subsequent population-level exposure to takeaways. In this paper we describe a novel two-stage method to achieve this.

METHODS

We used historic data on locations of takeaways and a time-series auto-regressive integrated moving average (ARIMA) model, to forecast numbers of outlets within management zones to 2031, based on historical trends, in six LAs with different urban/rural characteristics across England. Forecast performance was evaluated based on root mean squared error (RMSE) and mean absolute scaled error (MASE) scores in time-series cross-validation. Using travel-to-work data from the 2011 UK census, we then translated these forecasts of the number of takeaways within management zones into population-level exposures across home, work and commuting domains.

RESULTS

Our ARIMA models outperformed exponential smoothing equivalents according to RMSE and MASE. The model was able to forecast growth in the count of takeaways up to 2031 across all six LAs, with variable growth rates by RUC (min-max: 39.4-79.3%). Manchester (classified as a non-London urban with major conurbation LA) exhibited the highest forecast growth rate (79.3%, 95% CI 61.6, 96.9) and estimated population-level takeaway exposure within management zones, increasing by 65.5 outlets per capita to 148.2 (95% CI 133.6, 162.7) outlets. Overall, urban (vs. rural) LAs were forecast stronger growth and higher population exposures.

CONCLUSIONS

Our two-stage forecasting approach provides a novel way to estimate long-term future takeaway growth and population-level takeaway exposure. While Manchester exhibited the strongest growth, all six LAs were forecast marked growth that might be considered a risk to public health. Our methods can be used to model future growth in other types of retail outlets and in other areas.

Improving ED admissions forecasting by using generative AI: An approach based on DGAN

BACKGROUND AND OBJECTIVE

Generative Deep Learning has emerged in recent years as a significant player in the Artificial Intelligence field. Synthesizing new data while maintaining the features of reality has revolutionized the field of Deep Learning, proving to be particularly useful in contexts where obtaining data is challenging. The objective of this study is to employ the DoppelGANger algorithm, a cutting-edge approach based on Generative Adversarial Networks for time series, to enhance patient admissions forecasting in a hospital Emergency Department.

METHODS

We employed the DoppelGANger algorithm in a sequential methodology, conditioning generated time series with unique attributes to optimize data utilization. After confirming the successful creation of synthetic data with new attribute values, we adopted the Train-Synthetic-Test-Real framework to ensure the reliability of our synthetic data validation. We then augmented the original series with synthetic data to enhance the Prophet model's performance. This process was applied to two datasets derived from the original: one with four years of training followed by one year of testing, and another with three years of training and two years of testing.

RESULTS

The experimental results show that the generative model outperformed Prophet on the forecasting task, improving the SMAPE from 7.30 to 6.99 with the four-year training set, and from 22.84 to 7.41 for the three-year training set, all in daily aggregations. For the data replacement task, the Prophet SMAPE values decreased to 6.84 and 7.18 for four and three-year sets on the same aggregation. Additionally, data augmentation reduced the SMAPE to 6.79 for a one-year test set and achieved 8.56 for the two-year test set, surpassing the performance achieved by the same Prophet model when trained only on real data. Results for the remaining aggregations were consistent.

CONCLUSIONS

The findings of this study suggest that employing a generative algorithm to extend a training dataset can effectively enhance predictive models within the domain of Emergency Department admissions. The improvement can lead to more efficient resource allocation and patient management.

Physician performance scores used to predict emergency department admission numbers and excessive admissions burden

BACKGROUND

Overcrowding in hospitals is associated with a panoply of adverse events. Inappropriate decisions in the emergency department (ED) contribute to overcrowding. The performance of individual physicians as part of the admitting team is a critical factor in determining the overall rate of admissions. While previous attempts to model admission numbers have been based on a range of variables, none have included measures of individual staff performance. We construct reliable objective measures of staff performance and use these, among other factors, to predict the number of daily admissions. Such modelling will enable enhanced workforce planning and timely intervention to reduce inappropriate admissions and overcrowding.

METHODS

A database was created of 232 245 ED attendances at Meir Medical Center in central Israel, spanning the years 2016-2021. We use several measures of physician performance together with historic caseload data and other variables to derive statistical models for the prediction of ED arrival and admission numbers.

RESULTS

Our models predict arrival numbers with a mean absolute percentage error (MAPE) of 6.85%, and admission numbers with a MAPE of 10.6%, and provide a same-day alert for heavy admissions burden with 75% sensitivity for a false-positive rate of 20%. The inclusion of physician performance measures provides an essential boost to model performance.

CONCLUSIONS

Arrival number and admission numbers can be predicted with sufficient fidelity to enable interventions to reduce excess admissions and smooth patient flow. Individual staff performance has a strong effect on admission rates and is a critical variable for the effective modelling of admission numbers.

Diagnosing patient flow issues in the emergency department: an Australasian hospital case study

PURPOSE

This study investigates how a hospital can increase the flow of patients through its emergency department by using benchmarking and process improvement techniques borrowed from the manufacturing sector.

DESIGN/METHODOLOGY/APPROACH

An in-depth case study of an Australasian public hospital utilises rigorous, multi-method data collection procedures with systems thinking to benchmark an emergency department (ED) value stream and identify the performance inhibitors.

FINDINGS

High levels of value stream uncertainty result from inefficient processes and weak controls. Reduced patient flow arises from senior management's commitment to simplistic government targets, clinical staff that lack basic operations management skills, and fragmented information systems. High junior/senior staff ratios aggravate the lack of inter-functional integration and poor use of time and material resources, increasing the risk of a critical patient incident.

RESEARCH LIMITATIONS/IMPLICATIONS

This research is limited to a single case; hence, further research should assess value stream maturity and associated performance enablers and inhibitors in other emergency departments experiencing patient flow delays.

PRACTICAL IMPLICATIONS

This study illustrates how hospital managers can use systems thinking and a context-free performance benchmarking measure to identify needed interventions and transferable best practices for achieving seamless patient flow.

ORIGINALITY/VALUE

This study is the first to operationalise the theoretical concept of the seamless healthcare system to acute care as defined by Parnaby and Towill (2008). It is also the first to use the uncertainty circle model in an Australasian public healthcare setting to objectively benchmark an emergency department's value stream maturity.

An explainable machine learning approach for hospital emergency department visits forecasting using continuous training and multi-model regression

BACKGROUND AND OBJECTIVE

In the last years, the Emergency Department (ED) has become an important source of admissions for hospitals. Since late 90s, the number of ED visits has been steadily increasing, and since Covid19 pandemic this trend has been much stronger. Accurate prediction of ED visits, even for moderate forecasting time-horizons, can definitively improve operational efficiency, quality of care, and patient outcomes in hospitals.

METHODS

In this paper we propose two different interpretable approaches, based on Machine Learning algorithms, to accurately forecast hospital emergency visits. The proposed approaches involve a first step of data segmentation based on two different criteria, depending on the approach considered: first, a threshold-based strategy is adopted, where data is divided depending on the value of specific predictor variables. In a second approach, a cluster-based ensemble learning is proposed, in such a way that a clustering algorithm is applied to the training dataset, and ML models are then trained for each cluster.

RESULTS

The two proposed methodologies have been evaluated in real data from two hospital ED visits datasets in Spain. We have shown that the proposed approaches are able to obtain accurate ED visits forecasting, in short-term and also long-term prediction time-horizons up to one week, improving the efficiency of alternative prediction methods for this problem.

CONCLUSIONS

The proposed forecasting approaches have a strong emphasis on providing explainability to the problem. An analysis on which variables govern the problem and are pivotal for obtaining accurate predictions is finally carried out and included in the discussion of the paper.

Alcohol-Related Presentations to Emergency Departments on Days with Holidays, Social, and Sporting Events: An Integrative Literature Review

INTRODUCTION

Events, specifically those where excessive alcohol consumption is common, pose a risk to increase alcohol-related presentations to emergency departments (EDs). Limited evidence exists that synthesizes the impact from events on alcohol-related presentations to EDs.

STUDY OBJECTIVE

This integrative review aimed to synthesize the literature regarding the impact events have on alcohol-related presentations to EDs.

METHODS

An integrative literature review methodology was guided by the Preferred Reporting Items of Systematic Reviews and Meta-Analysis (PRISMA) Guidelines for data collection, and Whittemore and Knafl's framework for data analysis. Information sources used to identify studies were MEDLINE, CINAHL, and EMBASE, last searched May 26, 2021.

RESULTS

In total, 23 articles describing 46 events met criteria for inclusion. There was a noted increase in alcohol-related presentations to EDs from 27 events, decrease from eight events, and no change from 25 events. Public holidays, music festivals, and sporting events resulted in the majority of increased alcohol-related presentations to EDs. Few articles focused on ED length-of-stay (LOS), treatment, and disposition.

CONCLUSION

An increase in the consumption of alcohol from holiday, social, and sporting events pose the risk for an influx of presentations to EDs and as a result may negatively impact departmental flow. Further research examining health service outcomes is required that considers the impact of events from a local, national, and global perspective.

Combining machine learning and optimization for the operational patient-bed assignment problem

Assigning inpatients to hospital beds impacts patient satisfaction and the workload of nurses and doctors. The assignment is subject to unknown inpatient arrivals, in particular for emergency patients. Hospitals, therefore, need to deal with uncertainty on actual bed requirements and potential shortage situations as bed capacities are limited. This paper develops a model and solution approach for solving the patient bed-assignment problem that is based on a machine learning (ML) approach to forecasting emergency patients. First, it contributes by improving the anticipation of emergency patients using ML approaches, incorporating weather data, time and dates, important local and regional events, as well as current and historical occupancy levels. Drawing on real-life data from a large case hospital, we were able to improve forecasting accuracy for emergency inpatient arrivals. We achieved up to 17% better root mean square error (RMSE) when using ML methods compared to a baseline approach relying on averages for historical arrival rates. We further show that the ML methods outperform time series forecasts. Second, we develop a new hyper-heuristic for solving real-life problem instances based on the pilot method and a specialized greedy look-ahead (GLA) heuristic. When applying the hyper-heuristic in test sets we were able to increase the objective function by up to 5.3% in comparison to the benchmark approach in [40]. A benchmark with a Genetic Algorithm shows also the superiority of the hyper-heuristic. Third, the combination of ML for emergency patient admission forecasting with advanced optimization through the hyper-heuristic allowed us to obtain an improvement of up to 3.3% on a real-life problem.

Early Warning Software for Emergency Department Crowding

Emergency department (ED) crowding is a well-recognized threat to patient safety and it has been repeatedly associated with increased mortality. Accurate forecasts of future service demand could lead to better resource management and has the potential to improve treatment outcomes. This logic has motivated an increasing number of research articles but there has been little to no effort to move these findings from theory to practice. In this article, we present first results of a prospective crowding early warning software, that was integrated to hospital databases to create real-time predictions every hour over the course of 5 months in a Nordic combined ED using Holt-Winters' seasonal methods. We show that the software could predict next hour crowding with an AUC of 0.94 (95% CI: 0.91-0.97) and 24 hour crowding with an AUC of 0.79 (95% CI: 0.74-0.84) using simple statistical models. Moreover, we suggest that afternoon crowding can be predicted at 1 p.m. with an AUC of 0.84 (95% CI: 0.74-0.91).

Probabilistic forecasting of hourly emergency department arrivals

An accurate forecast of Emergency Department (ED) arrivals by an hour of the day is critical to meet patients' demand. It enables planners to match ED staff to the number of arrivals, redeploy staff, and reconfigure units. In this study, we develop a model based on Generalised Additive Models and an advanced dynamic model based on exponential smoothing to generate an hourly probabilistic forecast of ED arrivals for a prediction window of 48 hours. We compare the forecast accuracy of these models against appropriate benchmarks, including TBATS, Poisson Regression, Prophet, and simple empirical distribution. We use Root Mean Squared Error to examine the point forecast accuracy and assess the forecast distribution accuracy using Quantile Bias, PinBall Score and Pinball Skill Score. Our results indicate that the proposed models outperform their benchmarks. Our developed models can also be generalised to other services, such as hospitals, ambulances or clinical desk services.

XAIRE: An ensemble-based methodology for determining the relative importance of variables in regression tasks. Application to a hospital emergency department

Nowadays it is increasingly important in many applications to understand how different factors influence a variable of interest in a predictive modeling process. This task becomes particularly important in the context of Explainable Artificial Intelligence. Knowing the relative impact of each variable on the output allows us to acquire more information about the problem and about the output provided by a model. This paper proposes a new methodology, XAIRE, that determines the relative importance of input variables in a prediction environment, considering multiple prediction models in order to increase generality and avoid bias inherent in a particular learning algorithm. Concretely, we present an ensemble-based methodology that promotes the aggregation of results from several prediction methods to obtain a relative importance ranking. Also, statistical tests are considered in the methodology in order to reveal significant differences between the relative importance of the predictor variables. As a case study, XAIRE is applied to the arrival of patients in a Hospital Emergency Department, which has resulted in one of the largest sets of different predictor variables in the literature. Results show the extracted knowledge related to the relative importance of the predictors involved in the case study.

Use of Real-Time Information to Predict Future Arrivals in the Emergency Department

STUDY OBJECTIVE

We aimed to build prediction models for shift-level emergency department (ED) patient volume that could be used to facilitate prediction-driven staffing. We sought to evaluate the predictive power of rich real-time information and understand 1) which real-time information had predictive power and 2) what prediction techniques were appropriate for forecasting ED demand.

METHODS

We conducted a retrospective study in an ED site in a large academic hospital in New York City. We examined various prediction techniques, including linear regression, regression trees, extreme gradient boosting, and time series models. By comparing models with and without real-time predictors, we assessed the potential gain in prediction accuracy from real-time information.

RESULTS

Real-time predictors improved prediction accuracy on models without contemporary information from 5% to 11%. Among extensive real-time predictors examined, recent patient arrival counts, weather, Google trends, and concurrent patient comorbidity information had significant predictive power. Out of all the forecasting techniques explored, SARIMAX (Seasonal Autoregressive Integrated Moving Average with eXogenous factors) achieved the smallest out-of-sample the root mean square error (RMSE) of 14.656 and mean absolute prediction error (MAPE) of 8.703%. Linear regression was the second best, with out-of-sample RMSE and MAPE equal to 15.366 and 9.109%, respectively.

CONCLUSION

Real-time information was effective in improving the prediction accuracy of ED demand. Practice and policy implications for designing staffing paradigms with real-time demand forecasts to reduce ED congestion were discussed.

Forecasting admissions in psychiatric hospitals before and during Covid-19: a retrospective study with routine data

The COVID-19 pandemic has strong effects on most health care systems. Forecasting of admissions can help for the efficient organisation of hospital care. We aimed to forecast the number of admissions to psychiatric hospitals before and during the COVID-19 pandemic and we compared the performance of machine learning models and time series models. This would eventually allow to support timely resource allocation for optimal treatment of patients. We used admission data from 9 psychiatric hospitals in Germany between 2017 and 2020. We compared machine learning models with time series models in weekly, monthly and yearly forecasting before and during the COVID-19 pandemic. A total of 90,686 admissions were analysed. The models explained up to 90% of variance in hospital admissions in 2019 and 75% in 2020 with the effects of the COVID-19 pandemic. The best models substantially outperformed a one-step seasonal naïve forecast (seasonal mean absolute scaled error (sMASE) 2019: 0.59, 2020: 0.76). The best model in 2019 was a machine learning model (elastic net, mean absolute error (MAE): 7.25). The best model in 2020 was a time series model (exponential smoothing state space model with Box-Cox transformation, ARMA errors and trend and seasonal components, MAE: 10.44). Models forecasting admissions one week in advance did not perform better than monthly and yearly models in 2019 but they did in 2020. The most important features for the machine learning models were calendrical variables. Model performance did not vary much between different modelling approaches before the COVID-19 pandemic and established forecasts were substantially better than one-step seasonal naïve forecasts. However, weekly time series models adjusted quicker to the COVID-19 related shock effects. In practice, multiple individual forecast horizons could be used simultaneously, such as a yearly model to achieve early forecasts for a long planning period and weekly models to adjust quicker to sudden changes.

COVID-19 ICU demand forecasting: A two-stage Prophet-LSTM approach

Recent literature has revealed a growing interest in methods for anticipating the demand for medical items and personnel at hospital, especially during turbulent scenarios such as the COVID-19 pandemic. In times like those, new variables appear and affect the once known demand behavior. This paper investigates the hypothesis that the combined Prophet-LSTM method results in more accurate forecastings for COVID-19 hospital Intensive Care Units (ICUs) demand than both standalone models, Prophet and LSTM (Long Short-Term Memory Neural Network). We also compare the model to well-established demand forecasting benchmarks. The model is tested to a representative Brazilian municipality that serves as a medical reference to other cities within its region. In addition to traditional time series components, such as trend and seasonality, other variables such as the current number of daily COVID-19 cases, vaccination rates, non-pharmaceutical interventions, social isolation index, and regional hospital beds occupation are also used to explain the variations in COVID-19 hospital ICU demand. Results indicate that the proposed method produced Mean Average Errors (MAE) from 13% to 45% lower than well established statistical and machine learning forecasting models, including the standalone models.

Accurate Forecasting of Emergency Department Arrivals With Internet Search Index and Machine Learning Models: Model Development and Performance Evaluation

BACKGROUND

Emergency department (ED) overcrowding is a concerning global health care issue, which is mainly caused by the uncertainty of patient arrivals, especially during the pandemic. Accurate forecasting of patient arrivals can allow health resource allocation in advance to reduce overcrowding. Currently, traditional data, such as historical patient visits, weather, holiday, and calendar, are primarily used to create forecasting models. However, data from an internet search engine (eg, Google) is less studied, although they can provide pivotal real-time surveillance information. The internet data can be employed to improve forecasting performance and provide early warning, especially during the epidemic. Moreover, possible nonlinearities between patient arrivals and these variables are often ignored.

OBJECTIVE

This study aims to develop an intelligent forecasting system with machine learning models and internet search index to provide an accurate prediction of ED patient arrivals, to verify the effectiveness of the internet search index, and to explore whether nonlinear models can improve the forecasting accuracy.

METHODS

Data on ED patient arrivals were collected from July 12, 2009, to June 27, 2010, the period of the 2009 H1N1 pandemic. These included 139,910 ED visits in our collaborative hospital, which is one of the biggest public hospitals in Hong Kong. Traditional data were also collected during the same period. The internet search index was generated from 268 search queries on Google to comprehensively capture the information about potential patients. The relationship between the index and patient arrivals was verified by Pearson correlation coefficient, Johansen cointegration, and Granger causality. Linear and nonlinear models were then developed with the internet search index to predict patient arrivals. The accuracy and robustness were also examined.

RESULTS

All models could accurately predict patient arrivals. The causality test indicated internet search index as a strong predictor of ED patient arrivals. With the internet search index, the mean absolute percentage error (MAPE) and the root mean square error (RMSE) of the linear model reduced from 5.3% to 5.0% and from 24.44 to 23.18, respectively, whereas the MAPE and RMSE of the nonlinear model decreased even more, from 3.5% to 3% and from 16.72 to 14.55, respectively. Compared with each other, the experimental results revealed that the forecasting system with extreme learning machine, as well as the internet search index, had the best performance in both forecasting accuracy and robustness analysis.

CONCLUSIONS

The proposed forecasting system can make accurate, real-time prediction of ED patient arrivals. Compared with the static traditional variables, the internet search index significantly improves forecasting as a reliable predictor monitoring continuous behavior trend and sudden changes during the epidemic (P=.002). The nonlinear model performs better than the linear counterparts by capturing the dynamic relationship between the index and patient arrivals. Thus, the system can facilitate staff planning and workflow monitoring.

A Comparison of Univariate and Multivariate Forecasting Models Predicting Emergency Department Patient Arrivals during the COVID-19 Pandemic

The COVID-19 pandemic has heightened the existing concern about the uncertainty surrounding patient arrival and the overutilization of resources in emergency departments (EDs). The prediction of variations in patient arrivals is vital for managing limited healthcare resources and facilitating data-driven resource planning. The objective of this study was to forecast ED patient arrivals during a pandemic over different time horizons. A secondary objective was to compare the performance of different forecasting models in predicting ED patient arrivals. We included all ED patient encounters at an urban teaching hospital between January 2019 and December 2020. We divided the data into training and testing datasets and applied univariate and multivariable forecasting models to predict daily ED visits. The influence of COVID-19 lockdown and climatic factors were included in the multivariable models. The model evaluation consisted of the root mean square error (RMSE) and mean absolute error (MAE) over different forecasting horizons. Our exploratory analysis illustrated that monthly and weekly patterns impact daily demand for care. The Holt-Winters approach outperformed all other univariate and multivariable forecasting models for short-term predictions, while the Long Short-Term Memory approach performed best in extended predictions. The developed forecasting models are able to accurately predict ED patient arrivals and peaks during a surge when tested on two years of data from a high-volume urban ED. These short- and long-term prediction models can potentially enhance ED and hospital resource planning.

Forecasting daily emergency department arrivals using high-dimensional multivariate data: a feature selection approach

Background and objective

Emergency Department (ED) overcrowding is a chronic international issue that is associated with adverse treatment outcomes. Accurate forecasts of future service demand would enable intelligent resource allocation that could alleviate the problem. There has been continued academic interest in ED forecasting but the number of used explanatory variables has been low, limited mainly to calendar and weather variables. In this study we investigate whether predictive accuracy of next day arrivals could be enhanced using high number of potentially relevant explanatory variables and document two feature selection processes that aim to identify which subset of variables is associated with number of next day arrivals. Performance of such predictions over longer horizons is also shown.

Methods

We extracted numbers of total daily arrivals from Tampere University Hospital ED between the time period of June 1, 2015 and June 19, 2019. 158 potential explanatory variables were collected from multiple data sources consisting not only of weather and calendar variables but also an extensive list of local public events, numbers of website visits to two hospital domains, numbers of available hospital beds in 33 local hospitals or health centres and Google trends searches for the ED. We used two feature selection processes: Simulated Annealing (SA) and Floating Search (FS) with Recursive Least Squares (RLS) and Least Mean Squares (LMS). Performance of these approaches was compared against autoregressive integrated moving average (ARIMA), regression with ARIMA errors (ARIMAX) and Random Forest (RF). Mean Absolute Percentage Error (MAPE) was used as the main error metric.

Results

Calendar variables, load of secondary care facilities and local public events were dominant in the identified predictive features. RLS-SA and RLS-FA provided slightly better accuracy compared ARIMA. ARIMAX was the most accurate model but the difference between RLS-SA and RLS-FA was not statistically significant.

Conclusions

Our study provides new insight into potential underlying factors associated with number of next day presentations. It also suggests that predictive accuracy of next day arrivals can be increased using high-dimensional feature selection approach when compared to both univariate and nonfiltered high-dimensional approach. Performance over multiple horizons was similar with a gradual decline for longer horizons. However, outperforming ARIMAX remains a challenge when working with daily data. Future work should focus on enhancing the feature selection mechanism, investigating its applicability to other domains and in identifying other potentially relevant explanatory variables.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-01878-7.

Predicting emergency department visits in a large teaching hospital

Background

Emergency department (ED) visits show a high volatility over time. Therefore, EDs are likely to be crowded at peak-volume moments. ED crowding is a widely reported problem with negative consequences for patients as well as staff. Previous studies on the predictive value of weather variables on ED visits show conflicting results. Also, no such studies were performed in the Netherlands. Therefore, we evaluated prediction models for the number of ED visits in our large the Netherlands teaching hospital based on calendar and weather variables as potential predictors.

Methods

Data on all ED visits from June 2016 until December 31, 2019, were extracted. The 2016–2018 data were used as training set, the 2019 data as test set. Weather data were extracted from three publicly available datasets from the Royal Netherlands Meteorological Institute. Weather observations in proximity of the hospital were used to predict the weather in the hospital’s catchment area by applying the inverse distance weighting interpolation method. The predictability of daily ED visits was examined by creating linear prediction models using stepwise selection; the mean absolute percentage error (MAPE) was used as measurement of fit.

Results

The number of daily ED visits shows a positive time trend and a large impact of calendar events (higher on Mondays and Fridays, lower on Saturdays and Sundays, higher at special times such as carnival, lower in holidays falling on Monday through Saturday, and summer vacation). The weather itself was a better predictor than weather volatility, but only showed a small effect; the calendar-only prediction model had very similar coefficients to the calendar+weather model for the days of the week, time trend, and special time periods (both MAPE’s were 8.7%).

Conclusions

Because of this similar performance, and the inaccuracy caused by weather forecasts, we decided the calendar-only model would be most useful in our hospital; it can probably be transferred for use in EDs of the same size and in a similar region. However, the variability in ED visits is considerable. Therefore, one should always anticipate potential unforeseen spikes and dips in ED visits that are not shown by the model.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12245-021-00357-6.

NO2 exposure increases eczema outpatient visits in Guangzhou, China: an indication for hospital management

Background

Ambient nitrogen dioxide (NO₂) is a common air pollutant in developing countries and causes skin conditions, but its effect on eczema in subtropical areas is not clear in China.

Object

To measure the effect of short-term exposure of NO₂ on the incidence of eczema and the change of outpatient visits.

Methods

Data of daily temperature, air pollutants, and outpatient visits from 2013 to 2018 were collected in a row. The generalized additive model (GAM) and Poisson distribution were used to assess the association between short-term exposure of NO₂ and the outpatient visits of patients with eczema. The cumulative exposure effect of lag 0–3 days and the displacement effect of NO₂ and other pollutants were considered as well. A single pollutant model was used to examine the independent association, and a two-pollutant model was adopted to control the confounding effect.

Results

The daily outpatient visits of eczema increased from 75.26 to 190.85 from 2013 to 2018 (P < 0.001). The combined influence of NO₂ and the related pollutant exerted a stronger influence on the incidence of eczema. The maximum effect of NO₂ appeared on the exposed day.

(lag 0) and disappeared on day 4 (lag 3). The children and seniors were more vulnerable to NO₂ exposure.

Conclusion

Exposure to NO₂ is tightly associated with eczema incidence and outpatient visits. The hospitals should react to the visit fluctuations and adjust physician duty shifts to improve outpatient service efficiency.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-021-10549-7.

Peak Outpatient and Emergency Department Visit Forecasting for Patients With Chronic Respiratory Diseases Using Machine Learning Methods: Retrospective Cohort Study

BACKGROUND

The overcrowding of hospital outpatient and emergency departments (OEDs) due to chronic respiratory diseases in certain weather or under certain environmental pollution conditions results in the degradation in quality of medical care, and even limits its availability.

OBJECTIVE

To help OED managers to schedule medical resource allocation during times of excessive health care demands after short-term fluctuations in air pollution and weather, we employed machine learning (ML) methods to predict the peak OED arrivals of patients with chronic respiratory diseases.

METHODS

In this paper, we first identified 13,218 visits from patients with chronic respiratory diseases to OEDs in hospitals from January 1, 2016, to December 31, 2017. Then, we divided the data into three datasets: weather-based visits, air quality-based visits, and weather air quality-based visits. Finally, we developed ML methods to predict the peak event (peak demand days) of patients with chronic respiratory diseases (eg, asthma, respiratory infection, and chronic obstructive pulmonary disease) visiting OEDs on the three weather data and environmental pollution datasets in Guangzhou, China.

RESULTS

The adaptive boosting-based neural networks, tree bag, and random forest achieved the biggest receiver operating characteristic area under the curve, 0.698, 0.714, and 0.809, on the air quality dataset, the weather dataset, and weather air quality dataset, respectively. Overall, random forests reached the best classification prediction performance.

CONCLUSIONS

The proposed ML methods may act as a useful tool to adapt medical services in advance by predicting the peak of OED arrivals. Further, the developed ML methods are generic enough to cope with similar medical scenarios, provided that the data is available.

A machine learning-based 1-year mortality prediction model after hospital discharge for clinical patients with acute coronary syndrome

Cardiovascular disease is the leading cause of death worldwide so, early prediction and diagnosis of cardiovascular disease is essential for patients affected by this fatal disease. The goal of this article is to propose a machine learning-based 1-year mortality prediction model after discharge in clinical patients with acute coronary syndrome. We used the Korea Acute Myocardial Infarction Registry data set, a cardiovascular disease database registered in 52 hospitals in Korea for 1 November 2005-30 January 2008 and selected 10,813 subjects with 1-year follow-up traceability. The ranges of hyperparameters to find the best prediction model were selected from four different machine learning models. Then, we generated each machine learning-based mortality prediction model with hyperparameters completed the range fitness via grid search using training data and was evaluated by fourfold stratified cross-validation. The best prediction model with the highest performance was found, and its hyperparameters were extracted. Finally, we compared the performance of machine learning-based mortality prediction models with GRACE in area under the receiver operating characteristic curve, precision, recall, accuracy, and F-score. The area under the receiver operating characteristic curve in applied machine learning algorithms was averagely improved up to 0.08 than in GRACE, and their major prognostic factors were different. This implementation would be beneficial for prediction and early detection of major adverse cardiovascular events in acute coronary syndrome patients.

A comprehensive modelling framework to forecast the demand for all hospital services

BACKGROUND

Because of increasing demand, hospitals in England are currently under intense pressure resulting in shortages of beds, nurses, clinicians, and equipment. To be able to effectively cope with this demand, the management needs to accurately find out how many patients are expected to use their services in the future. This applies not just to one service but for all hospital services.

PURPOSE

A forecasting modelling framework is developed for all hospital's acute services, including all specialties within outpatient and inpatient settings and the accident and emergency (A&E) department. The objective is to support the management to better deal with demand and plan ahead effectively.

METHODOLOGY/APPROACH

Having established a theoretical framework, we used the national episodes statistics dataset to systematically capture demand for all specialties. Three popular forecasting methodologies, namely, autoregressive integrated moving average (ARIMA), exponential smoothing, and multiple linear regression were used. A fourth technique known as the seasonal and trend decomposition using loess function (STLF) was applied for the first time within the context of health-care forecasting.

RESULTS

According to goodness of fit and forecast accuracy measures, 64 best forecasting models and periods (daily, weekly, or monthly forecasts) were selected out of 760 developed models; ie, demand was forecasted for 38 outpatient specialties (first referrals and follow-ups), 25 inpatient specialties (elective and non-elective admissions), and for A&E.

CONCLUSION

This study has confirmed that the best demand estimates arise from different forecasting methods and forecasting periods (ie, one size does not fit all). Despite the fact that the STLF method was applied for the first time, it outperformed traditional time series forecasting methods (ie, ARIMA and exponential smoothing) for a number of specialties.

PRACTISE IMPLICATIONS

Knowing the peaks and troughs of demand for an entire hospital will enable the management to (a) effectively plan ahead; (b) ensure necessary resources are in place (eg, beds and staff); (c) better manage budgets, ensuring enough cash is available; and (d) reduce risk.