Skåne Emergency Department Assessment of Patient Load (SEAL)-A Model to Estimate Crowding Based on Workload in Swedish Emergency Departments

Artificial intelligence-driven forecasting and shift optimization for pediatric emergency department crowding

Objective

This study aimed to develop and evaluate an artificial intelligence (AI)-driven system for forecasting Pediatric Emergency Department (PED) overcrowding and optimizing physician shift schedules using machine learning operations (MLOps).

Materials and Methods

Data from 352 843 PED admissions between January 2018 and May 2023 were analyzed. Twenty time-series forecasting models-including classical methods and advanced deep learning architectures like Temporal Convolutional Network, Time-series Dense Encoder and Reversible Instance Normalization, Neural High-order Time Series model, and Neural Basis Expansion Analysis-were developed and compared using Python 3.8. Starting in January 2023, an MLOps simulation automated data updates and model retraining. Shift schedules were optimized based on forecasted patient volumes using integer linear programming.

Results

Advanced deep learning models outperformed traditional models, achieving initial R2 scores up to 75%. Throughout the simulation, the median R2 score for all models was 44% after MLOps-based model selection, the median R2 improved to 60%. The MLOps architecture facilitated continuous model updates, enhancing forecast accuracy. Shift optimization adjusted staffing in 69 out of 84 shifts, increasing physician allocation by up to 30.4% during peak hours. This adjustment reduced the patient-to-physician ratio by an average of 4.32 patients during the 8-16 shift and 4.40 patients during the 16-24 shift.

Discussion

The integration of advanced deep learning models with MLOps architecture allowed for continuous model updates, enhancing the accuracy of PED overcrowding forecasts and outperforming traditional methods. The AI-driven system demonstrated resilience against data drift caused by events like the COVID-19 pandemic, adapting to changing conditions. Optimizing physician shifts based on these forecasts improved workforce distribution without increasing staff numbers, reducing patient load per physician during peak hours. However, limitations include the single-center design and a fixed staffing model, indicating the need for multicenter validation and implementation in settings with dynamic staffing practices. Future research should focus on expanding datasets through multicenter collaborations and developing forecasting models that provide longer lead times without compromising accuracy.

Conclusions

The AI-driven forecasting and shift optimization system demonstrated the efficacy of integrating AI and MLOps in predicting PED overcrowding and optimizing physician shifts. This approach outperformed traditional methods, highlighting its potential for managing overcrowding in emergency departments. Future research should focus on multicenter validation and real-world implementation to fully leverage the benefits of this innovative system.

Quantifying emergency department nursing workload at the task level using NASA-TLX: An exploratory descriptive study

BACKGROUND

Emergency departments (ED) nurses experience high mental workloads because of unpredictable work environments; however, research evaluating ED nursing workload using a tool incorporating nurses' perception is lacking. Quantify ED nursing subjective workload and explore the impact of work experience on perceived workload.

METHODS

Thirty-two ED nurses at a tertiary academic hospital in the Republic of Korea were surveyed to assess their subjective workload for ED procedures using the National Aeronautics and Space Administration Task Load Index (NASA-TLX). Nonparametric statistical analysis was performed to describe the data, and linear regression analysis was conducted to estimate the impact of work experience on perceived workload.

RESULTS

Cardiopulmonary resuscitation (CPR) had the highest median workload, followed by interruption from a patient and their family members. Although inexperienced nurses perceived the 'special care' procedures (CPR and defibrillation) as more challenging compared with other categories, analysis revealed that nurses with more than 107 months of experience reported a significantly higher workload than those with less than 36 months of experience.

CONCLUSION

Addressing interruptions and customizing training can alleviate ED nursing workload. Quantified perceived workload is useful for identifying acceptable thresholds to maintain optimal workload, which ultimately contributes to predicting nursing staffing needs and ED crowding.

Emergency department crowding and mortality: an observational multicenter study in Sweden

Background

Emergency department (ED) crowding is a serious problem worldwide causing decreased quality of care. It is reasonable to assume that the negative effects of crowding are at least partially due to high staff workload, but previous crowding metrics based on high workload have not been generalisable to Swedish EDs and have not been associated with increased mortality, in contrast to, e.g., occupancy rate. We recently derived and validated the modified Skåne Emergency Department Assessment of Patient Load model (mSEAL) that measures crowding based on staff workload in Swedish EDs, but its ability to identify situations with increased mortality is unclear. In this study, we aimed to investigate the association between ED crowding measured by mSEAL model, or occupancy rate, and mortality.

Methods

All ED patients from 2017-01-01 to 2017-06-30 from two regional healthcare systems (Skåne and Östergötland Counties with a combined population of approximately 1.8 million) in Sweden were included. Exposure was ED- and hour-adjusted mSEAL or occupancy rate. Primary outcome was mortality within 7 days of ED arrival, with one-day and 30-day mortality as secondary outcomes. We used Cox regression hazard ratio (HR) adjusted for age, sex, arrival by ambulance, hospital admission and chief complaint.

Results

We included a total of 122,893 patients with 168,900 visits to the six participating EDs. Arriving at an hour with a mSEAL score above the 95th percentile for that ED and hour of day was associated with an non-significant HR for death at 7 days of 1.04 (95% CI 0.96-1.13). For one- and 30-day mortality the HR was non-significant at 1.03 (95% CI 0.9-1.18) and 1.03 (95% CI 0.97-1.09). Similarly, occupancy rate above the 95th percentile with a HR of 1.04 (95% CI 0.9-1.19), 1.03 (95%CI 0.95-1.13) and 1.04 (95% CI 0.98-1.11) for one-, 7- and 30-day mortality, respectively.

Conclusion

In this multicenter study in Sweden, ED crowding measured by mSEAL or occupancy rate was not associated with a significant increase in short-term mortality.

Wilhelms D. Differentiating properties of occupancy rate and workload to estimate crowding: A Swedish national cross-sectional study

BACKGROUND

Emergency department (ED) crowding causes increased patient morbidity and mortality. ED occupancy rate (OR; patients by treatment beds) is a common measure of crowding, but the comparability of ORs between EDs is unknown. The objective of this investigation was to investigate differences in ORs between EDs using staff-perceived workload as reference.

METHODS

This was a national cross-sectional study in Sweden. EDs provided data on census, treatment beds, staffing, and workload (1-6) at 5 time points. A baseline patient turnover was calculated as the average daily census by treatment beds, denoted turnover per treatment bed (TTB), for each ED. A census ratio (CR), current by daily census, was calculated to adjust for differences in the number of treatment beds.

RESULTS

Data were returned from 37 (51%) EDs. TTB varied considerably (mean = 4, standard deviation = 1.6; range, 2.1-9.2), and the OR was higher in EDs with TTB >4 compared with ≤4, 0.86 versus 0.43 (0.43; 95% confidence interval [CI], 0.27-0.59), but not workload, 2.75 versus 2.52 (0.23; 95% CI, -0.19 to 0.64). After adjusting for confounders, both TTB (k = -0.3; 95% CI, -0.49 to -0.14) and OR (k = 3.4; 95% CI, 1.76-5.03) affected workload. Correlation with workload was better for CR than for OR (r = 0.75 vs 0.60, respectively).

CONCLUSION

OR is affected by patient-to-treatment bed ratios that differ significantly between EDs and should be accounted for when measuring crowding. CR is not affected by baseline treatment beds and is a better comparable measure of crowding compared with OR in this national comparator study.

Real-Time Measurement of Crowding in Pediatric Emergency Department: Derivation and Validation Using Consensual Perception of Crowding (SOTU-PED)

ABSTRACT

Our study aimed to develop and validate a real-time crowding composite scale for pediatric emergency department (PED). The study took place in one teaching PED for 2 months. The outcome was the perception of crowding evaluated by triage nurses and pediatricians on a 10-level Likert scale. Triage nurses evaluated crowding at each moment of a child's admission and pediatrician at each moment of a child's discharge. The outcome was the hourly mean of all evaluations of crowding (hourly crowding perception). For analysis, originally, we only selected hours during which more than 2 nurses and more than 2 pediatricians evaluated crowding and, moreover, during which evaluations were the most consensual. As predictors, we used hourly means of 10 objective crowding indicators previously selected as consensual in a published French national Delphi study and collected automatically in our software system. The model (SOTU-PED) was developed over a 1-month data set using a backward multivariable linear regression model. Then, we applied the SOTU-PED model on a 1-month validation data set. During the study period, 7341 children were admitted in the PED. The outcome was available for 1352/1392 hours, among which 639 were included in the analysis as "consensual hours." Five indicators were included in the final model, the SOTU-PED (R2 = 0.718). On the validation data set, the correlation between the outcome (perception of crowding) and the SOTU-PED was 0.824. To predict crowded hours (hourly crowding perception >5), the area under the curve was 0.957 (0.933-0.980). The positive and negative likelihood ratios were 8.16 (3.82-17.43) and 0.153 (0.111-0.223), respectively. Using a simple model, it is possible to estimate in real time how crowded a PED is.

Machine learning techniques for mortality prediction in emergency departments: a systematic review

OBJECTIVES

This systematic review aimed to assess the performance and clinical feasibility of machine learning (ML) algorithms in prediction of in-hospital mortality for medical patients using vital signs at emergency departments (EDs).

DESIGN

A systematic review was performed.

SETTING

The databases including Medline (PubMed), Scopus and Embase (Ovid) were searched between 2010 and 2021, to extract published articles in English, describing ML-based models utilising vital sign variables to predict in-hospital mortality for patients admitted at EDs. Critical appraisal and data extraction for systematic reviews of prediction modelling studies checklist was used for study planning and data extraction. The risk of bias for included papers was assessed using the prediction risk of bias assessment tool.

PARTICIPANTS

Admitted patients to the ED.

MAIN OUTCOME MEASURE

In-hospital mortality.

RESULTS

Fifteen articles were included in the final review. We found that eight models including logistic regression, decision tree, K-nearest neighbours, support vector machine, gradient boosting, random forest, artificial neural networks and deep neural networks have been applied in this domain. Most studies failed to report essential main analysis steps such as data preprocessing and handling missing values. Fourteen included studies had a high risk of bias in the statistical analysis part, which could lead to poor performance in practice. Although the main aim of all studies was developing a predictive model for mortality, nine articles did not provide a time horizon for the prediction.

CONCLUSION

This review provided an updated overview of the state-of-the-art and revealed research gaps; based on these, we provide eight recommendations for future studies to make the use of ML more feasible in practice. By following these recommendations, we expect to see more robust ML models applied in the future to help clinicians identify patient deterioration earlier.

Improving Machine Learning 30-Day Mortality Prediction by Discounting Surprising Deaths

BACKGROUND

Machine learning (ML) is an emerging tool for predicting need of end-of-life discussion and palliative care, by using mortality as a proxy. But deaths, unforeseen by emergency physicians at time of the emergency department (ED) visit, might have a weaker association with the ED visit.

OBJECTIVES

To develop an ML algorithm that predicts unsurprising deaths within 30 days after ED discharge.

METHODS

In this retrospective registry study, we included all ED attendances within the Swedish region of Halland in 2015 and 2016. All registered deaths within 30 days after ED discharge were classified as either "surprising" or "unsurprising" by an adjudicating committee with three senior specialists in emergency medicine. ML algorithms were developed for the death subclasses by using Logistic Regression (LR), Random Forest (RF), and Support Vector Machine (SVM).

RESULTS

Of all 30-day deaths (n = 148), 76% (n = 113) were not surprising to the adjudicating committee. The most common diseases were advanced stage cancer, multidisease/frailty, and dementia. By using LR, RF, and SVM, mean area under the receiver operating characteristic curve (ROC-AUC) of unsurprising deaths in the test set were 0.950 (SD 0.008), 0.944 (SD 0.007), and 0.949 (SD 0.007), respectively. For all mortality, the ROC-AUCs for LR, RF, and SVM were 0.924 (SD 0.012), 0.922 (SD 0.009), and 0.931 (SD 0.008). The difference in prediction performance between all and unsurprising death was statistically significant (P < .001) for all three models.

CONCLUSION

In patients discharged to home from the ED, three-quarters of all 30-day deaths did not surprise an adjudicating committee with emergency medicine specialists. When only unsurprising deaths were included, ML mortality prediction improved significantly.

Validation of the modified Skåne emergency department assessment of patient load (mSEAL) model for emergency department crowding and comparison with international models; an observational study

Background

Emergency Department crowding is associated with increased morbidity and mortality but no measure of crowding has been validated in Sweden. We have previously derived and internally validated the Skåne Emergency Department Assessment of Patient Load (SEAL) score as a measure of crowding in Emergency Departments (ED) in a large regional healthcare system in Sweden. Due to differences in electronic health records (EHRs) between health care systems in Sweden, all variables in the original SEAL-score could not be measured reliably nationally. We aimed to derive and validate a modified SEAL (mSEAL) model and to compare it with established international measures of crowding.

Methods

This was an observational cross sectional study at four EDs in Sweden. All clinical staff assessed their workload (1–6 where 6 is the highest workload) at 5 timepoints each day. We used linear regression with stepwise backward elimination on the original SEAL dataset to derive and internally validate the mSEAL score against staff workload assessments. We externally validated the mSEAL at four hospitals and compared it with the National Emergency Department Overcrowding Score (NEDOCS), the simplified International Crowding Measure in Emergency Department (sICMED), and Occupancy Rate. Area under the receiver operating curve (AuROC) and coefficient of determination was used to compare crowding models. Crowding was defined as an average workload of 4.5 or higher.

Results

The mSEAL score contains the variables Patient Hours and Time to physician and showed strong correlation with crowding in the derivation (r² = 0.47), internal validation (r² = 0.64 and 0.69) and in the external validation (r² = 0.48 to 0.60). AuROC scores for crowding in the external validation were 0.91, 0.90, 0.97 and 0.80 for mSEAL, Occupancy Rate, NEDOCS and sICMED respectively.

Conclusions

The mSEAL model can measure crowding based on workload in Swedish EDs with good discriminatory capacity and has the potential to systematically evaluate crowding and help policymakers and researchers target its causes and effects. In Swedish EDs, Occupancy Rate and NEDOCS are good alternatives to measure crowding based on workload.

Prevalence of crowding, boarding and staffing levels in Swedish emergency departments - a National Cross Sectional Study

Background

Emergency Department (ED) crowding occurs when demand for care exceeds the available resources. Crowding has been associated with decreased quality of care and increased mortality, but the prevalence on a national level is unknown in most countries.

Method

We performed a national, cross-sectional study on staffing levels, staff workload, occupancy rate and patients waiting for an in-hospital bed (boarding) at five time points during 24 h in Swedish EDs.

Results

Complete data were collected from 37 (51% of all) EDs in Sweden. High occupancy rate indicated crowding at 12 hospitals (37.5%) at 31 out of 170 (18.2%) time points. Mean workload (measured on a scale from 1, no workload to 6, very high workload) was moderate at 2.65 (±1.25). Boarding was more prevalent in academic EDs than rural EDs (median 3 vs 0). There were an average of 2.6, 4.6 and 3.2 patients per registered nurse, enrolled nurse and physician, respectively.

Conclusion

ED crowding based on occupancy rate was prevalent on a national level in Sweden and comparable with international data. Staff workload, boarding and patient to staff ratios were generally lower than previously described.

Emergency Department Workload and Crowding During a Major Electronic Health Record Breakdown

Background: Emergency Departments (EDs) today rely heavily on Electronic Health Records (EHRs) and associated support systems. EHR updates are known to be associated with adverse events, but reports on the consequences of breakdowns in EDs are lacking.

Objectives: To describe the effects on workload, occupancy, patient Length Of Stay (LOS), and admissions at three EDs (a regional trauma center, a community hospital and a rural community hospital) during a 96 h period of EHR downtime, of which 48 h represented an unexpected breakdown.

Methods: Assessments of workload, on a scale from 1 (no workload) to 6 (very high workload), were obtained from all staff before, during and after the downtime period. Occupancy, LOS and hospital admissions were extracted from data recorded in the fallback system at each ED during the downtime, and compared with the period before and after (uptime).

Results: Workload increased considerably at two EDs during the downtime whereas the third ED lacked resources to assess workload due to the breakdown. The proportion of assessments ≥4 were 28.5% during uptime compared to 38.4% during downtime at the regional trauma center ED (difference 9.9%, p = 0.006, 95% CI 2.7–17%), and 22.9% compared to 41% at the rural community ED (difference 18.1%, p = 0.0002, 95%CI 7.9–28.3%). Median LOS increased by 19 min (3:56 vs. 4:15, p < 0.004) at the regional trauma center ED, by 76 min (3:34 vs. 4:50, p < 0.001) at the community ED and was unaltered at the rural community ED (2:47 vs. 2:51, p = 0.3) during downtime. Occupancy increased significantly at the community ED (1.59 vs. 0.71, p < 0.0001). Admissions rates remained unchanged during the breakdown. Fallback systems and initiatives to manage the effects of the breakdown differed between the EDs.

Conclusions: EHR downtime or unexpected breakdowns increased staff workload, and had variable effects on ED crowding as measured by LOS and occupancy. Additional staff and digital fallback systems may reduce the effects on ED crowding, but this descriptive study cannot determine causality.

Significant changes in emergency department length of stay and case mix over eight years at a large Swedish University Hospital

OBJECTIVE

Describe the longitudinal development of crowding and patient/emergency department (ED) characteristics at a Swedish University Hospital.

METHODS

A retrospective longitudinal registry study based on all ED visits with adult patients during 2009-2016 (N = 1,063,806). Patient characteristics and measures of ED crowding (ED occupancy ratio, length-of-stay [LOS], patients/clinician's ratios) were extracted from the hospital's electronic health record. Non-parametric analyses were conducted.

RESULTS

The proportion of unstable patients (triage level 1-2) increased while the proportion of admitted patients decreased. All crowding variables were stable, except for LOS, which increased by 9 min/visit/year (95% CI: 8.8-9.1). LOS for visits by patients ≥ 80 years increased more compared to those 18-79 (248 min vs. 190 min, p < 0.001). Unstable patients increased their median LOS compared to stable patients (triage level 3-5). LOS for discharged patients increased with an average of 7.7 min/year (95% CI: 7.5-7.9) compared to 15.5 min/year (95% CI: 15.2-15.8) for those being admitted.

CONCLUSION

Fewer admissions, despite an increase of unstable patients, is likely related to lack of in-hospital beds and contributes to ED crowding. The increase in median ED LOS, especially for patients in the subgroups unstable, ≥80 years and admitted to in-hospital care reflects this problem.