A machine learning system to optimise triage in an adult ophthalmic emergency department: a model development and validation study

Background

A substantial proportion of attendances to ophthalmic emergency departments are for non-urgent presentations. We developed and evaluated a machine learning system (DemDx Ophthalmology Triage System: DOTS) to optimise triage, with the aim of reducing inappropriate emergency attendances and streamlining case referral when necessary.

Methods

DOTS was built using retrospective tabular data from 11,315 attendances between July 1st, 2021, to June 15th, 2022 at Moorfields Eye Hospital Emergency Department (MEH) in London, UK. Demographic and clinical features were used as inputs and a triage recommendation was given ("see immediately", "see within a week", or "see electively"). DOTS was validated temporally and compared with triage nurses' performance (1269 attendances at MEH) and validated externally (761 attendances at the Federal University of Minas Gerais - UFMG, Brazil). It was also tested for biases and robustness to variations in disease incidences. All attendances from patients aged at least 18 years with at least one confirmed diagnosis were included in the study.

Findings

For identifying ophthalmic emergency attendances, on temporal validation, DOTS had a sensitivity of 94.5% [95% CI 92.3-96.1] and a specificity of 42.4% [38.8-46.1]. For comparison within the same dataset, triage nurses had a sensitivity of 96.4% [94.5-97.7] and a specificity of 25.1% [22.0-28.5]. On external validation at UFMG, DOTS had a sensitivity of 95.2% [92.5-97.0] and a specificity of 32.2% [27.4-37.0]. In simulated scenarios with varying disease incidences, the sensitivity was ≥92.2% and the specificity was ≥36.8%. No differences in sensitivity were found in subgroups of index of multiple deprivation, but the specificity was higher for Q2 when compared to Q4 (Q4 is less deprived than Q2).

Interpretation

At MEH, DOTS had similar sensitivity to triage nurses in determining attendance priority; however, with a specificity of 17.3% higher, DOTS resulted in lower rates of patients triaged to be seen immediately at emergency. DOTS showed consistent performance in temporal and external validation, in social-demographic subgroups and was robust to varying relative disease incidences. Further trials are necessary to validate these findings. This system will be prospectively evaluated, considering human-computer interaction, in a clinical trial.

Funding

The Artificial Intelligence in Health and Care Award (AI_AWARD01671) of the NHS AI Lab under National Institute for Health and Care Research (NIHR) and the Accelerated Access Collaborative (AAC).

Safety of video-based telemedicine compared to in-person triage in emergency ophthalmology during COVID-19

BACKGROUND

the need for social distancing midst the COVID-19 pandemic has forced ophthalmologists to innovate with telemedicine. The novel process of triaging emergency ophthalmology patients via videoconsultations should reduce hospital attendances. However, the safety profile of such services were unknown.

METHODS

in this retrospective cohort study, we reviewed case notes of 404 adults who used our videoconsultation service from 20/04/2020 to 03/05/2020. We compared these to 451 patient who attended eye casualty in person at the same time who were deemed not to require same day ophthalmic examination.

FINDINGS

patients seen by videoconsultations tended to be younger (Median = 43 years, Inter-quartile range = 27 vs Median= 49 years, Inter-quartile range = 28)'. More males used the face-to-face triage (55%) while more females used videoconsultation (54%)%. Fewer patients seen by videoconsultations required specialist review compared to face-face triage [X ² (1, N = 854) = 128.02, p<0.001)]. 35.5% of the patients initially seen by videoconsultation had unplanned reattendance within 1 month, compared to 15.7% in the group initially seen in person. X ² (1, N = 234) = 7.31, p = 0.007). The rate of actual harm was no different (at 0% for each method), with perfect inter-grader correlation when graded independently by two senior ophthalmologists. 97% of patients seen on the video platform surveyed were satisfied with their care.

INTERPRETATION

we demonstrate comparable patient safety of videoconsultations at one-month follow-up to in person review. The service is acceptable to patients and reduces the risk of COVID-19 transmission. We propose that videoconsultations are effective and desirable as a tool for triage in ophthalmology.

FUNDING

the research supported by the National Institute for Health Research (NIHR) Biomedical Research Centre based at Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology who fund PT and DS's time to conduct research. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

Creating the Moorfields' virtual eye casualty: video consultations to provide emergency teleophthalmology care during and beyond the COVID-19 pandemic

Background

The COVID-19 crisis forced hospitals in the UK dramatically to reduce outpatient activity. To provide continuity of care and to assist patients reluctant or unable to leave their homes, video consultations were rapidly implemented across routine and emergency ophthalmology services.

Objective

To describe the deployment and scaling to a large volume of teleophthalmology using a video consultation platform ‘Attend Anywhere’ in Moorfields Eye Hospital’s accident and emergency (A&E) department (London, UK).

Method

Patient satisfaction, waiting time, consultation duration, outcome and management were audited following the launch of the new virtual A&E service.

Results

In the 12 days following the service launch, 331 patients were seen by video consultation. 78.6% of patients (n=260) were determined not to need hospital A&E review and were managed with advice (n=126), remote prescription (n=57), general practitioner referral (n=27), direct referral to hospital subspecialty services (n=26) or diversion to a local eye unit (n=24). Mean patient satisfaction was 4.9 of 5.0 (n=62). The mean consultation duration was 12 min (range 5–31 min) and the wait time was 6 min (range 0–37 min).

Conclusion

Video consultations showed greater than expected usefulness in the remote management of eye disease and supported a substantial reduction in the number of people visiting the hospital.