Want improved quality? Improve your systems

Implementing an artificial intelligence command centre in the NHS: a mixed-methods study

Background

Hospital 'command centres' use digital technologies to collect, analyse and present real-time information that may improve patient flow and patient safety. Bradford Royal Infirmary has trialled this approach and presents an opportunity to evaluate effectiveness to inform future adoption in the United Kingdom.

Objective

To evaluate the impact of the Bradford Command Centre on patient care and organisational processes.

Design

A comparative mixed-methods study. Operational data from a study and control site were collected and analysed. The intervention was observed, and staff at both sites were interviewed. Analysis was grounded in a literature review and the results were synthesised to form conclusions about the intervention.

Setting

The study site was Bradford Royal Infirmary, a large teaching hospital in the city of Bradford, United Kingdom. The control site was Huddersfield Royal Infirmary in the nearby city of Huddersfield.

Participants

Thirty-six staff members were interviewed and/or observed.

Intervention

The implementation of a digitally enabled hospital command centre.

Main outcome measures

Qualitative perspectives on hospital management. Quantitative metrics on patient flow, patient safety, data quality.

Data sources

Anonymised electronic health record data. Ethnographic observations including interviews with hospital staff. Cross-industry review including relevant literature and expert panel interviews.

Results

The Command Centre was implemented successfully and has improved staff confidence of better operational control. Unintended consequences included tensions between localised and centralised decision-making and variable confidence in the quality of data available. The Command Centre supported the hospital through the COVID-19 pandemic, but the direct impact of the Command Centre was difficult to measure as the pandemic forced all hospitals, including the study and control sites, to innovate rapidly. Late in the study we learnt that the control site had visited the study site and replicated some aspects of the command centre themselves; we were unable to explore this in detail. There was no significant difference between pre- and post-intervention periods for the quantitative outcome measures and no conclusive impact on patient flow and data quality. Staff and patients supported the command-centre approaches but patients expressed concern that individual needs might get lost to 'the system'.

Conclusions

Qualitative evidence suggests the Command Centre implementation was successful, but it proved challenging to link quantitative evidence to specific technology interventions. Staff were positive about the benefits and emphasised that these came from the way they adapted to and used the new technology rather than the technology per se.

Limitations

The COVID-19 pandemic disrupted care patterns and forced rapid innovation which reduced our ability to compare study and control sites and data before, during and after the intervention.

Future work

We plan to follow developments at Bradford and in command centres in the National Health Service in order to share learning. Our mixed-methods approach should be of interest to future studies attempting similar evaluation of complex digitally enabled whole-system changes.

Study registration

The study is registered as IRAS No.: 285933.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (NIHR award ref: NIHR129483) and is published in full in Health and Social Care Delivery Research; Vol. 12, No. 41. See the NIHR Funding and Awards website for further award information.

Evaluating the safety and patient impacts of an artificial intelligence command centre in acute hospital care: a mixed-methods protocol

INTRODUCTION

This paper presents a mixed-methods study protocol that will be used to evaluate a recent implementation of a real-time, centralised hospital command centre in the UK. The command centre represents a complex intervention within a complex adaptive system. It could support better operational decision-making and facilitate identification and mitigation of threats to patient safety. There is, however, limited research on the impact of such complex health information technology on patient safety, reliability and operational efficiency of healthcare delivery and this study aims to help address that gap.

METHODS AND ANALYSIS

We will conduct a longitudinal mixed-method evaluation that will be informed by public-and-patient involvement and engagement. Interviews and ethnographic observations will inform iterations with quantitative analysis that will sensitise further qualitative work. Quantitative work will take an iterative approach to identify relevant outcome measures from both the literature and pragmatically from datasets of routinely collected electronic health records.

ETHICS AND DISSEMINATION

This protocol has been approved by the University of Leeds Engineering and Physical Sciences Research Ethics Committee (#MEEC 20-016) and the National Health Service Health Research Authority (IRAS No.: 285933). Our results will be communicated through peer-reviewed publications in international journals and conferences. We will provide ongoing feedback as part of our engagement work with local trust stakeholders.