Predicting Patient Deterioration: A Review of Tools in the Digital Hospital Setting

AI-enabled electrocardiography alert intervention and all-cause mortality: a pragmatic randomized clinical trial

The early identification of vulnerable patients has the potential to improve outcomes but poses a substantial challenge in clinical practice. This study evaluated the ability of an artificial intelligence (AI)-enabled electrocardiogram (ECG) to identify hospitalized patients with a high risk of mortality in a multisite randomized controlled trial involving 39 physicians and 15,965 patients. The AI-ECG alert intervention included an AI report and warning messages delivered to the physicians, flagging patients predicted to be at high risk of mortality. The trial met its primary outcome, finding that implementation of the AI-ECG alert was associated with a significant reduction in all-cause mortality within 90 days: 3.6% patients in the intervention group died within 90 days, compared to 4.3% in the control group (4.3%) (hazard ratio (HR) = 0.83, 95% confidence interval (CI) = 0.70-0.99). A prespecified analysis showed that reduction in all-cause mortality associated with the AI-ECG alert was observed primarily in patients with high-risk ECGs (HR = 0.69, 95% CI = 0.53-0.90). In analyses of secondary outcomes, patients in the intervention group with high-risk ECGs received increased levels of intensive care compared to the control group; for the high-risk ECG group of patients, implementation of the AI-ECG alert was associated with a significant reduction in the risk of cardiac death (0.2% in the intervention arm versus 2.4% in the control arm, HR = 0.07, 95% CI = 0.01-0.56). While the precise means by which implementation of the AI-ECG alert led to decreased mortality are to be fully elucidated, these results indicate that such implementation assists in the detection of high-risk patients, prompting timely clinical care and reducing mortality. ClinicalTrials.gov registration: NCT05118035 .

Machine learning for healthcare that matters: Reorienting from technical novelty to equitable impact

Despite significant technical advances in machine learning (ML) over the past several years, the tangible impact of this technology in healthcare has been limited. This is due not only to the particular complexities of healthcare, but also due to structural issues in the machine learning for healthcare (MLHC) community which broadly reward technical novelty over tangible, equitable impact. We structure our work as a healthcare-focused echo of the 2012 paper "Machine Learning that Matters", which highlighted such structural issues in the ML community at large, and offered a series of clearly defined "Impact Challenges" to which the field should orient itself. Drawing on the expertise of a diverse and international group of authors, we engage in a narrative review and examine issues in the research background environment, training processes, evaluation metrics, and deployment protocols which act to limit the real-world applicability of MLHC. Broadly, we seek to distinguish between machine learning ON healthcare data and machine learning FOR healthcare-the former of which sees healthcare as merely a source of interesting technical challenges, and the latter of which regards ML as a tool in service of meeting tangible clinical needs. We offer specific recommendations for a series of stakeholders in the field, from ML researchers and clinicians, to the institutions in which they work, and the governments which regulate their data access.

Patient outcomes following medical emergency team review on general wards: Development of predictive models

AIM

To develop and internally validate risk prediction models for subsequent clinical deterioration, unplanned ICU admission and death among ward patients following medical emergency team (MET) review.

DESIGN

A retrospective cohort study of 1500 patients who remained on a general ward following MET review at an Australian quaternary hospital.

METHOD

Logistic regression was used to model (1) subsequent MET review within 48 h, (2) unplanned ICU admission within 48 h and (3) hospital mortality. Models included demographic, clinical and illness severity variables. Model performance was evaluated using discrimination and calibration with optimism-corrected bootstrapped estimates. Findings are reported using the TRIPOD guideline for multivariable prediction models for prognosis or diagnosis. There was no patient or public involvement in the development and conduct of this study.

RESULTS

Within 48 h of index MET review, 8.3% (n = 125) of patients had a subsequent MET review, 7.2% (n = 108) had an unplanned ICU admission and in-hospital mortality was 16% (n = 240). From clinically preselected predictors, models retained age, sex, comorbidity, resuscitation limitation, acuity-dependency profile, MET activation triggers and whether the patient was within 24 h of hospital admission, ICU discharge or surgery. Models for subsequent MET review, unplanned ICU admission, and death had adequate accuracy in development and bootstrapped validation samples.

CONCLUSION

Patients requiring MET review demonstrate complex clinical characteristics and the majority remain on the ward after review for deterioration. A risk score could be used to identify patients at risk of poor outcomes after MET review and support general ward clinical decision-making.

RELEVANCE TO CLINICAL PRACTICE

Our risk calculator estimates risk for patient outcomes following MET review using clinical data available at the bedside. Future validation and implementation could support evidence-informed team communication and patient placement decisions.

Systematic review and longitudinal analysis of implementing Artificial Intelligence to predict clinical deterioration in adult hospitals: what is known and what remains uncertain

OBJECTIVE

To identify factors influencing implementation of machine learning algorithms (MLAs) that predict clinical deterioration in hospitalized adult patients and relate these to a validated implementation framework.

MATERIALS AND METHODS

A systematic review of studies of implemented or trialed real-time clinical deterioration prediction MLAs was undertaken, which identified: how MLA implementation was measured; impact of MLAs on clinical processes and patient outcomes; and barriers, enablers and uncertainties within the implementation process. Review findings were then mapped to the SALIENT end-to-end implementation framework to identify the implementation stages at which these factors applied.

RESULTS

Thirty-seven articles relating to 14 groups of MLAs were identified, each trialing or implementing a bespoke algorithm. One hundred and seven distinct implementation evaluation metrics were identified. Four groups reported decreased hospital mortality, 1 significantly. We identified 24 barriers, 40 enablers, and 14 uncertainties and mapped these to the 5 stages of the SALIENT implementation framework.

DISCUSSION

Algorithm performance across implementation stages decreased between in silico and trial stages. Silent plus pilot trial inclusion was associated with decreased mortality, as was the use of logistic regression algorithms that used less than 39 variables. Mitigation of alert fatigue via alert suppression and threshold configuration was commonly employed across groups.

CONCLUSIONS

: There is evidence that real-world implementation of clinical deterioration prediction MLAs may improve clinical outcomes. Various factors identified as influencing success or failure of implementation can be mapped to different stages of implementation, thereby providing useful and practical guidance for implementers.

Information displays for automated surveillance algorithms of in-hospital patient deterioration: a scoping review

OBJECTIVE

Surveillance algorithms that predict patient decompensation are increasingly integrated with clinical workflows to help identify patients at risk of in-hospital deterioration. This scoping review aimed to identify the design features of the information displays, the types of algorithm that drive the display, and the effect of these displays on process and patient outcomes.

MATERIALS AND METHODS

The scoping review followed Arksey and O'Malley's framework. Five databases were searched with dates between January 1, 2009 and January 26, 2022. Inclusion criteria were: participants-clinicians in inpatient settings; concepts-intervention as deterioration information displays that leveraged automated AI algorithms; comparison as usual care or alternative displays; outcomes as clinical, workflow process, and usability outcomes; and context as simulated or real-world in-hospital settings in any country. Screening, full-text review, and data extraction were reviewed independently by 2 researchers in each step. Display categories were identified inductively through consensus.

RESULTS

Of 14 575 articles, 64 were included in the review, describing 61 unique displays. Forty-one displays were designed for specific deteriorations (eg, sepsis), 24 provided simple alerts (ie, text-based prompts without relevant patient data), 48 leveraged well-accepted score-based algorithms, and 47 included nurses as the target users. Only 1 out of the 10 randomized controlled trials reported a significant effect on the primary outcome.

CONCLUSIONS

Despite significant advancements in surveillance algorithms, most information displays continue to leverage well-understood, well-accepted score-based algorithms. Users' trust, algorithmic transparency, and workflow integration are significant hurdles to adopting new algorithms into effective decision support tools.

A Delphi Process to Identify Relevant Outcomes That May Be Associated With a Predictive Analytic Tool to Detect Hemodynamic Deterioration in the Intensive Care Unit

Background The critical care literature has seen an increase in the development and validation of tools using artificial intelligence for early detection of patient events or disease onset in the intensive care unit (ICU). The hemodynamic stability index (HSI) was found to have an AUC of 0.82 in predicting the need for hemodynamic intervention in the ICU. Future studies using this tool may benefit from targeting those outcomes that are more relevant to clinicians and most achievable. Methods A three-round Delphi study was conducted with a panel of 10 critical care physicians and three nurses in the United States to identify outcomes that may be most relevant and achievable with the HSI when evaluated for use in the ICU. To achieve criteria for relevance, at least 65% of panelists had to rate an outcome as a 4 or 5 on a 5-point scale. Results Nineteen of 24 outcomes that may be associated with the HSI achieved consensus for relevance. The Kemeny-Young approach was used to develop a matrix depicting the distribution of outcomes considering both relevance and achievability. "Reduces time spent in hemodynamic instability" and "reduces times to recognition of hemodynamic instability" were the highest-ranking outcomes considering both relevance and achievability. Conclusion This Delphi study was a feasible method to identify relevant outcomes that may be associated with an appropriate predictive analytic tool in the ICU. These findings can provide insight to researchers looking to study such tools to impact outcomes relevant to critical care practitioners. Future studies should test these tools in the ICU that target the most clinically relevant and achievable outcomes, such as time spent hemodynamically unstable or time until actionable nursing assessment or treatment.

Six-Lead Electrocardiography Enables Identification of Rhythm and Conduction Anomalies of Patients in the Telemedicine-Based, Hospital-at-Home Setting: A Prospective Validation Study

BACKGROUND

The hospital-at-home (HAH) model is a viable alternative for conventional in-hospital stays worldwide. Serum electrolyte abnormalities are common in acute patients, especially in those with many comorbidities. Pathologic changes in cardiac electrophysiology pose a potential risk during HAH stays. Periodical electrocardiogram (ECG) tracing is therefore advised, but few studies have evaluated the accuracy and efficiency of compact, self-activated ECG devices in HAH settings. This study aimed to evaluate the reliability of such a device in comparison with a standard 12-lead ECG.

METHODS

We prospectively recruited consecutive patients admitted to the Sheba Beyond Virtual Hospital, in the HAH department, during a 3-month duration. Each patient underwent a 12-lead ECG recording using the legacy device and a consecutive recording by a compact six-lead device. Baseline patient characteristics during hospitalization were collected. The level of agreement between devices was measured by Cohen's kappa coefficient for inter-rater reliability (Ϗ).

RESULTS

Fifty patients were included in the study (median age 80 years, IQR 14). In total, 26 (52%) had electrolyte disturbances. Abnormal D-dimer values were observed in 33 (66%) patients, and 12 (24%) patients had elevated troponin values. We found a level of 94.5% raw agreement between devices with regards to nine of the options included in the automatic read-out of the legacy device. The calculated Ϗ was 0.72, classified as a substantial consensus. The rate of raw consensus regarding the ECG intervals' measurement (PR, RR, and QT) was 78.5%, and the calculated Ϗ was 0.42, corresponding to a moderate level of agreement.

CONCLUSION

This is the first report to our knowledge regarding the feasibility of using a compact, six-lead ECG device in the setting of an HAH to be safe and bearing satisfying agreement level with a legacy, 12-lead ECG device, enabling quick, accessible arrythmia detection in this setting. Our findings bear a promise to the future development of telemedicine-based hospital-at-home methodology.

Clinical Instability Is a Sign of Severity of Illness: A Cohort Study

OBJECTIVES

Test the hypothesis that within patient clinical instability measured by deterioration and improvement in mortality risk over 3-, 6-, 9-, and 12-hour time intervals is indicative of increasing severity of illness.

DESIGN

Analysis of electronic health data from January 1, 2018, to February 29, 2020.

SETTING

PICU and cardiac ICU at an academic children's hospital.

PATIENTS

All PICU patients. Data included descriptive information, outcome, and independent variables used in the Criticality Index-Mortality.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

There were 8,399 admissions with 312 deaths (3.7%). Mortality risk determined every three hours using the Criticality Index-Mortality, a machine learning algorithm calibrated to this hospital. Since the sample sizes were sufficiently large to expect statical differences, we also used two measures of effect size, the proportion of time deaths had greater instability than survivors, and the rank-biserial correlation, to assess the magnitude of the effect and complement our hypothesis tests. Within patient changes were compared for survivors and deaths. All comparisons of survivors versus deaths were less than 0.001. For all time intervals, two measures of effect size indicated that the differences between deaths and survivors were not clinically important. However, the within-patient maximum risk increase (clinical deterioration) and maximum risk decrease (clinical improvement) were both substantially greater in deaths than survivors for all time intervals. For deaths, the maximum risk increase ranged from 11.1% to 16.1% and the maximum decrease ranged from -7.3% to -10.0%, while the median maximum increases and decreases for survivors were all less than ± 0.1%. Both measures of effect size indicated moderate to high clinical importance. The within-patient volatility was greater than 4.5-fold greater in deaths than survivors during the first ICU day, plateauing at ICU days 4-5 at 2.5 greater volatility.

CONCLUSIONS

Episodic clinical instability measured with mortality risk is a reliable sign of increasing severity of illness. Mortality risk changes during four time intervals demonstrated deaths have greater maximum and within-patient clinical instability than survivors. This observation confirms the clinical teaching that clinical instability is a sign of severity of illness.

Early Warning Scores to Support Continuous Wireless Vital Sign Monitoring for Complication Prediction in Patients on Surgical Wards: Retrospective Observational Study

BACKGROUND

Wireless vital sign sensors are increasingly being used to monitor patients on surgical wards. Although early warning scores (EWSs) are the current standard for the identification of patient deterioration in a ward setting, their usefulness for continuous monitoring is unknown.

OBJECTIVE

This study aimed to explore the usability and predictive value of high-rate EWSs obtained from continuous vital sign recordings for early identification of postoperative complications and compares the performance of a sensor-based EWS alarm system with manual intermittent EWS measurements and threshold alarms applied to individual vital sign recordings (single-parameter alarms).

METHODS

Continuous vital sign measurements (heart rate, respiratory rate, blood oxygen saturation, and axillary temperature) collected with wireless sensors in patients on surgical wards were used for retrospective simulation of EWSs (sensor EWSs) for different time windows (1-240 min), adopting criteria similar to EWSs based on manual vital signs measurements (nurse EWSs). Hourly sensor EWS measurements were compared between patients with (event group: 14/46, 30%) and without (control group: 32/46, 70%) postoperative complications. In addition, alarms were simulated for the sensor EWSs using a range of alarm thresholds (1-9) and compared with alarms based on nurse EWSs and single-parameter alarms. Alarm performance was evaluated using the sensitivity to predict complications within 24 hours, daily alarm rate, and false discovery rate (FDR).

RESULTS

The hourly sensor EWSs of the event group (median 3.4, IQR 3.1-4.1) was significantly higher (P<.004) compared with the control group (median 2.8, IQR 2.4-3.2). The alarm sensitivity of the hourly sensor EWSs was the highest (80%-67%) for thresholds of 3 to 5, which was associated with alarm rates of 2 (FDR=85%) to 1.2 (FDR=83%) alarms per patient per day respectively. The sensitivity of sensor EWS-based alarms was higher than that of nurse EWS-based alarms (maximum=40%) but lower than that of single-parameter alarms (87%) for all thresholds. In contrast, the (false) alarm rates of sensor EWS-based alarms were higher than that of nurse EWS-based alarms (maximum=0.6 alarm/patient/d; FDR=80%) but lower than that of single-parameter alarms (2 alarms/patient/d; FDR=84%) for most thresholds. Alarm rates for sensor EWSs increased for shorter time windows, reaching 70 alarms per patient per day when calculated every minute.

CONCLUSIONS

EWSs obtained using wireless vital sign sensors may contribute to the early recognition of postoperative complications in a ward setting, with higher alarm sensitivity compared with manual EWS measurements. Although hourly sensor EWSs provide fewer alarms compared with single-parameter alarms, high false alarm rates can be expected when calculated over shorter time spans. Further studies are recommended to optimize care escalation criteria for continuous monitoring of vital signs in a ward setting and to evaluate the effects on patient outcomes.

Classification of health deterioration by geometric invariants

BACKGROUND AND OBJECTIVES

Prediction of patient deterioration is essential in medical care, and its automation may reduce the risk of patient death. The precise monitoring of a patient's medical state requires devices placed on the body, which may cause discomfort. Our approach is based on the processing of long-term ballistocardiography data, which were measured using a sensory pad placed under the patient's mattress.

METHODS

The investigated dataset was obtained via long-term measurements in retirement homes and intensive care units (ICU). Data were measured unobtrusively using a measuring pad equipped with piezoceramic sensors. The proposed approach focused on the processing methods of the measured ballistocardiographic signals, Cartan curvature (CC), and Euclidean arc length (EAL).

RESULTS

For analysis, 218,979 normal and 216,259 aberrant 2-second samples were collected and classified using a convolutional neural network. Experiments using cross-validation with expert threshold and data length revealed the accuracy, sensitivity, and specificity of the proposed method to be 86.51 CONCLUSIONS: The proposed method provides a unique approach for an early detection of health concerns in an unobtrusive manner. In addition, the suitability of EAL over the CC was determined.

National Early Warning Score Deployment in a Veterans Affairs Facility: A Quality Improvement Initiative and Analysis

Early warning scores are algorithms designed to identify clinical deterioration. Current literature is predominantly in non-Veteran populations. Studies in Veterans are lacking. This study was a prospective quality improvement project deploying and assessing the National Early Warning Score (NEWS) at Kansas City VA Medical Center. Performance of NEWS was assessed as follows: discrimination for predicting a composite outcome of intensive care unit transfer or mortality within 24 hours via area under the receiver operating curve. A total of 4781 Veterans with 142 375 NEWS values were included. The NEWS area under the receiver operating curve for the composite outcome was 0.72 (95% CI, 0.71-0.74), indicating acceptable predictive accuracy. A NEWS of ≥7 was more likely associated with the composite outcome versus <7 (13.6% vs 0.8%; P < 0.001). This is one of the first studies to demonstrate successful deployment of NEWS in a Veteran population, with resultant important implications across the Veterans Health Administration.

Evaluation of machine learning-based models for prediction of clinical deterioration: A systematic literature review

BACKGROUND AND OBJECTIVE

Early identification of patients at risk of deterioration can prevent life-threatening adverse events and shorten length of stay. Although there are numerous models applied to predict patient clinical deterioration, most are based on vital signs and have methodological shortcomings that are not able to provide accurate estimates of deterioration risk. The aim of this systematic review is to examine the effectiveness, challenges, and limitations of using machine learning (ML) techniques to predict patient clinical deterioration in hospital settings.

METHODS

A systematic review was performed in accordance with the Preferred Reporting Items for Systematic Reviews and meta-Analyses (PRISMA) guidelines using EMBASE, MEDLINE Complete, CINAHL Complete, and IEEExplore databases. Citation searching was carried out for studies that met inclusion criteria. Two reviewers used the inclusion/exclusion criteria to independently screen studies and extract data. To address any discrepancies in the screening process, the two reviewers discussed their findings and a third reviewer was consulted as needed to reach a consensus. Studies focusing on use of ML in predicting patient clinical deterioration that were published from inception to July 2022 were included.

RESULTS

A total of 29 primary studies that evaluated ML models to predict patient clinical deterioration were identified. After reviewing these studies, we found that 15 types of ML techniques have been employed to predict patient clinical deterioration. While six studies used a single technique exclusively, several others utilised a combination of classical techniques, unsupervised and supervised learning, as well as other novel techniques. Depending on which ML model was applied and the type of input features, ML models predicted outcomes with an area under the curve from 0.55 to 0.99.

CONCLUSIONS

Numerous ML methods have been employed to automate the identification of patient deterioration. Despite these advancements, there is still a need for further investigation to examine the application and effectiveness of these methods in real-world situations.

Construction and evaluation of networks among multiple postoperative complications

BACKGROUND AND OBJECTIVE

Postoperative complications confer an increased risk of reoperation, prolonged length of hospital stay, and increased mortality. Many studies have attempted to identify the complex associations among complications to preemptively interrupt their progression, but few studies have looked at complications as a whole to reveal and quantify their possible trajectories of progression. The main objective of this study was to construct and quantify the association network among multiple postoperative complications from a comprehensive perspective to elucidate the possible evolution trajectories.

METHODS

In this study, a Bayesian network model was proposed to analyze the associations among 15 complications. Prior evidence and score-based hill-climbing algorithms were used to build the structure. The severity of complications was graded according to their connection to death, with the association between them quantified using conditional probabilities. The data of surgical inpatients used in this study were collected from four regionally representative academic/teaching hospitals in a prospective cohort study in China.

RESULTS

In the network obtained, 15 nodes represented complications or death, and 35 arcs with arrows represented the directly dependent relationship between them. With three grades classified on that basis, the correlation coefficients of complications within grades increased with increased grade, ranging from -0.11 to -0.06, 0.16, and 0.21 to 0.4 in grade 1 to grade 3, respectively. Moreover, the probability of each complication in the network increased with the occurrence of any other complication, even mild complications. Most seriously, once cardiac arrest requiring cardiopulmonary resuscitation occurs, the probability of death will be up to 88.1%.

CONCLUSIONS

The present evolving network can facilitate the identification of strong associations among specific complications and provides a basis for the development of targeted measures to prevent further deterioration in high-risk patients.

Assessment of Remote Vital Sign Monitoring and Alarms in a Real-World Healthcare at Home Dataset

The importance of vital sign monitoring to detect deterioration increases during healthcare at home. Continuous monitoring with wearables increases assessment frequency but may create information overload for clinicians. The goal of this work was to demonstrate the impact of vital sign observation frequency and alarm settings on alarms in a real-world dataset. Vital signs were collected from 76 patients admitted to healthcare at home programs using the Current Health (CH) platform; its wearable continuously measured respiratory rate (RR), pulse rate (PR), and oxygen saturation (SpO₂). Total alarms, alarm rate, patient rate, and detection time were calculated for three alarm rulesets to detect changes in SpO₂, PR, and RR under four vital sign observation frequencies and four window sizes for the alarm algorithms' median filter. Total alarms ranged from 65 to 3113. The alarm rate and early detection increased with the observation frequency for all alarm rulesets. Median filter windows reduced alarms triggered by normal fluctuations in vital signs without compromising the granularity of time between assessments. Frequent assessments enabled with continuous monitoring support early intervention but need to pair with settings that balance sensitivity, specificity, clinical risk, and provider capacity to respond when a patient is home to minimize clinician burden.

Protocol describing a systematic review and mixed methods consensus process to define the deteriorated ward patient

INTRODUCTION

Most patients admitted to hospital recover with treatments that can be administered on the general ward. A small but important group deteriorate however and require augmented organ support in areas with increased nursing to patient ratios. In observational studies evaluating this cohort, proxy outcomes such as unplanned intensive care unit admission, cardiac arrest and death are used. These outcome measures introduce subjectivity and variability, which in turn hinders the development and accuracy of the increasing numbers of electronic medical record (EMR) linked digital tools designed to predict clinical deterioration. Here, we describe a protocol for developing a new outcome measure using mixed methods to address these limitations.

METHODS AND ANALYSIS

We will undertake firstly, a systematic literature review to identify existing generic, syndrome-specific and organ-specific definitions for clinically deteriorated, hospitalised adult patients. Secondly, an international modified Delphi study to generate a short list of candidate definitions. Thirdly, a nominal group technique (NGT) (using a trained facilitator) will take a diverse group of stakeholders through a structured process to generate a consensus definition. The NGT process will be informed by the data generated from the first two stages. The definition(s) for the deteriorated ward patient will be readily extractable from the EMR.

ETHICS AND DISSEMINATION

This study has ethics approval (reference 16399) from the Central Adelaide Local Health Network Human Research Ethics Committee. Results generated from this study will be disseminated through publication and presentation at national and international scientific meetings.

Early Prediction of All-Cause Clinical Deterioration in General Wards Patients: Development and Validation of a Biomarker-Based Machine Learning Model Derived From Rapid Response Team Activations

OBJECTIVE

The aim of the study is to evaluate the performance of a biomarker-based machine learning (ML) model (not including vital signs) derived from reviewed rapid response team (RRT) activations in predicting all-cause deterioration in general wards patients.

DESIGN

This is a retrospective single-institution study. All consecutive adult patients' cases on noncritical wards identified by RRT calls occurring at least 24 hours after patient admission, between April 2018 and June 2020, were included. The cases were reviewed and labeled for clinical deterioration by a multidisciplinary expert consensus panel. A supervised learning approach was adopted based on a set of biomarkers and demographic data available in the patient's electronic medical record (EMR).

SETTING

The setting is a 250-bed tertiary university hospital with a basic EMR, with adult (>18 y) patients on general wards.

PATIENTS

The study analyzed the cases of 514 patients for which the RRT was activated. Rapid response teams were extracted from the hospital telephone log data. Two hundred eighteen clinical deterioration cases were identified in these patients after expert chart review and complemented by 146 "nonevent" cases to build the training and validation data set.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

The best performance was achieved with the random forests algorithm, with a maximal area under the receiver operating curve of 0.90 and F1 score of 0.85 obtained at prediction time T0-6h, slightly decreasing but still acceptable (area under the receiver operating curve, >0.8; F1 score, >0.75) at T0-42h. The system outperformed most classical track-and-trigger systems both in terms of prediction performance and prediction horizon.

CONCLUSIONS

In hospitals with a basic EMR, a biomarker-based ML model could be used to predict clinical deterioration in general wards patients earlier than classical track-and-trigger systems, thus enabling appropriate clinical interventions for patient safety and improved outcomes.

Explainable machine learning for real-time deterioration alert prediction to guide pre-emptive treatment

The Electronic Medical Record (EMR) provides an opportunity to manage patient care efficiently and accurately. This includes clinical decision support tools for the timely identification of adverse events or acute illnesses preceded by deterioration. This paper presents a machine learning-driven tool developed using real-time EMR data for identifying patients at high risk of reaching critical conditions that may demand immediate interventions. This tool provides a pre-emptive solution that can help busy clinicians to prioritize their efforts while evaluating the individual patient risk of deterioration. The tool also provides visualized explanation of the main contributing factors to its decisions, which can guide the choice of intervention. When applied to a test cohort of 18,648 patient records, the tool achieved 100% sensitivity for prediction windows 2–8 h in advance for patients that were identified at 95%, 85% and 70% risk of deterioration.

How technology can save lives in cardiac arrest

PURPOSE OF REVIEW

Technology is being increasingly implemented in the fields of cardiac arrest and cardiopulmonary resuscitation. In this review, we describe how recent technological advances have been implemented in the chain of survival and their impact on outcomes after cardiac arrest. Breakthrough technologies that are likely to make an impact in the future are also presented.

RECENT FINDINGS

Technology is present in every link of the chain of survival, from prediction, prevention, and rapid recognition of cardiac arrest to early cardiopulmonary resuscitation and defibrillation. Mobile phone systems to notify citizen first responders of nearby out-of-hospital cardiac arrest have been implemented in numerous countries with improvement in bystanders' interventions and outcomes. Drones delivering automated external defibrillators and artificial intelligence to support the dispatcher in recognising cardiac arrest are already being used in real-life out-of-hospital cardiac arrest. Wearables, smart speakers, surveillance cameras, and artificial intelligence technologies are being developed and studied to prevent and recognize out-of-hospital and in-hospital cardiac arrest.

SUMMARY

This review highlights the importance of technology applied to every single step of the chain of survival to improve outcomes in cardiac arrest. Further research is needed to understand the best role of different technologies in the chain of survival and how these may ultimately improve outcomes.

Artificial intelligence and clinical deterioration

PURPOSE OF REVIEW

To provide an overview of the systems being used to identify and predict clinical deterioration in hospitalised patients, with focus on the current and future role of artificial intelligence (AI).

RECENT FINDINGS

There are five leading AI driven systems in this field: the Advanced Alert Monitor (AAM), the electronic Cardiac Arrest Risk Triage (eCART) score, Hospital wide Alert Via Electronic Noticeboard, the Mayo Clinic Early Warning Score, and the Rothman Index (RI). Each uses Electronic Patient Record (EPR) data and machine learning to predict adverse events. Less mature but relevant evolutions are occurring in the fields of Natural Language Processing, Time and Motion Studies, AI Sepsis and COVID-19 algorithms.

SUMMARY

Research-based AI-driven systems to predict clinical deterioration are increasingly being developed, but few are being implemented into clinical workflows. Escobar et al. (AAM) provide the current gold standard for robust model development and implementation methodology. Multiple technologies show promise, however, the pathway to meaningfully affect patient outcomes remains challenging.

Machine learning models for diabetes management in acute care using electronic medical records: A systematic review

BACKGROUND

Machine learning (ML) is a subset of Artificial Intelligence (AI) that is used to predict and potentially prevent adverse patient outcomes. There is increasing interest in the application of these models in digital hospitals to improve clinical decision-making and chronic disease management, particularly for patients with diabetes. The potential of ML models using electronic medical records (EMR) to improve the clinical care of hospitalised patients with diabetes is currently unknown.

OBJECTIVE

The aim was to systematically identify and critically review the published literature examining the development and validation of ML models using EMR data for improving the care of hospitalised adult patients with diabetes.

METHODS

The Preferred Reporting Items for Systematic Reviews and Meta Analyses (PRISMA) guidelines were followed. Four databases were searched (Embase, PubMed, IEEE and Web of Science) for studies published between January 2010 to January 2022. The reference lists of the eligible articles were manually searched. Articles that examined adults and both developed and validated ML models using EMR data were included. Studies conducted in primary care and community care settings were excluded. Studies were independently screened and data was extracted using Covidence® systematic review software. For data extraction and critical appraisal, the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS) was followed. Risk of bias was assessed using the Prediction model Risk Of Bias Assessment Tool (PROBAST). Quality of reporting was assessed by adherence to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guideline. The IJMEDI checklist was followed to assess quality of ML models and the reproducibility of their outcomes. The external validation methodology of the studies was appraised.

RESULTS

Of the 1317 studies screened, twelve met inclusion criteria. Eight studies developed ML models to predict disglycaemic episodes for hospitalized patients with diabetes, one study developed a ML model to predict total insulin dosage, two studies predicted risk of readmission, and one study improved the prediction of hospital readmission for inpatients with diabetes. All included studies were heterogeneous with regard to ML types, cohort, input predictors, sample size, performance and validation metrics and clinical outcomes. Two studies adhered to the TRIPOD guideline. The methodological reporting of all the studies was evaluated to be at high risk of bias. The quality of ML models in all studies was assessed as poor. Robust external validation was not performed on any of the studies. No models were implemented or evaluated in routine clinical care.

CONCLUSIONS

This review identified a limited number of ML models which were developed to improve inpatient management of diabetes. No ML models were implemented in real hospital settings. Future research needs to enhance the development, reporting and validation steps to enable ML models for integration into routine clinical care.