Automated Detection of Patient Harm: Implementation and Prospective Evaluation of a Real-Time Broad-Spectrum Surveillance Application in a Hospital With Limited Resources

OBJECTIVES

This study aimed to prospectively validate an application that automates the detection of broad categories of hospital adverse events (AEs) extracted from a basic hospital information system, and to efficiently mobilize resources to reduce the level of acquired patient harm.

METHODS

Data were collected from an internally designed software, extracting results from 14 triggers indicative of patient harm, querying clinical and administrative databases including all inpatient admissions (n = 8760) from October 2019 to June 2020. Representative samples of the triggered cases were clinically validated using chart review by a consensus expert panel. The positive predictive value (PPV) of each trigger was evaluated, and the detection sensitivity of the surveillance system was estimated relative to incidence ranges in the literature.

RESULTS

The system identified 394 AEs among 946 triggered cases, associated with 291 patients, yielding an overall PPV of 42%. Variability was observed among the trigger PPVs and among the estimated detection sensitivities across the harm categories, the highest being for the healthcare-associated infections. The median length of stay of patients with an AE showed to be significantly higher than the median for the overall patient population.

CONCLUSIONS

This application was able to identify AEs across a broad spectrum of harm categories, in a real-time manner, while reducing the use of resources required by other harm detection methods. Such a system could serve as a promising patient safety tool for AE surveillance, allowing for timely, targeted, and resource-efficient interventions, even for hospitals with limited resources.

Early Prediction of All-Cause Clinical Deterioration in General Wards Patients: Development and Validation of a Biomarker-Based Machine Learning Model Derived From Rapid Response Team Activations

OBJECTIVE

The aim of the study is to evaluate the performance of a biomarker-based machine learning (ML) model (not including vital signs) derived from reviewed rapid response team (RRT) activations in predicting all-cause deterioration in general wards patients.

DESIGN

This is a retrospective single-institution study. All consecutive adult patients' cases on noncritical wards identified by RRT calls occurring at least 24 hours after patient admission, between April 2018 and June 2020, were included. The cases were reviewed and labeled for clinical deterioration by a multidisciplinary expert consensus panel. A supervised learning approach was adopted based on a set of biomarkers and demographic data available in the patient's electronic medical record (EMR).

SETTING

The setting is a 250-bed tertiary university hospital with a basic EMR, with adult (>18 y) patients on general wards.

PATIENTS

The study analyzed the cases of 514 patients for which the RRT was activated. Rapid response teams were extracted from the hospital telephone log data. Two hundred eighteen clinical deterioration cases were identified in these patients after expert chart review and complemented by 146 "nonevent" cases to build the training and validation data set.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

The best performance was achieved with the random forests algorithm, with a maximal area under the receiver operating curve of 0.90 and F1 score of 0.85 obtained at prediction time T0-6h, slightly decreasing but still acceptable (area under the receiver operating curve, >0.8; F1 score, >0.75) at T0-42h. The system outperformed most classical track-and-trigger systems both in terms of prediction performance and prediction horizon.

CONCLUSIONS

In hospitals with a basic EMR, a biomarker-based ML model could be used to predict clinical deterioration in general wards patients earlier than classical track-and-trigger systems, thus enabling appropriate clinical interventions for patient safety and improved outcomes.

Evaluation of a Broad-Spectrum Partially Automated Adverse Event Surveillance System: A Potential Tool for Patient Safety Improvement in Hospitals With Limited Resources

OBJECTIVE

The aim of the study was to evaluate the sensitivity and resource efficiency of a partially automated adverse event (AE) surveillance system for routine patient safety efforts in hospitals with limited resources.

METHODS

Twenty-eight automated triggers from the hospital information system's clinical and administrative databases identified cases that were then filtered by exclusion criteria per trigger and then reviewed by an interdisciplinary team. The system, developed and implemented using in-house resources, was applied for 45 days of surveillance, for all hospital inpatient admissions (N = 1107). Each trigger was evaluated for its positive predictive value (PPV). Furthermore, the sensitivity of the surveillance system (overall and by AE category) was estimated relative to incidence ranges in the literature.

RESULTS

The surveillance system identified a total of 123 AEs among 283 reviewed medical records, yielding an overall PPV of 52%. The tool showed variable levels of sensitivity across and within AE categories when compared with the literature, with a relatively low overall sensitivity estimated between 21% and 44%. Adverse events were detected in 23 of the 36 AE categories defined by an established harm classification system. Furthermore, none of the detected AEs were voluntarily reported.

CONCLUSIONS

The surveillance system showed variable sensitivity levels across a broad range of AE categories with an acceptable PPV, overcoming certain limitations associated with other harm detection methods. The number of cases captured was substantial, and none had been previously detected or voluntarily reported. For hospitals with limited resources, this methodology provides valuable safety information from which interventions for quality improvement can be formulated.

Comparison of Machine Learning Algorithms for Classifying Adverse-Event Related 30-Day Hospital Readmissions: Potential Implications for Patient Safety

Studies in the last decade have focused on identifying patients at risk of readmission using predictive models, in an objective to decrease costs to the healthcare system. However, real-time models specifically identifying readmissions related to hospital adverse-events are still to be elaborated. A supervised learning approach was adopted using different machine learning algorithms based on features available directly from the hospital information system and on a validated dataset elaborated by a multidisciplinary expert consensus panel. Accuracy results upon testing were in line with comparable studies, and variable across algorithms, with the highest prediction given by Artificial Neuron Networks. Features importances relative to the prediction were identified, in order to provide better representation and interpretation of results. Such a model can pave the way to predictive models for readmissions related to patient harm, the establishment of a learning platform for clinical quality measurement and improvement, and in some cases for an improved clinical management of readmitted patients.

Abstract 1189: Combination treatments with retinoic acid and the synthetic retinoid ST1926 in 2D and 3D breast cancer models overcome retinoic acid resistance and eradicate breast cancer stem/progenitor cells

Despite recent advances in breast cancer therapy, achieving complete remission in metastatic breast cancer patients remains a challenge. Retinoids are crucial regulators of cellular proliferation, differentiation, and cell death, and have shown potent chemotherapeutic and chemopreventive properties. The major drawback of the use of all-trans retinoic acid (ATRA) in cancer therapy is acquired resistance. Therefore, synthetic retinoids such as ST1926 emerged as potent anti-cancer agents. Here, we investigated the anti-tumor activities of ATRA, ST1926, and their combination treatments in 2D and 3D human breast cancer models and their targeting of breast cancer stem cells (CSCs)/progenitor cells. We have shown that in 2D culture models, MCF-7 and MDA-MB-231 cells are resistant to ATRA while being sensitive to ST1926 at sub-micromolar (μM) concentrations in an irreversible manner. Importantly, ST1926 had no effect on the ‘normal-like’ MCF-10A breast epithelial cells. ST1926 induced massive apoptosis in MCF-7 cells and it resulted in S-phase arrest and necrosis in the triple negative and metastatic MDA-MB-231 cells. Furthermore, ST1926 caused early DNA damage, increased the expression of the tumor suppressors p53 and p21, downregulated the Wnt/β-catenin pathway, and modulated the expression levels of the different retinoid receptors. Interestingly, combination treatments as low as 0.1 μM ST1926 and 0.5 μM ATRA synergistically inhibited the proliferation in 2D models of MCF-7 and MDA-MB-231 cells, independently of retinoid receptor signaling, while sparing the normal breast epithelial cells. Anchorage-independent growth of MCF-7 and MDA-MB-231 cells was examined using the soft agar colony formation assay where sub-μM concentrations of ST1926 or μM concentrations of ATRA were shown to reduce the size and the number of breast cancer colonies. ST1926 drastically induced cell death in 3D Matrigel ‘on-top’ assay cultures of breast cancer cells while the lumen of the normal-like breast epithelial cell line S1 was maintained. Finally, treatment with 0.01 μM ST1926 alone or 0.001 μM ST1926 in combination with 0.1 μM ATRA abrogated sphere formation and the self-renewal ability of breast CSCs in the 3D sphere formation assay. In summary, ST1926, ATRA, and their combination treatments were shown to display more potent anti-tumor properties in 3D versus 2D human breast cancer models. Our results also demonstrate the therapeutic potential of ST1926 in sensitizing breast cancer cells to ATRA and in targeting the population of breast CSCs. As 3D culture models are more representative of the tumor microenvironment and serve as valid tools in drug discovery, our results highlight the promising use of ATRA/ST1926 combinations in metastatic and triple negative breast cancers.

Citation Format: Patrick Aouad, Melody Saikali, Rana Abdel-Samad, Leeanna El–Houjeiri, Claudio Pisano, Rabih Talhouk, Nadine Darwiche. Combination treatments with retinoic acid and the synthetic retinoid ST1926 in 2D and 3D breast cancer models overcome retinoic acid resistance and eradicate breast cancer stem/progenitor cells. [abstract]. In: Proceedings of the 107th Annual Meeting of the American Association for Cancer Research; 2016 Apr 16-20; New Orleans, LA. Philadelphia (PA): AACR; Cancer Res 2016;76(14 Suppl):Abstract nr 1189.

Sesquiterpene lactones isolated from indigenous Middle Eastern plants inhibit tumor promoter-induced transformation of JB6 cells

Background

Sesquiterpene lactones (SL) are plant secondary metabolites that are known for their anti-fungal, anti-bacterial, anti-inflammatory, and anti-tumor properties. Considering that several SL-derived drugs are currently in cancer clinical trials, we have tested two SL molecules, 3-β-methoxy-iso-seco-tanapartholide (β-tan) isolated from Achillea falcata and salograviolide A (Sal A) isolated from Centaurea ainetensis, for their anti-tumor properties. We used the mouse epidermal JB6P + cells as a model for tumor promotion and cellular transformation. Key players that are involved in cellular transformation and tumorigenesis are the AP-1 and NF-κB transcription factors; therefore, we assessed how β-tan and Sal A modulate their signaling pathways in JB6P + cells.

Methods

The effects of β-tan and Sal A on the growth of normal and neoplastic keratinocytes and on the tumor promotion-responsive JB6P + cells were determined using the MTT assay. Anchorage-independent cell growth transformation assays were used to evaluate the anti-tumor promoting properties of these SL molecules in JB6P + cells and dual luciferase reporter assays and western blot analysis were used to investigate their effects on tumor promoter-induced AP-1 and NF-κB activities and protein levels of key AP-1 and NF-кB target genes.

Results

β-tan and Sal A selectively inhibited tumor promoter-induced cell growth and transformation of JB6P + cells at concentrations that do not affect JB6P + and primary keratinocytes basal cell growth. In addition, both molecules reduced basal and tumor promoter-induced NF-κB transcriptional activities, differentially regulated basal and tumor promoter-induced AP-1 transcriptional activities, and modulated key players of the AP-1 and NF-κB signaling pathways.

Conclusions

These results highlight the anti-tumor promoting properties of β-tan and Sal A. These SL molecules isolated from two plant species native to the Middle East may provide opportunities for complementary medicine practices.