Use of artificial intelligence in critical care: opportunities and obstacles

BACKGROUND

Perhaps nowhere else in the healthcare system than in the intensive care unit environment are the challenges to create useful models with direct time-critical clinical applications more relevant and the obstacles to achieving those goals more massive. Machine learning-based artificial intelligence (AI) techniques to define states and predict future events are commonplace activities of modern life. However, their penetration into acute care medicine has been slow, stuttering and uneven. Major obstacles to widespread effective application of AI approaches to the real-time care of the critically ill patient exist and need to be addressed.

MAIN BODY

Clinical decision support systems (CDSSs) in acute and critical care environments support clinicians, not replace them at the bedside. As will be discussed in this review, the reasons are many and include the immaturity of AI-based systems to have situational awareness, the fundamental bias in many large databases that do not reflect the target population of patient being treated making fairness an important issue to address and technical barriers to the timely access to valid data and its display in a fashion useful for clinical workflow. The inherent "black-box" nature of many predictive algorithms and CDSS makes trustworthiness and acceptance by the medical community difficult. Logistically, collating and curating in real-time multidimensional data streams of various sources needed to inform the algorithms and ultimately display relevant clinical decisions support format that adapt to individual patient responses and signatures represent the efferent limb of these systems and is often ignored during initial validation efforts. Similarly, legal and commercial barriers to the access to many existing clinical databases limit studies to address fairness and generalizability of predictive models and management tools.

CONCLUSIONS

AI-based CDSS are evolving and are here to stay. It is our obligation to be good shepherds of their use and further development.

Using rapid response system trigger clusters to characterize patterns of clinical deterioration among hospitalized adult patients

BACKGROUND

Many rapid response system (RRS) events are activated using multiple triggers. However, the patterns in which multiple RRS triggers occur together to activate RRS events are unknown. The purpose of this study was to identify these patterns (RRS trigger clusters) and determine their association with outcomes among hospitalized adult patients.

METHODS

RRS events among adult patients from January 2015 to December 2019 in the Get With The Guidelines- Resuscitation registry's MET module were examined (n = 134,406). Cluster analysis methods were performed to identify RRS trigger clusters. Pearson's chi-squared and ANOVA tests were used to examine differences in patient characteristics across RRS trigger clusters. Multilevel logistic regressions were used to examine the associations between RRS trigger clusters and outcomes.

RESULTS

Six RRS trigger clusters were identified. Predominant RRS triggers for each cluster were: tachypnea, new onset difficulty in breathing, decreased oxygen saturation (Cluster 1); tachypnea, decreased oxygen saturation, staff concern (Cluster 2); respiratory depression, decreased oxygen saturation, mental status changes (Cluster 3); tachycardia, staff concern (Cluster 4); mental status changes (Cluster 5); hypotension, staff concern (Cluster 6). Significant differences in patient characteristics were observed across clusters. Patients in Clusters 3 and 6 had an increased likelihood of in-hospital cardiac arrest (p < 0.01). All clusters had an increased risk of mortality (p < 0.01).

CONCLUSIONS

We discovered six novel RRS trigger clusters with differing relationships to adverse patient outcomes. RRS trigger clusters may prove crucial in clarifying the associations between RRS events and adverse outcomes and aiding in clinician decision-making during RRS events.

National Early Warning Score Deployment in a Veterans Affairs Facility: A Quality Improvement Initiative and Analysis

Early warning scores are algorithms designed to identify clinical deterioration. Current literature is predominantly in non-Veteran populations. Studies in Veterans are lacking. This study was a prospective quality improvement project deploying and assessing the National Early Warning Score (NEWS) at Kansas City VA Medical Center. Performance of NEWS was assessed as follows: discrimination for predicting a composite outcome of intensive care unit transfer or mortality within 24 hours via area under the receiver operating curve. A total of 4781 Veterans with 142 375 NEWS values were included. The NEWS area under the receiver operating curve for the composite outcome was 0.72 (95% CI, 0.71-0.74), indicating acceptable predictive accuracy. A NEWS of ≥7 was more likely associated with the composite outcome versus <7 (13.6% vs 0.8%; P < 0.001). This is one of the first studies to demonstrate successful deployment of NEWS in a Veteran population, with resultant important implications across the Veterans Health Administration.

Using rapid response system trigger clusters to characterize patterns of clinical deterioration among hospitalized adult patients

Background

Many rapid response system (RRS) events are activated using multiple triggers. However, the patterns in which RRS triggers co-occur to activate the medical emergency team (MET) to respond to RRS events is unknown. The purpose of this study was to identify and describe the patterns (RRS trigger clusters) in which RRS triggers co-occur when used to activate the MET and determine the association of these clusters with outcomes using a sample of hospitalized adult patients.

Methods

RRS events among adult patients from January 2015 to December 2019 in the Get With The Guidelines- Resuscitation registry's MET module were examined (n=134,406). A combination of cluster analyses methods was performed to group patients into RRS trigger clusters based on the triggers used to activate their RRS events. Pearson's chi-squared and ANOVA tests were used to examine differences in patient characteristics across RRS trigger clusters. Multilevel logistic regression was used to examine the associations between RRS trigger clusters and outcomes following RRS events.

Results

Six RRS trigger clusters were identified in the study sample. The RRS triggers that predominantly identified each cluster were as follows: tachypnea, new onset difficulty in breathing, and decreased oxygen saturation (Cluster 1); tachypnea, decreased oxygen saturation, and staff concern (Cluster 2); respiratory depression, decreased oxygen saturation, and mental status changes (Cluster 3); tachycardia and staff concern (Cluster 4); mental status changes (Cluster 5); hypotension and staff concern (Cluster 6). Significant differences in patient characteristics were observed across RRS trigger clusters. Patients in Clusters 3 and 6 were associated with an increased likelihood of in-hospital cardiac arrest (IHCA [p<0.01]), while Cluster 4 was associated with a decreased likelihood of IHCA (p<0.01). All clusters were associated with an increased risk of mortality (p<0.01).

Conclusions

We discovered six novel RRS trigger clusters with differing relationships to adverse patient outcomes following RRS events. RRS trigger clusters may prove crucial in clarifying the associations between RRS events and adverse outcomes and may aid in clinician decision-making during RRS events.

Predicting outcome in acute respiratory admissions using patterns of National Early Warning Scores

AIMS

Accurately predicting risk of patient deterioration is vital. Altered physiology in chronic disease affects the prognostic ability of vital signs based early warning score systems. We aimed to assess the potential of early warning score patterns to improve outcome prediction in patients with respiratory disease.

METHODS

Patients admitted under respiratory medicine between April 2015 and March 2017 had their National Early Warning Score 2 (NEWS2) calculated retrospectively from vital sign observations. Prediction models (including temporal patterns) were constructed and assessed for ability to predict death within 24 hours using all observations collected not meeting exclusion criteria. The best performing model was tested on a validation cohort of admissions from April 2017 to March 2019.

RESULTS

The derivation cohort comprised 7,487 admissions and the validation cohort included 8,739 admissions. Adding the maximum score in the preceding 24 hours to the most recently recorded NEWS2 improved area under the receiver operating characteristic curve for death in 24 hours from 0.888 (95% confidence interval (CI) 0.881-0.895) to 0.902 (95% CI 0.895-0.909) in the overall respiratory population.

CONCLUSION

Combining the most recently recorded score and the maximum NEWS2 score from the preceding 24 hours demonstrated greater accuracy than using snapshot NEWS2. This simple inclusion of a scoring pattern should be considered in future iterations of early warning scoring systems.

Predicting neurological outcomes after in-hospital cardiac arrests for patients with Coronavirus Disease 2019

Background

Machine learning models are more accurate than standard tools for predicting neurological outcomes in patients resuscitated after cardiac arrest. However, their accuracy in patients with Coronavirus Disease 2019 (COVID-19) is unknown. Therefore, we compared their performance in a cohort of cardiac arrest patients with COVID-19.

Methods

We conducted a retrospective analysis of resuscitation survivors in the Get With The Guidelines®-Resuscitation (GWTG-R) COVID-19 registry between February 2020 and May 2021. The primary outcome was a favorable neurological outcome, indicated by a discharge Cerebral Performance Category score ≤ 2. Pre- and peri-arrest variables were used as predictors. We applied our published logistic regression, neural network, and gradient boosted machine models developed in patients without COVID-19 to the COVID-19 cohort. We also updated the neural network model using transfer learning. Performance was compared between models and the Cardiac Arrest Survival Post-Resuscitation In-Hospital (CASPRI) score.

Results

Among the 4,125 patients with COVID-19 included in the analysis, 484 (12 %) patients survived with favorable neurological outcomes. The gradient boosted machine, trained on non-COVID-19 patients was the best performing model for predicting neurological outcomes in COVID-19 patients, significantly better than the CASPRI score (c-statistic: 0.75 vs 0.67, P < 0.001). While calibration improved for the neural network with transfer learning, it did not surpass the gradient boosted machine in terms of discrimination.

Conclusion

Our gradient boosted machine model developed in non-COVID patients had high discrimination and adequate calibration in COVID-19 resuscitation survivors and may provide clinicians with important information for these patients.

Predicting survival after surgery for pancreatic adenocarcinoma: Testing accuracy of current models

e16286

Background: Pancreatic adenocarcinoma (PDAC) remains highly morbid, and outcomes are difficult to prognosticate. Multiple models have also been developed to predict survival following surgical resection of PDAC, but their clinical utility remains unclear. This study aims to determine the accuracy of these algorithms for predicting PDAC survival. Methods: We performed a retrospective analysis using a de-identified dataset of patients who received neoadjuvant chemotherapy and underwent surgical resection of PDAC at four academic medical centers across the United States between 2010 and 2020. For this analysis the prognostic accuracy of the Memorial Sloan Kettering Cancer Center Pancreatic Adenocarcinoma Nomogram (MSKCCPAN) and the American Joint Committee on Cancer (AJCC) staging system were evaluated. For the MSKCCPAN, the prognostic index (PI) for each patient was calculated using the nomogram’s underlying Cox model. The PI was then used as input for the model’s survival function, which calculates the probability of DSS at 12-, 24-, and 36-month intervals. For the AJCC staging system, we assessed statistical discrimination between stage and DSS. We evaluated the concordance between predicted and actual DSS of both algorithms using the Uno C-statistic. Results: A total of 303 patients with complete information were included in the study, with 155 (51.2%) female patients and a mean age of 65.0 years (SD, 9.2 years). At the time of surgery, 47 (15.5%) patients had AJCC Stage IA disease, 45 (14.9%) had Stage IB, 49 (16.2%) had Stage IIA, 105 (34.6%) had Stage IIB, and 57 (18.8%) had Stage III. The median follow-up time of the cohort was 37.7 months (IQR, 16.4-70.7). One hundred eleven (36.6%) patients were alive at last follow-up, while 155 (51.2%) patients had died of disease. Thirty-seven (12.2%) patients died of another cause. The Kaplan-Meier estimated 12-, 24-, and 36-month DSS of the cohort was 74.3% (95% CI, 69.1-79.9), 46.6% (95% CI, 40.3-53.9) and 39.3% (95% CI, 32.9-46.9), respectively. The MSKCCPAN-predicted DSS at 12, 24, and 36 months was 78.1% (95% CI, 76.1-80.0), 56.4% (95% CI, 53.7-59.0), and 45.0% (95%0.781 CI, 42.3-47.6). For the MSKCCPAN, the calculated Uno C-statistic was 0.61 at 12 months and 0.62 at both 24 and 36 months. For the AJCC staging system, the Uno C-statistic was 0.61 at 12 months and 0.60 at both 24 months and 36 months. Conclusions: The present analysis suggests that concordance of current predictive models and staging systems with real world survival in patients with PDAC undergoing surgical resection after neoadjuvant chemotherapy has limited accuracy. Improving risk stratification of patients with PDAC will afford greater opportunities for tailoring patient selection, precision medicine initiatives, and trial design. There is an urgent need to develop novel models that better prognosticate outcomes following surgical resection of PDAC.

Validation and utility of ARDS subphenotypes identified by machine-learning models using clinical data: an observational, multicohort, retrospective analysis

BACKGROUND

Two acute respiratory distress syndrome (ARDS) subphenotypes (hyperinflammatory and hypoinflammatory) with distinct clinical and biological features and differential treatment responses have been identified using latent class analysis (LCA) in seven individual cohorts. To facilitate bedside identification of subphenotypes, clinical classifier models using readily available clinical variables have been described in four randomised controlled trials. We aimed to assess the performance of these models in observational cohorts of ARDS.

METHODS

In this observational, multicohort, retrospective study, we validated two machine-learning clinical classifier models for assigning ARDS subphenotypes in two observational cohorts of patients with ARDS: Early Assessment of Renal and Lung Injury (EARLI; n=335) and Validating Acute Lung Injury Markers for Diagnosis (VALID; n=452), with LCA-derived subphenotypes as the gold standard. The primary model comprised only vital signs and laboratory variables, and the secondary model comprised all predictors in the primary model, with the addition of ventilatory variables and demographics. Model performance was assessed by calculating the area under the receiver operating characteristic curve (AUC) and calibration plots, and assigning subphenotypes using a probability cutoff value of 0·5 to determine sensitivity, specificity, and accuracy of the assignments. We also assessed the performance of the primary model in EARLI using data automatically extracted from an electronic health record (EHR; EHR-derived EARLI cohort). In Large Observational Study to Understand the Global Impact of Severe Acute Respiratory Failure (LUNG SAFE; n=2813), a multinational, observational ARDS cohort, we applied a custom classifier model (with fewer variables than the primary model) to determine the prognostic value of the subphenotypes and tested their interaction with the positive end-expiratory pressure (PEEP) strategy, with 90-day mortality as the dependent variable.

FINDINGS

The primary clinical classifier model had an area under receiver operating characteristic curve (AUC) of 0·92 (95% CI 0·90-0·95) in EARLI and 0·88 (0·84-0·91) in VALID. Performance of the primary model was similar when using exclusively EHR-derived predictors compared with manually curated predictors (AUC=0·88 [95% CI 0·81-0·94] vs 0·92 [0·88-0·97]). In LUNG SAFE, 90-day mortality was higher in patients assigned the hyperinflammatory subphenotype than in those with the hypoinflammatory phenotype (414 [57%] of 725 vs 694 [33%] of 2088; p<0·0001). There was a significant treatment interaction with PEEP strategy and ARDS subphenotype (p=0·041), with lower 90-day mortality in the high PEEP group of patients with the hyperinflammatory subphenotype (hyperinflammatory subphenotype: 169 [54%] of 313 patients in the high PEEP group vs 127 [62%] of 205 patients in the low PEEP group; hypoinflammatory subphenotype: 231 [34%] of 675 patients in the high PEEP group vs 233 [32%] of 734 patients in the low PEEP group).

INTERPRETATION

Classifier models using clinical variables alone can accurately assign ARDS subphenotypes in observational cohorts. Application of these models can provide valuable prognostic information and could inform management strategies for personalised treatment, including application of PEEP, once prospectively validated.

FUNDING

US National Institutes of Health and European Society of Intensive Care Medicine.

Association of COVID-19 Infection With Survival After In-Hospital Cardiac Arrest Among US Adults

This cohort study examines the association of COVID-19 infection with survival outcomes of US adults after in-hospital cardiac arrest.

The Addition of United States Census-Tract Data Does Not Improve the Prediction of Substance Misuse

Predictors from the structured data in the electronic health record (EHR) have previously been used for case-identification in substance misuse. We aim to examine the added benefit from census-tract data, a proxy for socioeconomic status, to improve identification. A cohort of 186,611 hospitalizations was derived between 2007 and 2017. Reference labels included alcohol misuse only, opioid misuse only, and both alcohol and opioid misuse. Baseline models were created using 24 EHR variables, and enhanced models were created with the addition of 48 census-tract variables from the United States American Community Survey. The absolute net reclassification index (NRI) was applied to measure the benefit in adding census-tract variables to baseline models. The baseline models already had good calibration and discrimination. Adding census-tract variables provided negligible improvement to sensitivity and specificity and NRI was less than 1% across substance groups. Our results show the census-tract added minimal value to prediction models.

The impact of vaccination to control COVID-19 burden in the United States: A simulation modeling approach

INTRODUCTION

Vaccination programs aim to control the COVID-19 pandemic. However, the relative impacts of vaccine coverage, effectiveness, and capacity in the context of nonpharmaceutical interventions such as mask use and physical distancing on the spread of SARS-CoV-2 are unclear. Our objective was to examine the impact of vaccination on the control of SARS-CoV-2 using our previously developed agent-based simulation model.

METHODS

We applied our agent-based model to replicate COVID-19-related events in 1) Dane County, Wisconsin; 2) Milwaukee metropolitan area, Wisconsin; 3) New York City (NYC). We evaluated the impact of vaccination considering the proportion of the population vaccinated, probability that a vaccinated individual gains immunity, vaccination capacity, and adherence to nonpharmaceutical interventions. We estimated the timing of pandemic control, defined as the date after which only a small number of new cases occur.

RESULTS

The timing of pandemic control depends highly on vaccination coverage, effectiveness, and adherence to nonpharmaceutical interventions. In Dane County and Milwaukee, if 50% of the population is vaccinated with a daily vaccination capacity of 0.25% of the population, vaccine effectiveness of 90%, and the adherence to nonpharmaceutical interventions is 60%, controlled spread could be achieved by June 2021 versus October 2021 in Dane County and November 2021 in Milwaukee without vaccine.

DISCUSSION

In controlling the spread of SARS-CoV-2, the impact of vaccination varies widely depending not only on effectiveness and coverage, but also concurrent adherence to nonpharmaceutical interventions.

Sharing ICU Patient Data Responsibly Under the Society of Critical Care Medicine/European Society of Intensive Care Medicine Joint Data Science Collaboration: The Amsterdam University Medical Centers Database (AmsterdamUMCdb) Example

OBJECTIVES

Critical care medicine is a natural environment for machine learning approaches to improve outcomes for critically ill patients as admissions to ICUs generate vast amounts of data. However, technical, legal, ethical, and privacy concerns have so far limited the critical care medicine community from making these data readily available. The Society of Critical Care Medicine and the European Society of Intensive Care Medicine have identified ICU patient data sharing as one of the priorities under their Joint Data Science Collaboration. To encourage ICUs worldwide to share their patient data responsibly, we now describe the development and release of Amsterdam University Medical Centers Database (AmsterdamUMCdb), the first freely available critical care database in full compliance with privacy laws from both the United States and Europe, as an example of the feasibility of sharing complex critical care data.

SETTING

University hospital ICU.

SUBJECTS

Data from ICU patients admitted between 2003 and 2016.

INTERVENTIONS

We used a risk-based deidentification strategy to maintain data utility while preserving privacy. In addition, we implemented contractual and governance processes, and a communication strategy. Patient organizations, supporting hospitals, and experts on ethics and privacy audited these processes and the database.

MEASUREMENTS AND MAIN RESULTS

AmsterdamUMCdb contains approximately 1 billion clinical data points from 23,106 admissions of 20,109 patients. The privacy audit concluded that reidentification is not reasonably likely, and AmsterdamUMCdb can therefore be considered as anonymous information, both in the context of the U.S. Health Insurance Portability and Accountability Act and the European General Data Protection Regulation. The ethics audit concluded that responsible data sharing imposes minimal burden, whereas the potential benefit is tremendous.

CONCLUSIONS

Technical, legal, ethical, and privacy challenges related to responsible data sharing can be addressed using a multidisciplinary approach. A risk-based deidentification strategy, that complies with both U.S. and European privacy regulations, should be the preferred approach to releasing ICU patient data. This supports the shared Society of Critical Care Medicine and European Society of Intensive Care Medicine vision to improve critical care outcomes through scientific inquiry of vast and combined ICU datasets.

Safety and efficacy of catheter-directed therapy versus anticoagulation alone in a higher-risk acute pulmonary embolism population

There is little data comparing safety and efficacy outcomes in patients with pulmonary embolism (PE) receiving catheter directed therapies (CDT) compared to a similar-risk cohort of PE patients receiving anticoagulation alone. 1094 patients with acute PE were studied. CDT and conservatively-managed patients were compared using propensity score matching to assess safety outcomes, which included bleeding and acute kidney injury at 2 and 7 days after PE diagnosis. Efficacy outcomes included change in vital signs over 72 h and in-hospital mortality. PE patients with RV strain who underwent CDT (n = 76) had more bleeding at 2 days (additional 1.04 g/dL loss, 95% CI − 1.48 to − 0.60, p < 0.001) and 7 days (additional 1.36 g/dL loss, 95% CI − 1.88 to − 0.84, p < 0.001) compared to those receiving anticoagulation alone (n = 303). There was a significant increase in creatinine at 2 days (additional 0.22 mg/dL elevation, 95% CI 0.02 to 0.42, p = 0.03), but not at 7 days (additional 0.12 mg/dL elevation, 95% CI − 0.11 to 0.35, p = 0.30). In-hospital mortality for patients receiving CDT versus anticoagulation alone was similar (OR 1.21, 95% CI 0.53 to 2.77; p = 0.65). In patients with baseline abnormal vital signs who received CDT versus anticoagulation alone, heart rate, respiratory rate and oxygen requirement improved significantly faster and to levels closer to normal (p ≤ 0.001). CDT was associated with a small but increased risk of bleeding, but no significant worsening of renal function. CDT may be associated with more rapid improvements in heart rate, respiratory rate, and oxygen requirement.

Effect of Timing of and Adherence to Social Distancing Measures on COVID-19 Burden in the United States : A Simulation Modeling Approach

BACKGROUND

Across the United States, various social distancing measures were implemented to control the spread of coronavirus disease 2019 (COVID-19). However, the effectiveness of such measures for specific regions with varying population demographic characteristics and different levels of adherence to social distancing is uncertain.

OBJECTIVE

To determine the effect of social distancing measures in unique regions.

DESIGN

An agent-based simulation model.

SETTING

Agent-based model applied to Dane County, Wisconsin; the Milwaukee metropolitan (metro) area; and New York City (NYC).

PATIENTS

Synthetic population at different ages.

INTERVENTION

Different times for implementing and easing social distancing measures at different levels of adherence.

MEASUREMENTS

The model represented the social network and interactions among persons in a region, considering population demographic characteristics, limited testing availability, "imported" infections, asymptomatic disease transmission, and age-specific adherence to social distancing measures. The primary outcome was the total number of confirmed COVID-19 cases.

RESULTS

The timing of and adherence to social distancing had a major effect on COVID-19 occurrence. In NYC, implementing social distancing measures 1 week earlier would have reduced the total number of confirmed cases from 203 261 to 41 366 as of 31 May 2020, whereas a 1-week delay could have increased the number of confirmed cases to 1 407 600. A delay in implementation had a differential effect on the number of cases in the Milwaukee metro area versus Dane County, indicating that the effect of social distancing measures varies even within the same state.

LIMITATION

The effect of weather conditions on transmission dynamics was not considered.

CONCLUSION

The timing of implementing and easing social distancing measures has major effects on the number of COVID-19 cases.

PRIMARY FUNDING SOURCE

National Institute of Allergy and Infectious Diseases.

Predicting clinical deterioration with Q-ADDS compared to NEWS, Between the Flags, and eCART track and trigger tools

BACKGROUND

Early warning tools have been widely implemented without evidence to guide (a) recognition and (b) response team expertise optimisation. With growing databases from MET-calls and digital hospitals, we now have access to guiding information. The Queensland Adult-Deterioration-Detection-System (Q-ADDS) is widely used and requires validation.

AIM

Compare the accuracy of Q-ADDS to National Early Warning Score (NEWS), Between-the-Flags (BTF) and the electronic Cardiac Arrest Risk Triage Score (eCART)).

METHODS

Data from the Chicago University hospital database were used. Clinical deterioration was defined as unplanned admission to ICU or death. Currently used NEWS, BTF and eCART trigger thresholds were compared with a clinically endorsed Q-ADDS variant.

RESULTS

Of 224,912 admissions, 11,706 (5%) experienced clinical deterioration. Q-ADDS (AUC 0.71) and NEWS (AUC 0.72) had similar predictive accuracy, BTF (AUC 0.64) had the lowest, and eCART (AUC 0.76) the highest. Early warning alert (advising ward MO review) had similar NPV (99.2-99.3%), for all the four tools however sensitivity varied (%: Q-ADDS = 47/NEWS = 49/BTF = 66/eCART = 40), as did alerting rate (% vitals sets: Q-ADDS = 1.4/NEWS = 3.5/BTF = 4.1/eCART = 3.4). MET alert (advising MET/critical-care review) had similar NPV for all the four tools (99.1-99.2%), however sensitivity varied (%: Q-ADDS = 14/NEWS = 24/BTF = 19/eCART = 29), as did MET alerting rate (%: Q-ADDS = 1.4/NEWS = 3.5/BTF = 4.1/eCART = 3.4). High-severity alert (advising advanced ward review, Q-ADDS only): NPV = 99.1%, sensitivity = 26%, alerting rate = 3.5%.

CONCLUSION

The accuracy of Q-ADDS is comparable to NEWS, and higher than BTF, with eCART being the most accurate. Q-ADDS provides an additional high-severity ward alert, and generated significantly fewer MET alerts. Impacts of increased ward awareness and fewer MET alerts on actual MET call numbers and patient outcomes requires further evaluation.

Impact of Timing of and Adherence to Social Distancing Measures on COVID-19 Burden in the US: A Simulation Modeling Approach

Background

Across the U.S., various social distancing measures were implemented to control COVID-19 pandemic. However, there is uncertainty in the effectiveness of such measures for specific regions with varying population demographics and different levels of adherence to social distancing. The objective of this paper is to determine the impact of social distancing measures in unique regions.

Methods

We developed COVid-19 Agent-based simulation Model (COVAM), an agent-based simulation model (ABM) that represents the social network and interactions among the people in a region considering population demographics, limited testing availability, imported infections from outside of the region, asymptomatic disease transmission, and adherence to social distancing measures. We adopted COVAM to represent COVID-19-associated events in Dane County, Wisconsin, Milwaukee metropolitan area, and New York City (NYC). We used COVAM to evaluate the impact of three different aspects of social distancing: 1) Adherence to social distancing measures; 2) timing of implementing social distancing; and 3) timing of easing social distancing.

Results

We found that the timing of social distancing and adherence level had a major effect on COVID-19 occurrence. For example, in NYC, implementing social distancing measures on March 5, 2020 instead of March 12, 2020 would have reduced the total number of confirmed cases from 191,984 to 43,968 as of May 30, whereas a 1-week delay in implementing such measures could have increased the number of confirmed cases to 1,299,420. Easing social distancing measures on June 1, 2020 instead of June 15, 2020 in NYC would increase the total number of confirmed cases from 275,587 to 379,858 as of July 31.

Conclusion

The timing of implementing social distancing measures, adherence to the measures, and timing of their easing have major effects on the number of COVID-19 cases.

Primary Funding Source

National Institute of Allergy and Infectious Diseases Institute.

Combining patient visual timelines with deep learning to predict mortality

BACKGROUND

Deep learning algorithms have achieved human-equivalent performance in image recognition. However, the majority of clinical data within electronic health records is inherently in a non-image format. Therefore, creating visual representations of clinical data could facilitate using cutting-edge deep learning models for predicting outcomes such as in-hospital mortality, while enabling clinician interpretability. The objective of this study was to develop a framework that first transforms longitudinal patient data into visual timelines and then utilizes deep learning to predict in-hospital mortality.

METHODS AND FINDINGS

All adult consecutive patient admissions from 2008-2016 at a tertiary care center were included in this retrospective study. Two-dimensional visual representations for each patient were created with clinical variables on one dimension and time on the other. Predictors included vital signs, laboratory results, medications, interventions, nurse examinations, and diagnostic tests collected over the first 48 hours of the hospital stay. These visual timelines were utilized by a convolutional neural network with a recurrent layer model to predict in-hospital mortality. Seventy percent of the cohort was used for model derivation and 30% for independent validation. Of 115,825 hospital admissions, 2,926 (2.5%) suffered in-hospital mortality. Our model predicted in-hospital mortality significantly better than the Modified Early Warning Score (area under the receiver operating characteristic curve [AUC]: 0.91 vs. 0.76, P < 0.001) and the Sequential Organ Failure Assessment score (AUC: 0.91 vs. 0.57, P < 0.001) in the independent validation set. Class-activation heatmaps were utilized to highlight areas of the picture that were most important for making the prediction, thereby providing clinicians with insight into each individual patient's prediction.

CONCLUSIONS

We converted longitudinal patient data into visual timelines and applied a deep neural network for predicting in-hospital mortality more accurately than current standard clinical models, while allowing for interpretation. Our framework holds promise for predicting several important outcomes in clinical medicine.

Annual Incidence of Adult and Pediatric In-Hospital Cardiac Arrest in the United States

Background:

Previous incidence estimates may no longer reflect the current public health burden of cardiac arrest in hospitalized adult and pediatric patients across the United States. The aim of this study was to estimate the contemporary annual incidence of in-hospital cardiac arrest in adults and children across the United States and to describe trends in incidence between 2008 and 2017.

Methods and Results:

Using the Get With The Guidelines–Resuscitation registry, we developed a negative binomial regression model to estimate the incidence of index pulseless in-hospital cardiac arrest based on hospital-level characteristics. The model was used to predict the number of in-hospital cardiac arrests in all US hospitals, using data from the American Hospital Association Annual Survey. We performed separate analyses for adult (≥18 years) and pediatric (<18 years) cardiac arrests. Additional analyses were performed for recurrent cardiac arrests and pediatric patients requiring cardiopulmonary resuscitation for poor perfusion (nonpulseless events). The average annual incidence of in-hospital cardiac arrest in the United States was estimated at 292 000 (95% prediction interval, 217 600–503 500) adult and 15 200 pediatric cases, of which 7100 (95% prediction interval, 4400–9900) cases were pulseless cardiac arrests and 8100 (95% prediction interval, 4700–11 500) cases were nonpulseless events. The rate of adult cardiac arrests increased over time, while pediatric events remained more stable. When including both index and recurrent in-hospital cardiac arrests, the average annual incidence was estimated at 357 900 (95% prediction interval, 247 100–598 400) adult and 19 900 pediatric cases, of which 8300 (95% prediction interval, 4900–11 200) cases were pulseless cardiac arrests and 11 600 (95% prediction interval, 6400–16 700) cases were nonpulseless events.

Conclusions:

There are ≈292 000 adult in-hospital cardiac arrests and 15 200 pediatric in-hospital events in the United States each year. This study provides contemporary estimates of the public health burden of cardiac arrest among hospitalized patients.

Comparison of the Between the Flags calling criteria to the MEWS, NEWS and the electronic Cardiac Arrest Risk Triage (eCART) score for the identification of deteriorating ward patients

INTRODUCTION

Traditionally, paper based observation charts have been used to identify deteriorating patients, with emerging recent electronic medical records allowing electronic algorithms to risk stratify and help direct the response to deterioration.

OBJECTIVE(S)

We sought to compare the Between the Flags (BTF) calling criteria to the Modified Early Warning Score (MEWS), National Early Warning Score (NEWS) and electronic Cardiac Arrest Risk Triage (eCART) score.

DESIGN AND PARTICIPANTS

Multicenter retrospective analysis of electronic health record data from all patients admitted to five US hospitals from November 2008-August 2013.

MAIN OUTCOME MEASURES

Cardiac arrest, ICU transfer or death within 24h of a score RESULTS: Overall accuracy was highest for eCART, with an AUC of 0.801 (95% CI 0.799-0.802), followed by NEWS, MEWS and BTF respectively (0.718 [0.716-0.720]; 0.698 [0.696-0.700]; 0.663 [0.661-0.664]). BTF criteria had a high risk (Red Zone) specificity of 95.0% and a moderate risk (Yellow Zone) specificity of 27.5%, which corresponded to MEWS thresholds of >=4 and >=2, NEWS thresholds of >=5 and >=2, and eCART thresholds of >=12 and >=4, respectively. At those thresholds, eCART caught 22 more adverse events per 10,000 patients than BTF using the moderate risk criteria and 13 more using high risk criteria, while MEWS and NEWS identified the same or fewer.

CONCLUSION(S)

An electronically generated eCART score was more accurate than commonly used paper based observation tools for predicting the composite outcome of in-hospital cardiac arrest, ICU transfer and death within 24h of observation. The outcomes of this analysis lend weight for a move towards an algorithm based electronic risk identification tool for deteriorating patients to ensure earlier detection and prevent adverse events in the hospital.

Phenotypic Clusters Predict Outcomes in a Longitudinal Interstitial Lung Disease Cohort

BACKGROUND

The current interstitial lung disease (ILD) classification has overlapping clinical presentations and outcomes. Cluster analysis modeling is a valuable tool in identifying distinct clinical phenotypes in heterogeneous diseases. However, this approach has yet to be implemented in ILD.

METHODS

Using cluster analysis, novel ILD phenotypes were identified among subjects from a longitudinal ILD cohort, and outcomes were stratified according to phenotypic clusters compared with subgroups according to current American Thoracic Society/European Respiratory Society ILD classification criteria.

RESULTS

Among subjects with complete data for baseline variables (N = 770), four clusters were identified. Cluster 1 (ie, younger white obese female subjects) had the highest baseline FVC and diffusion capacity of the lung for carbon monoxide (Dlco). Cluster 2 (ie, younger African-American female subjects with elevated antinuclear antibody titers) had the lowest baseline FVC. Cluster 3 (ie, elderly white male smokers with coexistent emphysema) had intermediate FVC and Dlco. Cluster 4 (ie, elderly white male smokers with severe honeycombing) had the lowest baseline Dlco. Compared with classification according to ILD subgroup, stratification according to phenotypic clusters was associated with significant differences in monthly FVC decline (Cluster 4, -0.30% vs Cluster 2, 0.01%; P < .0001). Stratification by using clusters also independently predicted progression-free survival (P < .001) and transplant-free survival (P < .001).

CONCLUSIONS

Among adults with diverse chronic ILDs, cluster analysis using baseline characteristics identified four distinct clinical phenotypes that might better predict meaningful clinical outcomes than current ILD diagnostic criteria.

Outcomes of immunosuppressive therapy in chronic hypersensitivity pneumonitis

In chronic hypersensitivity pneumonitis (CHP), lack of improvement or declining lung function may prompt use of immunosuppressive therapy. We hypothesised that use of azathioprine or mycophenolate mofetil with prednisone reduces adverse events and lung function decline, and improves transplant-free survival.

Patients with CHP were identified. Demographic features, pulmonary function tests, incidence of treatment-emergent adverse events (TEAEs) and transplant-free survival were characterised, compared and analysed between patients stratified by immunosuppressive therapy. A multicentre comparison was performed across four independent tertiary medical centres.

Among 131 CHP patients at the University of Chicago medical centre (Chicago, IL, USA), 93 (71%) received immunosuppressive therapy, and had worse baseline forced vital capacity (FVC) and diffusing capacity, and increased mortality compared with those who did not. Compared to patients treated with prednisone alone, TEAEs were 54% less frequent with azathioprine therapy (p=0.04) and 66% less frequent with mycophenolate mofetil (p=0.002). FVC decline and survival were similar between treatment groups. Analyses of datasets from four external tertiary medical centres confirmed these findings.

CHP patients who did not receive immunosuppressive therapy had better survival than those who did. Use of mycophenolate mofetil or azathioprine was associated with a decreased incidence of TEAEs, and no difference in lung function decline or survival when compared with prednisone alone. Early transition to mycophenolate mofetil or azathioprine may be an appropriate therapeutic approach in CHP, but more studies are needed.

Early transition to mycophenolate mofetil or azathioprine may be an appropriate therapeutic approach in CHPhttp://ow.ly/kAN130dRIX8

Life Expectancy Predictions for Older Diabetic Patients as Estimated by Physicians and a Prognostic Model

Background: Multiple medical organizations recommend using life expectancy (LE) to individualize diabetes care goals. We compare the performance of patient LE predictions made by physicians to LE predictions from a simulation model (the Chicago model) in a cohort of older diabetic patients. Design: Retrospective cohort study of a convenience sample (n = 447) of diabetes patients over 65 years and their physicians. Measurements: Physicians provided LE estimates for individual patients during a baseline survey (2000–2003). The prognostic model included a comprehensive geriatric type 2 diabetes simulation model (the Chicago model) and combinations of the physician estimate and the Chicago model (“And,” “Or,” and “Average” models). Observed survival was determined based on the National Death Index through 31 December 2010. The predictive accuracy of LE predictions was assessed using c-statistic for 5-year mortality; Harrell’s c-statistic, and Integrated Brier score for overall survival. Results: The patient cohort had a mean (SD) age of 73.4 (5.9) years. The majority were female (62.6%) and black (79.4%). At 5 years, 108 (24.2%) patients had died. The c-statistic for 5-year mortality was similar for physicians (0.69) and the Chicago model (0.68), while the average of estimates by physicians and Chicago model yielded the highest c-statistic of any method tested (0.73). The estimates of overall survival yielded a similar pattern of results. Limitations: Generalizability of patient cohort and lack of updated model parameters. Conclusions: Compared with individual methods, the average of LE estimates by physicians and the Chicago model had the best predictive performance. Prognostic models, such as the Chicago model, may complement and support physicians’ intuitions as they consider treatment decisions and goals for older patients with chronic conditions like diabetes.

Inherited mutations in breast cancer genes in African American breast cancer patients revealed by targeted genomic capture and next-generation sequencing

CRA1501

Background: African American (AA) women are disproportionately affected by early-onset and triple-negative breast cancer (TNBC). One explanation for these disparities may be a higher frequency of inherited mutations among AA women in genes in DNA repair pathways, including BRCA1 and BRCA2. Using targeted genomic capture and next generation sequencing (NGS), we screened DNA from AA women with breast cancer for mutations in all 18 known breast cancer genes. Methods: A total of 249 unrelated AA women with breast cancer were ascertained through the Cancer Risk Clinic at The University of Chicago. Genomic DNA was extracted from peripheral blood and 3 micrograms were used for targeted capture and sequencing. Average read depth across the 1.4 MB targeted region was 320-fold. Sequence reads were aligned and all classes of variants identified: point mutations, small insertions and deletions, and large genomic rearrangements. Only unambiguously damaging mutations were called: stops, complete genomic deletions, and missenses demonstrated experimentally to cause loss of protein function. Variants were validated by PCR or Taqman analysis. Results: Fifty-six of 249 subjects (22%) carried at least one loss-of-function mutation, distributed among BRCA1 (n=26), BRCA2 (n=20), CHEK2 (n=3), PALB2 (n=3), ATM (n=5), and PTEN (n=1). The majority of mutations were unique. Damaging mutations were carried by 30% of patients with TNBC, 27% of patients diagnosed at age ≤45, 49% with a second breast primary, and 30% with a family history of either breast or ovarian cancer in any close relative. Conclusions: We present the first comprehensive screen of all known breast cancer susceptibility genes among AA women using NGS. Mutation carrier frequencies are >25% for major subsets of patients defined by tumor or host characteristics. These high carrier frequencies suggest the importance of screening for mutations in all breast cancer genes in all AA breast cancer patients diagnosed at a young age, with a family history, or with TNBC as a way to identify at-risk family members for life-saving interventions.

Inherited mutations in breast cancer genes in African American breast cancer patients revealed by targeted genomic capture and next generation sequencing

CRA1501

The full, final text of this abstract will be available at abstract.asco.org at 7:30 AM (EDT) on Monday, June, 3, 2013, and in the Annual Meeting Proceedings online supplement to the June 20, 2013, issue of Journal of Clinical Oncology. Onsite at the Meeting, this abstract will be printed in the Monday edition of ASCO Daily News.