Retrospective derivation and validation of a search algorithm to identify emergent endotracheal intubations in the intensive care unit

Validation of automated data abstraction for SCCM discovery VIRUS COVID-19 registry: practical EHR export pathways (VIRUS-PEEP)

Background

The gold standard for gathering data from electronic health records (EHR) has been manual data extraction; however, this requires vast resources and personnel. Automation of this process reduces resource burdens and expands research opportunities.

Objective

This study aimed to determine the feasibility and reliability of automated data extraction in a large registry of adult COVID-19 patients.

Materials and methods

This observational study included data from sites participating in the SCCM Discovery VIRUS COVID-19 registry. Important demographic, comorbidity, and outcome variables were chosen for manual and automated extraction for the feasibility dataset. We quantified the degree of agreement with Cohen's kappa statistics for categorical variables. The sensitivity and specificity were also assessed. Correlations for continuous variables were assessed with Pearson's correlation coefficient and Bland-Altman plots. The strength of agreement was defined as almost perfect (0.81-1.00), substantial (0.61-0.80), and moderate (0.41-0.60) based on kappa statistics. Pearson correlations were classified as trivial (0.00-0.30), low (0.30-0.50), moderate (0.50-0.70), high (0.70-0.90), and extremely high (0.90-1.00).

Measurements and main results

The cohort included 652 patients from 11 sites. The agreement between manual and automated extraction for categorical variables was almost perfect in 13 (72.2%) variables (Race, Ethnicity, Sex, Coronary Artery Disease, Hypertension, Congestive Heart Failure, Asthma, Diabetes Mellitus, ICU admission rate, IMV rate, HFNC rate, ICU and Hospital Discharge Status), and substantial in five (27.8%) (COPD, CKD, Dyslipidemia/Hyperlipidemia, NIMV, and ECMO rate). The correlations were extremely high in three (42.9%) variables (age, weight, and hospital LOS) and high in four (57.1%) of the continuous variables (Height, Days to ICU admission, ICU LOS, and IMV days). The average sensitivity and specificity for the categorical data were 90.7 and 96.9%.

Conclusion and relevance

Our study confirms the feasibility and validity of an automated process to gather data from the EHR.

An Electronic Search Algorithm for Early Disseminated Intravascular Coagulopathy Diagnosis in the Intensive Care Unit: A Derivation and Validation Study

Aim: We aim to create and validate an electronic search algorithm for accurate detection of disseminated intravascular coagulopathy (DIC) from medical records.

Methods: Patients with DIC in Mayo Clinic’s intensive care units (ICUs) from Jan 1, 2007, to May 4, 2018, were included in the study. An algorithm was developed based on clinical notes and ICD diagnosis codes. A cohort of 50 patients was included with DIC diagnosis, its variations, and no diagnosis of DIC. Then, the next set of 50 patients was used to refine the algorithm. Results were compared with a manual reviewer and the disagreements were resolved by the third reviewer. The same process was repeated with 'revised clinical note search' for the first and second derivation cohort with additional exclusion terms. The obtained sensitivity and specificity were reported. The generated algorithm was applied to another set of 50 patients for validation.

Results: In the first derivation cohort- DIC search by clinical notes and diagnosis codes had 92% sensitivity and 100% specificity. Sensitivity dropped to 71% in the second cohort although specificity remains the same. Therefore, the algorithm was refined to clinical notes search only. The revised search was reapplied to first and second derivation cohorts and results obtained for the first derivation were the same but 91.3% sensitive and 100% specific for the second derivation. The search was locked and applied in the validation cohort with 95.8% sensitivity and 100% specificity, respectively.

Conclusion: The revised clinical note based electronic search algorithm was found to be highly sensitive and specific for DIC during the corresponding ICU duration.

Derivation and validation of a computable phenotype for acute decompensated heart failure in hospitalized patients

Background

With higher adoption of electronic health records at health-care centers, electronic search algorithms (computable phenotype) for identifying acute decompensated heart failure (ADHF) among hospitalized patients can be an invaluable tool to enhance data abstraction accuracy and efficacy in order to improve clinical research accrual and patient centered outcomes. We aimed to derive and validate a computable phenotype for ADHF in hospitalized patients.

Methods

We screened 256, 443 eligible (age > 18 years and with prior research authorization) individuals who were admitted to Mayo Clinic Hospital in Rochester, MN, from January 1, 2006, through December 31, 2014. Using a randomly selected derivation cohort of 938 patients, several iterations of a free-text electronic search were developed and refined. The computable phenotype was subsequently validated in an independent cohort 100 patients. The sensitivity and specificity of the computable phenotype were compared to the gold standard (expert review of charts) and International Classification of Diseases-9 (ICD-9) codes for Acute Heart Failure.

Results

In the derivation cohort, the computable phenotype achieved a sensitivity of 97.5%, and specificity of 100%, whereas ICD-9 codes for Acute Heart Failure achieved a sensitivity of 47.5% and specificity of 96.7%. When all Heart Failure codes (ICD-9) were used, sensitivity and specificity were 97.5 and 86.6%, respectively. In the validation cohort, the sensitivity and specificity of the computable phenotype were 100 and 98.5%. The sensitivity and specificity for the ICD-9 codes (Acute Heart Failure) were 42 and 98.5%. Upon use of all Heart Failure codes (ICD-9), sensitivity and specificity were 96.8 and 91.3%.

Conclusions

Our results suggest that using computable phenotype to ascertain ADHF from the clinical notes contained within the electronic medical record are feasible and reliable. Our computable phenotype outperformed ICD-9 codes for the detection of ADHF.

Association of septic shock definitions and standardized mortality ratio in a contemporary cohort of critically ill patients

PURPOSE

The newly proposed septic shock definition has provoked a substantial controversy in the emergency and critical care communities. We aim to compare new (SEPSIS-III) versus old (SEPSIS-II) definitions for septic shock in a contemporary cohort of critically ill patients.

MATERIAL AND METHODS

Retrospective cohort of consecutive patients, age ≥ 18 years admitted to intensive care units at the Mayo Clinic between January 2009 and October 2015. We compared patients who met old, new, both, or neither definition of sepsis shock. SMR were calculated using APACHE IV predicted mortality.

RESULTS

The initial cohort consisted of 16,720 patients who had suspicion of infection, 7463 required vasopressor support. The median (IQR) age was 65(54-75) years and 4167(55.8%) were male. Compared to patients with old definition, the patients with new definition had higher APACHE III score (median IQR); (73 (57-92) vs. 70 (56-89), p < .01); SOFA score; (6 (4-10) vs. 6 (4-9), p < .01), were older (70 (59-79) vs. 64 (54-74) years, p = .03). They also had higher hospital mortality, N (%) 71, (19.7%) vs. 40 (12.6%), p < .01) and a higher SMR (0.66 vs. 0.45, p < .01).

CONCLUSIONS

Compared to SEPSIS-II, SEPSIS-III definition of septic shock identifies patients further along disease trajectory with higher likelihood of poor outcome.

Incidence of and Risk Factors For Post-Intubation Hypotension in the Critically Ill

BACKGROUND

We aim to report the incidence of post-intubation hypotension in the critically ill, to report in-hospital mortality and length of stay in those who developed post-intubation hypotension, and to explore possible risk factors associated with post-intubation hypotension.

MATERIAL/METHODS

Adult (≥18 years) ICU patients who received emergent endotracheal intubation were included. We excluded patients if they were hemodynamically unstable 60 minutes pre-intubation. Post-intubation hypotension was defined as the administration of any vasopressor within 60 minutes following intubation.

RESULTS

Twenty-nine patients developed post-intubation hypotension (29/147, 20%). Post-intubation hypotension was associated with increased in-hospital mortality (11/29, 38% vs. 19/118, 16%) and length of stay (21 [10-37] vs. 12 [7-21] days) on multivariate analysis. Three risk factors for post-intubation hypotension were identified on multivariate analysis: 1) decreasing mean arterial pressure pre-intubation (per 5 mmHg decrease) (p-value=0.04; 95% CI 1.01-1.55); 2) administration of neuromuscular blockers (p-value=0.03; 95% CI 1.12-6.53); and 3) intubation complication (p-value=0.03; 95% CI 1.16-15.57).

CONCLUSIONS

Post-intubation hypotension was common in the ICU and was associated with increased in-hospital mortality and length of stay. These patients were more likely to have had lower mean arterial pressure prior to intubation, received neuromuscular blockers, or suffered a complication during intubation.

Vasopressor use as a surrogate for post-intubation hemodynamic instability is associated with in-hospital and 90-day mortality: a retrospective cohort study

BACKGROUND

Evidence is lacking for what defines post-intubation hypotension in the intensive care unit (ICU). If a valid definition could be used, the potential exists to evaluate possible risk factors and thereby improve post-intubation. Thus, our objectives were to arrive at the best surrogate for post-intubation hypotension that accurately predicts both in-hospital and 90-day mortality in a population of ICU patients and to report mortality rates between the exposed and unexposed cohorts.

METHODS

We conducted a retrospective cohort study of emergent endotracheal intubations in a medical-surgical ICU from January 1, 2010 to December 31, 2011 to evaluate surrogates for post-intubation hypotension that would predict in-hospital and 90-day mortality followed by an analysis of exposed versus unexposed using our best surrogate. Patients were ≥18 years of age, underwent emergent intubation during their first ICU admission, and did not meet any of the surrogates 60 min pre-intubation.

RESULTS

The six surrogates evaluated 60 min post-intubation were those with any systolic blood pressures ≤90 mmHg, any mean arterial pressures ≤65 mmHg, reduction in median systolic blood pressures of ≥20%, any vasopressor administration, any non-sinus rhythm and, fluid administration of ≥30 ml/kg. A total of 147 patients were included. Of the six surrogates, only the administration of any vasopressor 60 min post-intubation remained significant for mortality. Twenty-nine patients were then labeled as hemodynamically unstable and compared to the 118 patients labeled as hemodynamically stable. After adjusting for confounders, the hemodynamically unstable group had a significantly higher in-hospital and 90-day mortality [OR (95% CI); 3.84 (1.31-11.57) (p value = 0.01) and 2.37 (1.18-4.61) (p-value = 0.02)].

CONCLUSIONS

Emergently intubated patients manifesting hemodynamic instability after but not before intubation, as measured by vasoactive administration 60 min post-intubation, have a higher association with in-hospital and 90-day mortality.

Retrospective Derivation and Validation of an Automated Electronic Search Algorithm to Identify Post Operative Cardiovascular and Thromboembolic Complications

BACKGROUND

With increasing numbers of hospitals adopting electronic medical records, electronic search algorithms for identifying postoperative complications can be invaluable tools to expedite data abstraction and clinical research to improve patient outcomes.

OBJECTIVES

To derive and validate an electronic search algorithm to identify postoperative thromboembolic and cardiovascular complications such as deep venous thrombosis, pulmonary embolism, or myocardial infarction within 30 days of total hip or knee arthroplasty.

METHODS

A total of 34 517 patients undergoing total hip or knee arthroplasty between January 1, 1996 and December 31, 2013 were identified. Using a derivation cohort of 418 patients, several iterations of a free-text electronic search were developed and refined for each complication. Subsequently, the automated search algorithm was validated on an independent cohort of 2 857 patients, and the sensitivity and specificities were compared to the results of manual chart review.

RESULTS

In the final derivation subset, the automated search algorithm achieved a sensitivity of 91% and specificity of 85% for deep vein thrombosis, a sensitivity of 96% and specificity of 100% for pulmonary embolism, and a sensitivity of 100% and specificity of 95% for myocardial infarction. When applied to the validation cohort, the search algorithm achieved a sensitivity of 97% and specificity of 99% for deep vein thrombosis, a sensitivity of 97% and specificity of 100% for pulmonary embolism, and a sensitivity of 100% and specificity of 99% for myocardial infarction.

CONCLUSIONS

The derivation and validation of an electronic search strategy can accelerate the data abstraction process for research, quality improvement, and enhancement of patient care, while maintaining superb reliability compared to manual review.

Evaluation of preintubation shock index and modified shock index as predictors of postintubation hypotension and other short-term outcomes

PURPOSE

Preintubation shock index (SI) and modified shock index (MSI) have demonstrated predictive capability for postintubation hypotension in emergency department. The primary aim was to explore this relationship in the critical care environment. The secondary aims were to evaluate the relationship of shock indices with other short-term outcomes like mortality and length of stay in intensive care unit.

MATERIALS AND METHODS

This is a nonconcurrent cohort study, conducted in eligible 140 adult intensive care unit (ICU) patients of a tertiary care medical center. Eligibility criterion was emergent endotracheal intubation in apparently hemodynamically stable patients.

RESULTS

Preintubation SI ≥ 0.90 had a significant association with postintubation hypotension as defined by systolic blood pressure < 90 mm Hg in the univariate (P = .03; odds ratio [OR], 2.13; 95% confidence interval [CI], 1.07-4.35) and multivariate analyses (P = .01; OR, 3.17; 95% CI, 1.36-7.73) after adjusting for confounders. It was also associated with higher ICU mortality in both the univariate (P = .01; OR, 4.00; 95% CI, 1.26-12.67) and multivariate analyses (P = .01; OR, 5.75; 95% CI, 1.58-26.48). There was no association of preintubation MSI with postintubation hemodynamic instability and ICU mortality. No association was found between preintubation SI and MSI, with ICU length of stay and 30-day mortality.

CONCLUSIONS

Our findings indicate that preintubation SI greater than or equal to 0.90 is a predictor of postintubation hypotension (systolic blood pressure <90 mm Hg) and ICU mortality in emergently intubated adult patients in intensive care units.

From bench to bed: bridging from informatics theory to practice. An exploratory analysis

BACKGROUND

In 2009, the journal Applied Clinical Informatics (ACI) commenced publication. Focused on applications in clinical informatics, ACI was intended to be a companion journal to METHODS of Information in Medicine (MIM). Both journals are official journals of IMIA, the International Medical Informatics Association.

OBJECTIVES

To explore, after five years, which congruencies and interdependencies exist in publications of these journals and to determine if gaps exist. To achieve this goal, major topics discussed in ACI and in MIM had to be analysed. Finally, we wanted to explore, whether the intention of publishing these companion journals to provide an information bridge from informatics theory to informatics practice and from practice to theory could be supported by this model. In this manuscript we will report on congruencies and interdependencies from practise to theory and on major topis in ACI. Further results will be reported in a second paper.

METHODS

Retrospective, prolective observational study on recent publications of ACI and MIM. All publications of the years 2012 and 2013 from these journals were indexed and analysed.

RESULTS

Hundred and ninety-six publications have been analysed (87 ACI, 109 MIM). In ACI publications addressed care coordination, shared decision support, and provider communication in its importance for complex patient care and safety and quality. Other major themes included improving clinical documentation quality and efficiency, effectiveness of clinical decision support and alerts, implementation of health information technology systems including discussion of failures and succeses. An emerging topic in the years analyzed was a focus on health information technology to predict and prevent hospital admissions and managing population health including the application of mobile health technology. Congruencies between journals could be found in themes, but with different focus in its contents. Interdependencies from practise to theory found in these publications, were only limited.

CONCLUSIONS

Bridging from informatics theory to practise and vice versa remains a major component of successful research and practise as well as a major challenge.

Derivation and validation of a search algorithm to retrospectively identify mechanical ventilation initiation in the intensive care unit

BACKGROUND

The development and validation of automated electronic medical record (EMR) search strategies are important for establishing the timing of mechanical ventilation initiation in the intensive care unit (ICU).Thus, we sought to develop and validate an automated EMR search algorithm (strategy) for time zero, the moment of mechanical ventilation initiation in the critically ill patient.

METHODS

The EMR search algorithm was developed on the basis of several mechanical ventilation parameters, with the final parameter being positive end-expiratory pressure (PEEP), and was applied to a comprehensive institutional EMR database. The search algorithm was derived from a secondary retrospective analysis of a subset of 450 patients from a cohort of 2,684 patients admitted to a medical ICU and a surgical ICU from January 1, 2010, through December 31, 2011. It was then validated in an independent subset of 450 patients from the same period. The overall percent of agreement between our search algorithm and a comprehensive manual medical record review in the derivation and validation subsets, using peak inspiratory pressure (PIP) as the reference standard, was compared to assess timing of mechanical ventilation initiation.

RESULTS

In the derivation subset, the automated electronic search strategy achieved an 87% (κ = 0.87) perfect agreement, with 94% agreement to within one minute. In validating this search algorithm, perfect agreement was found in 92% (κ = 0.92) of patients, with 99% agreement occurring within one minute.

CONCLUSIONS

The use of an electronic search strategy resulted in highly accurate extraction of mechanical ventilation initiation in the ICU. The search algorithm of mechanical ventilation initiation is highly efficient and reliable and can facilitate both clinical research and patient care management in a timely manner.

Retrospective derivation and validation of a search algorithm to identify extubation failure in the intensive care unit

Background

Development and validation of automated electronic medical record (EMR) search strategies is important in identifying extubation failure in the intensive care unit (ICU). We developed and validated an automated search algorithm (strategy) for extubation failure in critically ill patients.

Methods

The EMR search algorithm was created through sequential steps with keywords applied to an institutional EMR database. The search strategy was derived retrospectively through secondary analysis of a 100-patient subset from the 978 patient cohort admitted to a neurological ICU from January 1, 2002, through December 31, 2011(derivation subset). It was, then, validated against an additional 100-patient subset (validation subset). Sensitivity, specificity, negative and positive predictive values of the automated search algorithm were compared with a manual medical record review (the reference standard) for data extraction of extubation failure.

Results

In the derivation subset of 100 random patients, the initial automated electronic search strategy achieved a sensitivity of 85% (95% CI, 56%-97%) and a specificity of 95% (95% CI, 87%-98%). With refinements in the search algorithm, the final sensitivity was 93% (95% CI, 64%-99%) and specificity increased to 100% (95% CI, 95%-100%) in this subset. In validation of the algorithm through a separate 100 random patient subset, the reported sensitivity and specificity were 94% (95% CI, 69%-99%) and 98% (95% CI, 92%-99%) respectively.

Conclusions

Use of electronic search algorithms allows for correct extraction of extubation failure in the ICU, with high degrees of sensitivity and specificity. Such search algorithms are a reliable alternative to manual chart review for identification of extubation failure.