A computable phenotype for asthma case identification in adult and pediatric patients: External validation in the Chicago Area Patient-Outcomes Research Network (CAPriCORN)

Development of a Computable Phenotype for Prehospital Pediatric Asthma Encounters

INTRODUCTION

Asthma exacerbations are a common cause of pediatric Emergency Medical Services (EMS) encounters. Accordingly, prehospital management of pediatric asthma exacerbations has been designated an EMS research priority. However, accurate identification of pediatric asthma exacerbations from the prehospital record is nuanced and difficult due to the heterogeneity of asthma symptoms, especially in children. Therefore, this study's objective was to develop a prehospital-specific pediatric asthma computable phenotype (CP) that could accurately identify prehospital encounters for pediatric asthma exacerbations.

METHODS

This is a retrospective observational study of patient encounters for ages 2-18 years from the ESO Data Collaborative between 2018 and 2021. We modified two existing rule-based pediatric asthma CPs and created three new CPs (one rule-based and two machine learning-based). Two pediatric emergency medicine physicians independently reviewed encounters to assign labels of asthma exacerbation or not. Taking that labeled encounter data, a 50/50 train/test split was used to create training and test sets from the labeled data. A 90/10 split was used to create a small validation set from the training set. We used specificity, sensitivity, positive predictive value (PPV), negative predictive value (NPV) and macro F1 to compare performance across all CP models.

RESULTS

After applying the inclusion and exclusion criteria, 24,283 patient encounters remained. The machine-learning models exhibited the best performance for the identification of pediatric asthma exacerbations. A multi-layer perceptron-based model had the best performance in all metrics, with an F1 score of 0.95, specificity of 1.00, sensitivity of 0.91, negative predictive value of 0.98, and positive predictive value of 1.00.

CONCLUSION

We modified existing and developed new pediatric asthma CPs to retrospectively identify prehospital pediatric asthma exacerbation encounters. We found that machine learning-based models greatly outperformed rule-based models. Given the high performance of the machine-learning models, the development and application of machine learning-based CPs for other conditions and diseases could help accelerate EMS research and ultimately enhance clinical care by accurately identifying patients with conditions of interest.

Using electronic health records to identify asthma-related acute care encounters

BACKGROUND

Leveraging "big data" to improve care requires that clinical concepts be operationalized using available data. Electronic health record (EHR) data can be used to evaluate asthma care, but relying solely on diagnosis codes may misclassify asthma-related encounters.

OBJECTIVE

We created streamlined, feasible and transparent prototype algorithms for EHR data to classify emergency department (ED) encounters and hospitalizations as "asthma-related."

METHODS

As part of an asthma program evaluation, expert clinicians conducted a multi-phase iterative chart review to evaluate 467 pediatric ED encounters and 136 hospitalizations from calendar years 2017 and 2019, rating the likelihood that each encounter was asthma-related. Using this as a reference standard, we developed rule-based algorithms for EHR data to classify visits. Accuracy was evaluated using sensitivity, specificity, and positive and negative predictive values (PPV, NPV).

RESULTS

Clinicians categorized 38% of ED encounters as "definitely" or "probably" asthma-related; 13% as "possibly" asthma-related; and 49% as "probably not" or "definitely not" related to asthma. Based on this reference standard, we created two rule-based algorithms to identify "definitely" or "probably" asthma-related encounters, one using text and non-text EHR fields and another using non-text fields only. Sensitivity, specificity, PPV, and NPV were >95% for the algorithm using text and non-text fields and >87% for the algorithm using only non-text fields compared to the reference standard. We created a two-rule algorithm to identify asthma-related hospitalizations using only non-text fields.

CONCLUSIONS

Diagnostic codes alone are insufficient to identify asthma-related visits, but EHR-based prototype algorithms that include additional methods of identification can predict clinician-identified visits with sufficient accuracy.

Scalable and interpretable alternative to chart review for phenotype evaluation using standardized structured data from electronic health records

OBJECTIVES

Chart review as the current gold standard for phenotype evaluation cannot support observational research on electronic health records and claims data sources at scale. We aimed to evaluate the ability of structured data to support efficient and interpretable phenotype evaluation as an alternative to chart review.

MATERIALS AND METHODS

We developed Knowledge-Enhanced Electronic Profile Review (KEEPER) as a phenotype evaluation tool that extracts patient's structured data elements relevant to a phenotype and presents them in a standardized fashion following clinical reasoning principles. We evaluated its performance (interrater agreement, intermethod agreement, accuracy, and review time) compared to manual chart review for 4 conditions using randomized 2-period, 2-sequence crossover design.

RESULTS

Case ascertainment with KEEPER was twice as fast compared to manual chart review. 88.1% of the patients were classified concordantly using charts and KEEPER, but agreement varied depending on the condition. Missing data and differences in interpretation accounted for most of the discrepancies. Pairs of clinicians agreed in case ascertainment in 91.2% of the cases when using KEEPER compared to 76.3% when using charts. Patient classification aligned with the gold standard in 88.1% and 86.9% of the cases respectively.

CONCLUSION

Structured data can be used for efficient and interpretable phenotype evaluation if they are limited to relevant subset and organized according to the clinical reasoning principles. A system that implements these principles can achieve noninferior performance compared to chart review at a fraction of time.

Risk of post-acute sequelae of SARS-CoV-2 infection associated with pre-coronavirus disease obstructive sleep apnea diagnoses: an electronic health record-based analysis from the RECOVER initiative

STUDY OBJECTIVES

Obstructive sleep apnea (OSA) has been associated with more severe acute coronavirus disease-2019 (COVID-19) outcomes. We assessed OSA as a potential risk factor for Post-Acute Sequelae of SARS-CoV-2 (PASC).

METHODS

We assessed the impact of preexisting OSA on the risk for probable PASC in adults and children using electronic health record data from multiple research networks. Three research networks within the REsearching COVID to Enhance Recovery initiative (PCORnet Adult, PCORnet Pediatric, and the National COVID Cohort Collaborative [N3C]) employed a harmonized analytic approach to examine the risk of probable PASC in COVID-19-positive patients with and without a diagnosis of OSA prior to pandemic onset. Unadjusted odds ratios (ORs) were calculated as well as ORs adjusted for age group, sex, race/ethnicity, hospitalization status, obesity, and preexisting comorbidities.

RESULTS

Across networks, the unadjusted OR for probable PASC associated with a preexisting OSA diagnosis in adults and children ranged from 1.41 to 3.93. Adjusted analyses found an attenuated association that remained significant among adults only. Multiple sensitivity analyses with expanded inclusion criteria and covariates yielded results consistent with the primary analysis.

CONCLUSIONS

Adults with preexisting OSA were found to have significantly elevated odds of probable PASC. This finding was consistent across data sources, approaches for identifying COVID-19-positive patients, and definitions of PASC. Patients with OSA may be at elevated risk for PASC after SARS-CoV-2 infection and should be monitored for post-acute sequelae.

A Phenotyping Algorithm to Identify People With HIV in Electronic Health Record Data (HIV-Phen): Development and Evaluation Study

Background

Identification of people with HIV from electronic health record (EHR) data is an essential first step in the study of important HIV outcomes, such as risk assessment. This task has been historically performed via manual chart review, but the increased availability of large clinical data sets has led to the emergence of phenotyping algorithms to automate this process. Existing algorithms for identifying people with HIV rely on a combination of International Classification of Disease codes and laboratory tests or closely mimic clinical testing guidelines for HIV diagnosis. However, we found that existing algorithms in the literature missed a significant proportion of people with HIV in our data.

Objective

The aim of this study is to develop and evaluate HIV-Phen, an updated criteria-based HIV phenotyping algorithm.

Methods

We developed an algorithm using HIV-specific laboratory tests and medications and compared it with previously published algorithms in national and local data sets to identify cohorts of people with HIV. Cohort demographics were compared with those reported in the national and local surveillance data. Chart reviews were performed on a subsample of patients from the local database to calculate the sensitivity, specificity, positive predictive value, negative predictive value, and accuracy of the algorithm.

Results

Our new algorithm identified substantially more people with HIV in both national (up to an 85.75% increase) and local (up to an 83.20% increase) EHR databases than the previously published algorithms. The demographic characteristics of people with HIV identified using our algorithm were similar to those reported in national and local HIV surveillance data. Our algorithm demonstrated improved sensitivity over existing algorithms (98% vs 56%-92%) while maintaining a similar overall accuracy (96% vs 80%-96%).

Conclusions

We developed and evaluated an updated criteria-based phenotyping algorithm for identifying people with HIV in EHR data that demonstrates improved sensitivity over existing algorithms.

Accuracy of Asthma Computable Phenotypes to Identify Pediatric Asthma at an Academic Institution

OBJECTIVES

Asthma is a heterogenous condition with significant diagnostic complexity, including variations in symptoms and temporal criteria. The disease can be difficult for clinicians to diagnose accurately. Properly identifying asthma patients from the electronic health record is consequently challenging as current algorithms (computable phenotypes) rely on diagnostic codes (e.g., International Classification of Disease, ICD) in addition to other criteria (e.g., inhaler medications)-but presume an accurate diagnosis. As such, there is no universally accepted or rigorously tested computable phenotype for asthma.

METHODS

We compared two established asthma computable phenotypes: the Chicago Area Patient-Outcomes Research Network (CAPriCORN) and Phenotype KnowledgeBase (PheKB). We established a large-scale, consensus gold standard (n = 1,365) from the University of California, Los Angeles Health System's clinical data warehouse for patients 5 to 17 years old. Results were manually reviewed and predictive performance (positive predictive value [PPV], sensitivity/specificity, F1-score) determined. We then examined the classification errors to gain insight for future algorithm optimizations.

RESULTS

As applied to our final cohort of 1,365 expert-defined gold standard patients, the CAPriCORN algorithms performed with a balanced PPV = 95.8% (95% CI: 94.4-97.2%), sensitivity = 85.7% (95% CI: 83.9-87.5%), and harmonized F1 = 90.4% (95% CI: 89.2-91.7%). The PheKB algorithm was performed with a balanced PPV = 83.1% (95% CI: 80.5-85.7%), sensitivity = 69.4% (95% CI: 66.3-72.5%), and F1 = 75.4% (95% CI: 73.1-77.8%). Four categories of errors were identified related to method limitations, disease definition, human error, and design implementation.

CONCLUSION

The performance of the CAPriCORN and PheKB algorithms was lower than previously reported as applied to pediatric data (PPV = 97.7 and 96%, respectively). There is room to improve the performance of current methods, including targeted use of natural language processing and clinical feature engineering.

Developing and evaluating a pediatric asthma severity computable phenotype derived from electronic health records

BACKGROUND

Extensive data available in electronic health records (EHRs) have the potential to improve asthma care and understanding of factors influencing asthma outcomes. However, this work can be accomplished only when the EHR data allow for accurate measures of severity, which at present are complex and inconsistent.

OBJECTIVE

Our aims were to create and evaluate a standardized pediatric asthma severity phenotype based in clinical asthma guidelines for use in EHR-based health initiatives and studies and also to examine the presence and absence of these data in relation to patient characteristics.

METHODS

We developed an asthma severity computable phenotype and compared the concordance of different severity components contributing to the phenotype to trends in the literature. We used multivariable logistic regression to assess the presence of EHR data relevant to asthma severity.

RESULTS

The asthma severity computable phenotype performs as expected in comparison with national statistics and the literature. Severity classification for a child is maximized when based on the long-term medication regimen component and minimized when based only on the symptom data component. Use of the severity phenotype results in better, clinically grounded classification. Children for whom severity could be ascertained from these EHR data were more likely to be seen for asthma in the outpatient setting and less likely to be older or Hispanic. Black children were less likely to have lung function testing data present.

CONCLUSION

We developed a pragmatic computable phenotype for pediatric asthma severity that is transportable to other EHRs.

Asthma Exacerbations in Patients with Type 2 Diabetes and Asthma on Glucagon-like Peptide-1 Receptor Agonists

Rationale: GLP-1R (glucagon-like peptide-1 receptor) agonists are approved to treat type 2 diabetes mellitus and obesity. GLP-1R agonists reduce airway inflammation and hyperresponsiveness in preclinical models.Objectives: To compare rates of asthma exacerbations and symptoms between adults with type 2 diabetes and asthma prescribed GLP-1R agonists and those prescribed SGLT-2 (sodium-glucose cotransporter-2) inhibitors, DPP-4 (dipeptidyl peptidase-4) inhibitors, sulfonylureas, or basal insulin for diabetes treatment intensification.Methods: This study was an electronic health records-based new-user, active-comparator, retrospective cohort study of patients with type 2 diabetes and asthma newly prescribed GLP-1R agonists or comparator drugs at an academic healthcare system from January 2000 to March 2018. The primary outcome was asthma exacerbations; the secondary outcome was encounters for asthma symptoms. Propensity scores were calculated for GLP-1R agonist and non-GLP-1R agonist use. Zero-inflated Poisson regression models included adjustment for multiple covariates.Measurements and Main Results: Patients initiating GLP-1R agonists (n = 448), SGLT-2 inhibitors (n = 112), DPP-4 inhibitors (n = 435), sulfonylureas (n = 2,253), or basal insulin (n = 2,692) were identified. At 6 months, asthma exacerbation counts were lower in persons initiating GLP-1R agonists (reference) compared with SGLT-2 inhibitors (incidence rate ratio [IRR], 2.98; 95% confidence interval [CI], 1.30-6.80), DPP-4 inhibitors (IRR, 2.45; 95% CI, 1.54-3.89), sulfonylureas (IRR, 1.83; 95% CI, 1.20-2.77), and basal insulin (IRR, 2.58; 95% CI, 1.72-3.88). Healthcare encounters for asthma symptoms were also lower among GLP-1R agonist users.Conclusions: Adult patients with asthma prescribed GLP-1R agonists for type 2 diabetes had lower counts of asthma exacerbations compared with other drugs initiated for treatment intensification. GLP-1R agonists may represent a novel treatment for asthma associated with metabolic dysfunction.

A case study evaluating the portability of an executable computable phenotype algorithm across multiple institutions and electronic health record environments

Electronic health record (EHR) algorithms for defining patient cohorts are commonly shared as free-text descriptions that require human intervention both to interpret and implement. We developed the Phenotype Execution and Modeling Architecture (PhEMA, http://projectphema.org) to author and execute standardized computable phenotype algorithms. With PhEMA, we converted an algorithm for benign prostatic hyperplasia, developed for the electronic Medical Records and Genomics network (eMERGE), into a standards-based computable format. Eight sites (7 within eMERGE) received the computable algorithm, and 6 successfully executed it against local data warehouses and/or i2b2 instances. Blinded random chart review of cases selected by the computable algorithm shows PPV ≥90%, and 3 out of 5 sites had >90% overlap of selected cases when comparing the computable algorithm to their original eMERGE implementation. This case study demonstrates potential use of PhEMA computable representations to automate phenotyping across different EHR systems, but also highlights some ongoing challenges.