A multicenter evaluation of computable phenotyping approaches for SARS-CoV-2 infection and COVID-19 hospitalizations

Computable phenotypes to identify respiratory viral infections in the All of Us research program

Electronic health records (EHRs) contain rich temporal data about respiratory viral infections, but methods to identify these infections from EHR data vary widely and lack robust validation. We developed computable phenotypes by integrating virus-specific International Classification of Diseases (ICD) billing codes, prescriptions, and laboratory results within 90-day episodes. Analysis of 265,222 participants with EHR data from the All of Us Research Program yielded national cohorts of varied size: large cohorts for SARS-CoV-2 (n = 28,729) and influenza (n = 19,784); medium cohorts for rhinovirus, human coronavirus, and respiratory syncytial virus (n = 1,161-1,620); and smaller cohorts for the other viruses (n = 238-486). Using laboratory results as a reference standard, phenotypes using virus-specific ICD codes and medications had variable sensitivity (8-67%) but high positive predictive value (PPV, 90-97%) for most viruses, while influenza virus and SARS-CoV-2 phenotypes had lower PPV (69-70%) that improved with the inclusion of additional ICD codes. Identified infections exhibited expected seasonal patterns matching CDC data. This integrated approach identified infections more effectively than individual components alone and demonstrated utility for severe infections in hospital settings. This method enables large-scale studies of host genetics, health disparities, and clinical outcomes across episodic diseases, with flexibility to optimize sensitivity or PPV depending on the specific research question.

Opportunities and Challenges in Using Electronic Health Record Systems to Study Postacute Sequelae of SARS-CoV-2 Infection: Insights From the NIH RECOVER Initiative

The benefits and challenges of electronic health records (EHRs) as data sources for clinical and epidemiologic research have been well described. However, several factors are important to consider when using EHR data to study novel, emerging, and multifaceted conditions such as postacute sequelae of SARS-CoV-2 infection or long COVID. In this article, we present opportunities and challenges of using EHR data to improve our understanding of long COVID, based on lessons learned from the National Institutes of Health (NIH)-funded RECOVER (REsearching COVID to Enhance Recovery) Initiative, and suggest steps to maximize the usefulness of EHR data when performing long COVID research.

Computable Phenotypes for Respiratory Viral Infections in the All of Us Research Program

Electronic health records (EHRs) contain rich temporal data about infectious diseases, but an optimal approach to identify infections remains undefined. Using the All of Us Research Program, we developed computable phenotypes for respiratory viruses by integrating billing codes, prescriptions, and laboratory results within 90-day episodes. Phenotypes computed from 265,222 participants yielded cohorts ranging from 238 (adenovirus) to 28,729 (SARS-CoV-2) cases. Virus-specific billing codes showed varied sensitivity (8-67%) and high positive predictive value (90-97%), except for influenza virus and SARS-CoV-2 where lower PPV (69-70%) improved with increasing billing codes. Identified infections exhibited expected seasonal patterns and virus proportions when compared with CDC data. This integrated approach identified episodic disease more effectively than individual components alone and demonstrated utility in identifying severe infections. The method enables large-scale studies of host genetics, health disparities, and clinical outcomes across episodic diseases.

A computable case definition for patients with SARS-CoV2 testing that occurred outside the hospital

Objective

To identify a cohort of COVID-19 cases, including when evidence of virus positivity was only mentioned in the clinical text, not in structured laboratory data in the electronic health record (EHR).

Materials and Methods

Statistical classifiers were trained on feature representations derived from unstructured text in patient EHRs. We used a proxy dataset of patients with COVID-19 polymerase chain reaction (PCR) tests for training. We selected a model based on performance on our proxy dataset and applied it to instances without COVID-19 PCR tests. A physician reviewed a sample of these instances to validate the classifier.

Results

On the test split of the proxy dataset, our best classifier obtained 0.56 F1, 0.6 precision, and 0.52 recall scores for SARS-CoV2 positive cases. In an expert validation, the classifier correctly identified 97.6% (81/84) as COVID-19 positive and 97.8% (91/93) as not SARS-CoV2 positive. The classifier labeled an additional 960 cases as not having SARS-CoV2 lab tests in hospital, and only 177 of those cases had the ICD-10 code for COVID-19.

Discussion

Proxy dataset performance may be worse because these instances sometimes include discussion of pending lab tests. The most predictive features are meaningful and interpretable. The type of external test that was performed is rarely mentioned.

Conclusion

COVID-19 cases that had testing done outside of the hospital can be reliably detected from the text in EHRs. Training on a proxy dataset was a suitable method for developing a highly performant classifier without labor-intensive labeling efforts.

Cardiovascular Care Innovation through Data-Driven Discoveries in the Electronic Health Record

The electronic health record (EHR) represents a rich source of patient information, increasingly being leveraged for cardiovascular research. Although its primary use remains the seamless delivery of health care, the various longitudinally aggregated structured and unstructured data elements for each patient within the EHR can define the computational phenotypes of disease and care signatures and their association with outcomes. Although structured data elements, such as demographic characteristics, laboratory measurements, problem lists, and medications, are easily extracted, unstructured data are underused. The latter include free text in clinical narratives, documentation of procedures, and reports of imaging and pathology. Rapid scaling up of data storage and rapid innovation in natural language processing and computer vision can power insights from unstructured data streams. However, despite an array of opportunities for research using the EHR, specific expertise is necessary to adequately address confidentiality, accuracy, completeness, and heterogeneity challenges in EHR-based research. These often require methodological innovation and best practices to design and conduct successful research studies. Our review discusses these challenges and their proposed solutions. In addition, we highlight the ongoing innovations in federated learning in the EHR through a greater focus on common data models and discuss ongoing work that defines such an approach to large-scale, multicenter, federated studies. Such parallel improvements in technology and research methods enable innovative care and optimization of patient outcomes.

Impact of changing case definitions for coronavirus disease 2019 (COVID-19) hospitalization on pandemic metrics

OBJECTIVE

To examine the impact of commonly used case definitions for coronavirus disease 2019 (COVID-19) hospitalizations on case counts and outcomes.

DESIGN, PATIENTS, AND SETTING

Retrospective analysis of all adults hospitalized between March 1, 2020, and March 1, 2022, at 5 Massachusetts acute-care hospitals.

INTERVENTIONS

We applied 6 commonly used definitions of COVID-19 hospitalization: positive severe acute respiratory coronavirus virus 2 (SARS-CoV-2) polymerase chain reaction (PCR) assay within 14 days of admission, PCR plus dexamethasone administration, PCR plus remdesivir, PCR plus hypoxemia, institutional COVID-19 flag, or COVID-19 International Classification of Disease, Tenth Revision (ICD-10) codes. Outcomes included case counts and in-hospital mortality. Overall, 100 PCR-positive cases were reviewed to determine each definition's accuracy for distinguishing primary or contributing versus incidental COVID-19 hospitalizations.

RESULTS

Of 306,387 hospital encounters, 15,436 (5.0%) met the PCR-based definition. COVID-19 hospitalization counts varied substantially between definitions: 4,628 (1.5% of all encounters) for PCR plus dexamethasone, 5,757 (1.9%) for PCR plus remdesivir, 11,801 (3.9%) for PCR plus hypoxemia, 15,673 (5.1%) for institutional flags, and 15,868 (5.2%) for ICD-10 codes. Definitions requiring dexamethasone, hypoxemia, or remdesivir selected sicker patients compared to PCR alone (mortality rates 12.2%, 10.7%, and 8.8% vs 8.3%, respectively). Definitions requiring PCR plus remdesivir or dexamethasone did not detect a reduction in in-hospital mortality associated with the SARS-CoV-2 Omicron variant. ICD-10 codes had the highest sensitivity (98.4%) but low specificity (39.5%) for distinguishing primary or contributing versus incidental COVID-19 hospitalizations. PCR plus dexamethasone had the highest specificity (92.1%) but low sensitivity (35.5%).

CONCLUSIONS

Commonly used definitions for COVID-19 hospitalizations generate variable case counts and outcomes and differentiate poorly between primary or contributing versus incidental COVID-19 hospitalizations. Surveillance definitions that better capture and delineate COVID-19-associated hospitalizations are needed.

Computable Clinical Phenotyping of Postacute Sequelae of COVID-19 in Pediatrics Using Real-World Data

Many questions remain unanswered concerning the long-term effects of COVID-19 on children. In this report, we describe a computable phenotyping algorithm for identifying children and adolescents with postacute sequelae of COVID-19 (PASC) and pilot this tool to characterize the clinical epidemiology of pediatric PASC in a large healthcare delivery network.

Accuracy of COVID-19-Like Illness Diagnoses in Electronic Health Record Data: Retrospective Cohort Study

BACKGROUND

Electronic health record (EHR) data provide a unique opportunity to study the epidemiology of COVID-19, clinical outcomes of the infection, comparative effectiveness of therapies, and vaccine effectiveness but require a well-defined computable phenotype of COVID-19-like illness (CLI).

OBJECTIVE

The objective of this study was to evaluate the performance of pathogen-specific and other acute respiratory illness (ARI) International Statistical Classification of Diseases-9 and -10 codes in identifying COVID-19 cases in emergency department (ED) or urgent care (UC) and inpatient settings.

METHODS

We conducted a retrospective observational cohort study using EHR, claims, and laboratory information system data of ED or UC and inpatient encounters from 4 health systems in the United States. Patients who were aged ≥18 years, had an ED or UC or inpatient encounter for an ARI, and underwent a SARS-CoV-2 polymerase chain reaction test between March 1, 2020, and March 31, 2021, were included. We evaluated various CLI definitions using combinations of International Statistical Classification of Diseases-10 codes as follows: COVID-19-specific codes; CLI definition used in VISION network studies; ARI signs, symptoms, and diagnosis codes only; signs and symptoms of ARI only; and random forest model definitions. We evaluated the sensitivity, specificity, positive predictive value, and negative predictive value of each CLI definition using a positive SARS-CoV-2 polymerase chain reaction test as the reference standard. We evaluated the performance of each CLI definition for distinct hospitalization and ED or UC cohorts.

RESULTS

Among 90,952 hospitalizations and 137,067 ED or UC visits, 5627 (6.19%) and 9866 (7.20%) were positive for SARS-CoV-2, respectively. COVID-19-specific codes had high sensitivity (91.6%) and specificity (99.6%) in identifying patients with SARS-CoV-2 positivity among hospitalized patients. The VISION CLI definition maintained high sensitivity (95.8%) but lowered specificity (45.5%). By contrast, signs and symptoms of ARI had low sensitivity and positive predictive value (28.9% and 11.8%, respectively) but higher specificity and negative predictive value (85.3% and 94.7%, respectively). ARI diagnoses, signs, and symptoms alone had low predictive performance. All CLI definitions had lower sensitivity for ED or UC encounters. Random forest approaches identified distinct CLI definitions with high performance for hospital encounters and moderate performance for ED or UC encounters.

CONCLUSIONS

COVID-19-specific codes have high sensitivity and specificity in identifying adults with positive SARS-CoV-2 test results. Separate combinations of COVID-19-specific codes and ARI codes enhance the utility of CLI definitions in studies using EHR data in hospital and ED or UC settings.