Real world performance of the 21st Century Cures Act population-level application programming interface

OBJECTIVE

To evaluate the real-world performance of the SMART/HL7 Bulk Fast Health Interoperability Resources (FHIR) Access Application Programming Interface (API), developed to enable push button access to electronic health record data on large populations, and required under the 21st Century Cures Act Rule.

MATERIALS AND METHODS

We used an open-source Bulk FHIR Testing Suite at 5 healthcare sites from April to September 2023, including 4 hospitals using electronic health records (EHRs) certified for interoperability, and 1 Health Information Exchange (HIE) using a custom, standards-compliant API build. We measured export speeds, data sizes, and completeness across 6 types of FHIR.

RESULTS

Among the certified platforms, Oracle Cerner led in speed, managing 5-16 million resources at over 8000 resources/min. Three Epic sites exported a FHIR data subset, achieving 1-12 million resources at 1555-2500 resources/min. Notably, the HIE's custom API outperformed, generating over 141 million resources at 12 000 resources/min.

DISCUSSION

The HIE's custom API showcased superior performance, endorsing the effectiveness of SMART/HL7 Bulk FHIR in enabling large-scale data exchange while underlining the need for optimization in existing EHR platforms. Agility and scalability are essential for diverse health, research, and public health use cases.

CONCLUSION

To fully realize the interoperability goals of the 21st Century Cures Act, addressing the performance limitations of Bulk FHIR API is critical. It would be beneficial to include performance metrics in both certification and reporting processes.

Moving Biosurveillance Beyond Coded Data Using AI for Symptom Detection From Physician Notes: Retrospective Cohort Study

BACKGROUND

Real-time surveillance of emerging infectious diseases necessitates a dynamically evolving, computable case definition, which frequently incorporates symptom-related criteria. For symptom detection, both population health monitoring platforms and research initiatives primarily depend on structured data extracted from electronic health records.

OBJECTIVE

This study sought to validate and test an artificial intelligence (AI)-based natural language processing (NLP) pipeline for detecting COVID-19 symptoms from physician notes in pediatric patients. We specifically study patients presenting to the emergency department (ED) who can be sentinel cases in an outbreak.

METHODS

Subjects in this retrospective cohort study are patients who are 21 years of age and younger, who presented to a pediatric ED at a large academic children's hospital between March 1, 2020, and May 31, 2022. The ED notes for all patients were processed with an NLP pipeline tuned to detect the mention of 11 COVID-19 symptoms based on Centers for Disease Control and Prevention (CDC) criteria. For a gold standard, 3 subject matter experts labeled 226 ED notes and had strong agreement (F1-score=0.986; positive predictive value [PPV]=0.972; and sensitivity=1.0). F1-score, PPV, and sensitivity were used to compare the performance of both NLP and the International Classification of Diseases, 10th Revision (ICD-10) coding to the gold standard chart review. As a formative use case, variations in symptom patterns were measured across SARS-CoV-2 variant eras.

RESULTS

There were 85,678 ED encounters during the study period, including 4% (n=3420) with patients with COVID-19. NLP was more accurate at identifying encounters with patients that had any of the COVID-19 symptoms (F1-score=0.796) than ICD-10 codes (F1-score =0.451). NLP accuracy was higher for positive symptoms (sensitivity=0.930) than ICD-10 (sensitivity=0.300). However, ICD-10 accuracy was higher for negative symptoms (specificity=0.994) than NLP (specificity=0.917). Congestion or runny nose showed the highest accuracy difference (NLP: F1-score=0.828 and ICD-10: F1-score=0.042). For encounters with patients with COVID-19, prevalence estimates of each NLP symptom differed across variant eras. Patients with COVID-19 were more likely to have each NLP symptom detected than patients without this disease. Effect sizes (odds ratios) varied across pandemic eras.

CONCLUSIONS

This study establishes the value of AI-based NLP as a highly effective tool for real-time COVID-19 symptom detection in pediatric patients, outperforming traditional ICD-10 methods. It also reveals the evolving nature of symptom prevalence across different virus variants, underscoring the need for dynamic, technology-driven approaches in infectious disease surveillance.

Beyond compliance with the 21st Century Cures Act Rule: a patient controlled electronic health information export application programming interface

OBJECTIVE

The 21st Century Cures Act Final Rule requires that certified electronic health records (EHRs) be able to export a patient's full set of electronic health information (EHI). This requirement becomes more powerful if EHI exports use interoperable application programming interfaces (APIs). We sought to advance the ecosystem, instantiating policy desiderata in a working reference implementation based on a consensus design.

MATERIALS AND METHODS

We formulate a model for interoperable, patient-controlled, app-driven access to EHI exports in an open source reference implementation following the Argonaut FHIR Accelerator consensus implementation guide for EHI Export.

RESULTS

The reference implementation, which asynchronously provides EHI across an API, has three central components: a web application for patients to request EHI exports, an EHI server to respond to requests, and an administrative export management web application to manage requests. It leverages mandated SMART on FHIR/Bulk FHIR APIs.

DISCUSSION

A patient-controlled app enabling full EHI export from any EHR across an API could facilitate national-scale patient-directed information exchange. We hope releasing these tools sparks engagement from the health IT community to evolve the design, implement and test in real-world settings, and develop patient-facing apps.

CONCLUSION

To advance regulatory innovation, we formulate a model that builds on existing requirements under the Cures Act Rule and takes a step toward an interoperable, scalable approach, simplifying patient access to their own health data; supporting the sharing of clinical data for both improved patient care and medical research; and encouraging the growth of an ecosystem of third-party applications.

GSDMB/ORMDL3 Rare/Common Variants Are Associated with Inhaled Corticosteroid Response among Children with Asthma

Inhaled corticosteroids (ICS) are efficacious in the treatment of asthma, which affects more than 300 million people in the world. While genome-wide association studies have identified genes involved in differential treatment responses to ICS in asthma, few studies have evaluated the effects of combined rare and common variants on ICS response among children with asthma. Among children with asthma treated with ICS with whole exome sequencing (WES) data in the PrecisionLink Biobank (91 White and 20 Black children), we examined the effect and contribution of rare and common variants with hospitalizations or emergency department visits. For 12 regions previously associated with asthma and ICS response (DPP10, FBXL7, NDFIP1, TBXT, GLCCI1, HDAC9, TBXAS1, STAT6, GSDMB/ORMDL3, CRHR1, GNGT2, FCER2), we used the combined sum test for the sequence kernel association test (SKAT) adjusting for age, sex, and BMI and stratified by race. Validation was conducted in the Biorepository and Integrative Genomics (BIG) Initiative (83 White and 134 Black children). Using a Bonferroni threshold for the 12 regions tested (i.e., 0.05/12 = 0.004), GSDMB/ORMDL3 was significantly associated with ICS response for the combined effect of rare and common variants (p-value = 0.003) among White children in the PrecisionLink Biobank and replicated in the BIG Initiative (p-value = 0.02). Using WES data, the combined effect of rare and common variants for GSDMB/ORMDL3 was associated with ICS response among asthmatic children in the PrecisionLink Biobank and replicated in the BIG Initiative. This proof-of-concept study demonstrates the power of biobanks of pediatric real-life populations in asthma genomic investigations.

Cumulus: A federated EHR-based learning system powered by FHIR and AI

Objective

To address challenges in large-scale electronic health record (EHR) data exchange, we sought to develop, deploy, and test an open source, cloud-hosted app 'listener' that accesses standardized data across the SMART/HL7 Bulk FHIR Access application programming interface (API).

Methods

We advance a model for scalable, federated, data sharing and learning. Cumulus software is designed to address key technology and policy desiderata including local utility, control, and administrative simplicity as well as privacy preservation during robust data sharing, and AI for processing unstructured text.

Results

Cumulus relies on containerized, cloud-hosted software, installed within a healthcare organization's security envelope. Cumulus accesses EHR data via the Bulk FHIR interface and streamlines automated processing and sharing. The modular design enables use of the latest AI and natural language processing tools and supports provider autonomy and administrative simplicity. In an initial test, Cumulus was deployed across five healthcare systems each partnered with public health. Cumulus output is patient counts which were aggregated into a table stratifying variables of interest to enable population health studies. All code is available open source. A policy stipulating that only aggregate data leave the institution greatly facilitated data sharing agreements.

Discussion and Conclusion

Cumulus addresses barriers to data sharing based on (1) federally required support for standard APIs (2), increasing use of cloud computing, and (3) advances in AI. There is potential for scalability to support learning across myriad network configurations and use cases.

The SMART Text2FHIR Pipeline

Objective: To implement an open source, free, and easily deployable high throughput natural language processing module to extract concepts from clinician notes and map them to Fast Healthcare Interoperability Resources (FHIR). Materials and Methods: Using a popular open-source NLP tool (Apache cTAKES), we create FHIR resources that use modifier extensions to represent negation and NLP sourcing, and another extension to represent provenance of extracted concepts. Results: The SMART Text2FHIR Pipeline is an open-source tool, released through standard package managers, and publicly available container images that implement the mappings, enabling ready conversion of clinical text to FHIR. Discussion: With the increased data liquidity because of new interoperability regulations, NLP processes that can output FHIR can enable a common language for transporting structured and unstructured data. This framework can be valuable for critical public health or clinical research use cases. Conclusion: Future work should include mapping more categories of NLP-extracted information into FHIR resources and mappings from additional open-source NLP tools.

Generative Language Models and Open Notes: Exploring the Promise and Limitations

Patients' online record access (ORA) is growing worldwide. In some countries, including the United States and Sweden, access is advanced with patients obtaining rapid access to their full records on the web including laboratory and test results, lists of prescribed medications, vaccinations, and even the very narrative reports written by clinicians (the latter, commonly referred to as "open notes"). In the United States, patient's ORA is also available in a downloadable form for use with other apps. While survey studies have shown that some patients report many benefits from ORA, there remain challenges with implementation around writing clinical documentation that patients may now read. With ORA, the functionality of the record is evolving; it is no longer only an aide memoire for doctors but also a communication tool for patients. Studies suggest that clinicians are changing how they write documentation, inviting worries about accuracy and completeness. Other concerns include work burdens; while few objective studies have examined the impact of ORA on workload, some research suggests that clinicians are spending more time writing notes and answering queries related to patients' records. Aimed at addressing some of these concerns, clinician and patient education strategies have been proposed. In this viewpoint paper, we explore these approaches and suggest another longer-term strategy: the use of generative artificial intelligence (AI) to support clinicians in documenting narrative summaries that patients will find easier to understand. Applied to narrative clinical documentation, we suggest that such approaches may significantly help preserve the accuracy of notes, strengthen writing clarity and signals of empathy and patient-centered care, and serve as a buffer against documentation work burdens. However, we also consider the current risks associated with existing generative AI. We emphasize that for this innovation to play a key role in ORA, the cocreation of clinical notes will be imperative. We also caution that clinicians will need to be supported in how to work alongside generative AI to optimize its considerable potential.

Measuring Real-Time Medication Effects From Electroencephalography

PURPOSE

Evaluating the effects of antiseizure medication (ASM) on patients with epilepsy remains a slow and challenging process. Quantifiable noninvasive markers that are measurable in real-time and provide objective and useful information could guide clinical decision-making. We examined whether the effect of ASM on patients with epilepsy can be quantitatively measured in real-time from EEGs.

METHODS

This retrospective analysis was conducted on 67 patients in the long-term monitoring unit at Boston Children's Hospital. Two 30-second EEG segments were selected from each patient premedication and postmedication weaning for analysis. Nonlinear measures including entropy and recurrence quantitative analysis values were computed for each segment and compared before and after medication weaning.

RESULTS

Our study found that ASM effects on the brain were measurable by nonlinear recurrence quantitative analysis on EEGs. Highly significant differences ( P < 1e-11) were found in several nonlinear measures within the seizure zone in response to antiseizure medication. Moreover, the size of the medication effect correlated with a patient's seizure frequency, seizure localization, number of medications, and reported seizure frequency reduction on medication.

CONCLUSIONS

Our findings show the promise of digital biomarkers to measure medication effects and epileptogenicity.

Emergency department visits and boarding for pediatric patients with suicidality before and during the COVID-19 pandemic

OBJECTIVE

To quantify the increase in pediatric patients presenting to the emergency department with suicidality before and during the COVID-19 pandemic, and the subsequent impact on emergency department length of stay and boarding.

METHODS

This retrospective cohort study from June 1, 2016, to October 31, 2022, identified patients ages 6 to 21 presenting to the emergency department at a pediatric academic medical center with suicidality using ICD-10 codes. Number of emergency department encounters for suicidality, demographic characteristics of patients with suicidality, and emergency department length of stay were compared before and during the COVID-19 pandemic. Unobserved components models were used to describe monthly counts of emergency department encounters for suicidality.

RESULTS

There were 179,736 patient encounters to the emergency department during the study period, 6,215 (3.5%) for suicidality. There were, on average, more encounters for suicidality each month during the COVID-19 pandemic than before the COVID-19 pandemic. A time series unobserved components model demonstrated a temporary drop of 32.7 encounters for suicidality in April and May of 2020 (p<0.001), followed by a sustained increase of 31.2 encounters starting in July 2020 (p = 0.003). The average length of stay for patients that boarded in the emergency department with a diagnosis of suicidality was 37.4 hours longer during the COVID-19 pandemic compared to before the COVID-19 pandemic (p<0.001).

CONCLUSIONS

The number of encounters for suicidality among pediatric patients and the emergency department length of stay for psychiatry boarders has increased during the COVID-19 pandemic. There is a need for acute care mental health services and solutions to emergency department capacity issues.

Real World Performance of the 21st Century Cures Act Population Level Application Programming Interface

Objective

To evaluate the real-world performance in delivering patient data on populations, of the SMART/HL7 Bulk FHIR Access API, required in Electronic Health Records (EHRs) under the 21st Century Cures Act Rule.

Materials and Methods

We used an open-source Bulk FHIR Testing Suite at five healthcare sites from April to September 2023, including four hospitals using EHRs certified for interoperability, and one Health Information Exchange (HIE) using a custom, standards-compliant API build. We measured export speeds, data sizes, and completeness across six types of FHIR resources.

Results

Among the certified platforms, Oracle Cerner led in speed, managing 5-16 million resources at over 8,000 resources/min. Three Epic sites exported a FHIR data subset, achieving 1-12 million resources at 1,555-2,500 resources/min. Notably, the HIE's custom API outperformed, generating over 141 million resources at 12,000 resources/min.

Discussion

The HIE's custom API showcased superior performance, endorsing the effectiveness of SMART/HL7 Bulk FHIR in enabling large-scale data exchange while underlining the need for optimization in existing EHR platforms. Agility and scalability are essential for diverse health, research, and public health use cases.

Conclusion

To fully realize the interoperability goals of the 21st Century Cures Act, addressing the performance limitations of Bulk FHIR API is critical. It would be beneficial to include performance metrics in both certification and reporting processes.

Knowledge-Driven Online Multimodal Automated Phenotyping System

Though electronic health record (EHR) systems are a rich repository of clinical information with large potential, the use of EHR-based phenotyping algorithms is often hindered by inaccurate diagnostic records, the presence of many irrelevant features, and the requirement for a human-labeled training set. In this paper, we describe a knowledge-driven online multimodal automated phenotyping (KOMAP) system that i) generates a list of informative features by an online narrative and codified feature search engine (ONCE) and ii) enables the training of a multimodal phenotyping algorithm based on summary data. Powered by composite knowledge from multiple EHR sources, online article corpora, and a large language model, features selected by ONCE show high concordance with the state-of-the-art AI models (GPT4 and ChatGPT) and encourage large-scale phenotyping by providing a smaller but highly relevant feature set. Validation of the KOMAP system across four healthcare centers suggests that it can generate efficient phenotyping algorithms with robust performance. Compared to other methods requiring patient-level inputs and gold-standard labels, the fully online KOMAP provides a significant opportunity to enable multi-center collaboration.

A computable case definition for patients with SARS-CoV2 testing that occurred outside the hospital

Objective

To identify a cohort of COVID-19 cases, including when evidence of virus positivity was only mentioned in the clinical text, not in structured laboratory data in the electronic health record (EHR).

Materials and Methods

Statistical classifiers were trained on feature representations derived from unstructured text in patient EHRs. We used a proxy dataset of patients with COVID-19 polymerase chain reaction (PCR) tests for training. We selected a model based on performance on our proxy dataset and applied it to instances without COVID-19 PCR tests. A physician reviewed a sample of these instances to validate the classifier.

Results

On the test split of the proxy dataset, our best classifier obtained 0.56 F1, 0.6 precision, and 0.52 recall scores for SARS-CoV2 positive cases. In an expert validation, the classifier correctly identified 97.6% (81/84) as COVID-19 positive and 97.8% (91/93) as not SARS-CoV2 positive. The classifier labeled an additional 960 cases as not having SARS-CoV2 lab tests in hospital, and only 177 of those cases had the ICD-10 code for COVID-19.

Discussion

Proxy dataset performance may be worse because these instances sometimes include discussion of pending lab tests. The most predictive features are meaningful and interpretable. The type of external test that was performed is rarely mentioned.

Conclusion

COVID-19 cases that had testing done outside of the hospital can be reliably detected from the text in EHRs. Training on a proxy dataset was a suitable method for developing a highly performant classifier without labor-intensive labeling efforts.

Clinical phenotypes and outcomes in children with multisystem inflammatory syndrome across SARS-CoV-2 variant eras: a multinational study from the 4CE consortium

Background

Multisystem inflammatory syndrome in children (MIS-C) is a severe complication of SARS-CoV-2 infection. It remains unclear how MIS-C phenotypes vary across SARS-CoV-2 variants. We aimed to investigate clinical characteristics and outcomes of MIS-C across SARS-CoV-2 eras.

Methods

We performed a multicentre observational retrospective study including seven paediatric hospitals in four countries (France, Spain, U.K., and U.S.). All consecutive confirmed patients with MIS-C hospitalised between February 1st, 2020, and May 31st, 2022, were included. Electronic Health Records (EHR) data were used to calculate pooled risk differences (RD) and effect sizes (ES) at site level, using Alpha as reference. Meta-analysis was used to pool data across sites.

Findings

Of 598 patients with MIS-C (61% male, 39% female; mean age 9.7 years [SD 4.5]), 383 (64%) were admitted in the Alpha era, 111 (19%) in the Delta era, and 104 (17%) in the Omicron era. Compared with patients admitted in the Alpha era, those admitted in the Delta era were younger (ES -1.18 years [95% CI -2.05, -0.32]), had fewer respiratory symptoms (RD -0.15 [95% CI -0.33, -0.04]), less frequent non-cardiogenic shock or systemic inflammatory response syndrome (SIRS) (RD -0.35 [95% CI -0.64, -0.07]), lower lymphocyte count (ES -0.16 × 109/uL [95% CI -0.30, -0.01]), lower C-reactive protein (ES -28.5 mg/L [95% CI -46.3, -10.7]), and lower troponin (ES -0.14 ng/mL [95% CI -0.26, -0.03]). Patients admitted in the Omicron versus Alpha eras were younger (ES -1.6 years [95% CI -2.5, -0.8]), had less frequent SIRS (RD -0.18 [95% CI -0.30, -0.05]), lower lymphocyte count (ES -0.39 × 109/uL [95% CI -0.52, -0.25]), lower troponin (ES -0.16 ng/mL [95% CI -0.30, -0.01]) and less frequently received anticoagulation therapy (RD -0.19 [95% CI -0.37, -0.04]). Length of hospitalization was shorter in the Delta versus Alpha eras (-1.3 days [95% CI -2.3, -0.4]).

Interpretation

Our study suggested that MIS-C clinical phenotypes varied across SARS-CoV-2 eras, with patients in Delta and Omicron eras being younger and less sick. EHR data can be effectively leveraged to identify rare complications of pandemic diseases and their variation over time.

Funding

None.

Engaging a national-scale cohort of smart thermometer users in participatory surveillance

Participatory surveillance systems crowdsource individual reports to rapidly assess population health phenomena. The value of these systems increases when more people join and persistently contribute. We examine the level of and factors associated with engagement in participatory surveillance among a retrospective, national-scale cohort of individuals using smartphone-connected thermometers with a companion app that allows them to report demographic and symptom information. Between January 1, 2020 and October 29, 2022, 1,325,845 participants took 20,617,435 temperature readings, yielding 3,529,377 episodes of consecutive readings. There were 1,735,805 (49.2%) episodes with self-reported symptoms (including reports of no symptoms). Compared to before the pandemic, participants were more likely to report their symptoms during pandemic waves, especially after the winter wave began (September 13, 2020) (OR across pandemic periods range from 3.0 to 4.0). Further, symptoms were more likely to be reported during febrile episodes (OR = 2.6, 95% CI = 2.6-2.6), and for new participants, during their first episode (OR = 2.4, 95% CI = 2.4-2.5). Compared with participants aged 50-65 years old, participants over 65 years were less likely to report their symptoms (OR = 0.3, 95% CI = 0.3-0.3). Participants in a household with both adults and children (OR = 1.6 [1.6-1.7]) were more likely to report symptoms. We find that the use of smart thermometers with companion apps facilitates the collection of data on a large, national scale, and provides real time insight into transmissible disease phenomena. Nearly half of individuals using these devices are willing to report their symptoms after taking their temperature, although participation varies among individuals and over pandemic stages.

Association between Neuroligin-1 polymorphism and plasma glutamine levels in individuals with autism spectrum disorder

BACKGROUND

Unravelling the relationships between candidate genes and autism spectrum disorder (ASD) phenotypes remains an outstanding challenge. Endophenotypes, defined as inheritable, measurable quantitative traits, might provide intermediary links between genetic risk factors and multifaceted ASD phenotypes. In this study, we sought to determine whether plasma metabolite levels could serve as endophenotypes in individuals with ASD and their family members.

METHODS

We employed an untargeted, high-resolution metabolomics platform to analyse 14,342 features across 1099 plasma samples. These samples were collected from probands and their family members participating in the Autism Genetic Resource Exchange (AGRE) (N = 658), compared with neurotypical individuals enrolled in the PrecisionLink Health Discovery (PLHD) program at Boston Children's Hospital (N = 441). We conducted a metabolite quantitative trait loci (mQTL) analysis using whole-genome genotyping data from each cohort in AGRE and PLHD, aiming to prioritize significant mQTL and metabolite pairs that were exclusively observed in AGRE.

FINDINGS

Within the AGRE group, we identified 54 significant associations between genotypes and metabolite levels (P < 5.27 × 10-11), 44 of which were not observed in the PLHD group. Plasma glutamine levels were found to be associated with variants in the NLGN1 gene, a gene that encodes post-synaptic cell-adhesion molecules in excitatory neurons. This association was not detected in the PLHD group. Notably, a significant negative correlation between plasma glutamine and glutamate levels was observed in the AGRE group, but not in the PLHD group. Furthermore, plasma glutamine levels showed a negative correlation with the severity of restrictive and repetitive behaviours (RRB) in ASD, although no direct association was observed between RRB severity and the NLGN1 genotype.

INTERPRETATION

Our findings suggest that plasma glutamine levels could potentially serve as an endophenotype, thus establishing a link between the genetic risk associated with NLGN1 and the severity of RRB in ASD. This identified association could facilitate the development of novel therapeutic targets, assist in selecting specific cohorts for clinical trials, and provide insights into target symptoms for future ASD treatment strategies.

FUNDING

This work was supported by the National Institute of Health (grant numbers: R01MH107205, U01TR002623, R24OD024622, OT2OD032720, and R01NS129188) and the PrecisionLink Biobank for Health Discovery at Boston Children's Hospital.

Smart Thermometer-Based Participatory Surveillance to Discern the Role of Children in Household Viral Transmission During the COVID-19 Pandemic

Importance

Children's role in spreading virus during the COVID-19 pandemic is yet to be elucidated, and measuring household transmission traditionally requires contact tracing.

Objective

To discern children's role in household viral transmission during the pandemic when enveloped viruses were at historic lows and the predominance of viral illnesses were attributed to COVID-19.

Design, Setting, and Participants

This cohort study of a voluntary US cohort tracked data from participatory surveillance using commercially available thermometers with a companion smartphone app from October 2019 to October 2022. Eligible participants were individuals with temperature measurements in households with multiple members between October 2019 and October 2022 who opted into data sharing.

Main Outcomes and Measures

Proportion of household transmissions with a pediatric index case and changes in transmissions during school breaks were assessed using app and thermometer data.

Results

A total of 862 577 individuals from 320 073 households with multiple participants (462 000 female [53.6%] and 463 368 adults [53.7%]) were included. The number of febrile episodes forecast new COVID-19 cases. Within-household transmission was inferred in 54 506 (15.4%) febrile episodes and increased from the fourth pandemic period, March to July 2021 (3263 of 32 294 [10.1%]) to the Omicron BA.1/BA.2 wave (16 516 of 94 316 [17.5%]; P < .001). Among 38 787 transmissions in 166 170 households with adults and children, a median (IQR) 70.4% (61.4%-77.6%) had a pediatric index case; proportions fluctuated weekly from 36.9% to 84.6%. A pediatric index case was 0.6 to 0.8 times less frequent during typical school breaks. The winter break decrease was from 68.4% (95% CI, 57.1%-77.8%) to 41.7% (95% CI, 34.3%-49.5%) at the end of 2020 (P < .001). At the beginning of 2022, it dropped from 80.3% (95% CI, 75.1%-84.6%) to 54.5% (95% CI, 51.3%-57.7%) (P < .001). During summer breaks, rates dropped from 81.4% (95% CI, 74.0%-87.1%) to 62.5% (95% CI, 56.3%-68.3%) by August 2021 (P = .02) and from 83.8% (95% CI, 79.2%-87.5) to 62.8% (95% CI, 57.1%-68.1%) by July 2022 (P < .001). These patterns persisted over 2 school years.

Conclusions and Relevance

In this cohort study using participatory surveillance to measure within-household transmission at a national scale, we discerned an important role for children in the spread of viral infection within households during the COVID-19 pandemic, heightened when schools were in session, supporting a role for school attendance in COVID-19 spread.

The SMART Text2FHIR Pipeline

Objective

To implement an open source, free, and easily deployable high throughput natural language processing module to extract concepts from clinician notes and map them to Fast Healthcare Interoperability Resources (FHIR).

Materials and Methods

Using a popular open-source NLP tool (Apache cTAKES), we create FHIR resources that use modifier extensions to represent negation and NLP sourcing, and another extension to represent provenance of extracted concepts.

Results

The SMART Text2FHIR Pipeline is an open-source tool, released through standard package managers, and publicly available container images that implement the mappings, enabling ready conversion of clinical text to FHIR.

Discussion

With the increased data liquidity because of new interoperability regulations, NLP processes that can output FHIR can enable a common language for transporting structured and unstructured data. This framework can be valuable for critical public health or clinical research use cases.

Conclusion

Future work should include mapping more categories of NLP-extracted information into FHIR resources and mappings from additional open-source NLP tools.

Computerization of the Work of General Practitioners: Mixed Methods Survey of Final-Year Medical Students in Ireland

BACKGROUND

The potential for digital health technologies, including machine learning (ML)-enabled tools, to disrupt the medical profession is the subject of ongoing debate within biomedical informatics.

OBJECTIVE

We aimed to describe the opinions of final-year medical students in Ireland regarding the potential of future technology to replace or work alongside general practitioners (GPs) in performing key tasks.

METHODS

Between March 2019 and April 2020, using a convenience sample, we conducted a mixed methods paper-based survey of final-year medical students. The survey was administered at 4 out of 7 medical schools in Ireland across each of the 4 provinces in the country. Quantitative data were analyzed using descriptive statistics and nonparametric tests. We used thematic content analysis to investigate free-text responses.

RESULTS

In total, 43.1% (252/585) of the final-year students at 3 medical schools responded, and data collection at 1 medical school was terminated due to disruptions associated with the COVID-19 pandemic. With regard to forecasting the potential impact of artificial intelligence (AI)/ML on primary care 25 years from now, around half (127/246, 51.6%) of all surveyed students believed the work of GPs will change minimally or not at all. Notably, students who did not intend to enter primary care predicted that AI/ML will have a great impact on the work of GPs.

CONCLUSIONS

We caution that without a firm curricular foundation on advances in AI/ML, students may rely on extreme perspectives involving self-preserving optimism biases that demote the impact of advances in technology on primary care on the one hand and technohype on the other. Ultimately, these biases may lead to negative consequences in health care. Improvements in medical education could help prepare tomorrow's doctors to optimize and lead the ethical and evidence-based implementation of AI/ML-enabled tools in medicine for enhancing the care of tomorrow's patients.

A computable phenotype for patients with SARS-CoV2 testing that occurred outside the hospital

Objective

To identify a cohort of COVID-19 cases, including when evidence of virus positivity was only mentioned in the clinical text, not in structured laboratory data in the electronic health record (EHR).

Materials and Methods

Statistical classifiers were trained on feature representations derived from unstructured text in patient electronic health records (EHRs). We used a proxy dataset of patients with COVID-19 polymerase chain reaction (PCR) tests for training. We selected a model based on performance on our proxy dataset and applied it to instances without COVID-19 PCR tests. A physician reviewed a sample of these instances to validate the classifier.

Results

On the test split of the proxy dataset, our best classifier obtained 0.56 F1, 0.6 precision, and 0.52 recall scores for SARS-CoV2 positive cases. In an expert validation, the classifier correctly identified 90.8% (79/87) as COVID-19 positive and 97.8% (91/93) as not SARS-CoV2 positive. The classifier identified an additional 960 positive cases that did not have SARS-CoV2 lab tests in hospital, and only 177 of those cases had the ICD-10 code for COVID-19.

Discussion

Proxy dataset performance may be worse because these instances sometimes include discussion of pending lab tests. The most predictive features are meaningful and interpretable. The type of external test that was performed is rarely mentioned.

Conclusion

COVID-19 cases that had testing done outside of the hospital can be reliably detected from the text in EHRs. Training on a proxy dataset was a suitable method for developing a highly performant classifier without labor intensive labeling efforts.

Comprehensive characterization of putative genetic influences on plasma metabolome in a pediatric cohort

BACKGROUND

The human exposome is composed of diverse metabolites and small chemical compounds originated from endogenous and exogenous sources, respectively. Genetic and environmental factors influence metabolite levels, while the extent of genetic contributions across metabolic pathways is not yet known. Untargeted profiling of human metabolome using high-resolution mass spectrometry (HRMS) combined with genome-wide genotyping allows comprehensive identification of genetically influenced metabolites. As such previous studies of adults discovered and replicated genotype-metabotype associations. However, these associations have not been characterized in children.

RESULTS

We conducted the largest genome by metabolome-wide association study to date of children (N = 441) using 619,688 common genetic variants and 14,342 features measured by HRMS. Narrow-sense heritability (h²) estimates of plasma metabolite concentrations using genomic relatedness matrix restricted maximum likelihood (GREML) method showed a bimodal distribution with high h² (> 0.8) for 15.9% of features and low h² (< 0.2) for most of features (62.0%). The features with high h² were enriched for amino acid and nucleic acid metabolism, while carbohydrate and lipid concentrations showed low h². For each feature, a metabolite quantitative trait loci (mQTL) analysis was performed to identify genetic variants that were potentially associated with plasma levels. Fifty-four associations among 29 features and 43 genetic variants were identified at a genome-wide significance threshold p < 3.5 × 10^-12 (= 5 × 10^-8/14,342 features). Previously reported associations such as UGT1A1 and bilirubin; PYROXD2 and methyl lysine; and ACADS and butyrylcarnitine were successfully replicated in our pediatric cohort. We found potential candidates for novel associations including CSMD1 and a monostearyl alcohol triglyceride (m/z 781.7483, retention time (RT) 89.3 s); CALN1 and Tridecanol (m/z 283.2741, RT 27.6). A gene-level enrichment analysis using MAGMA revealed highly interconnected modules for dADP biosynthesis, sterol synthesis, and long-chain fatty acid transport in the gene-feature network.

CONCLUSION

Comprehensive profiling of plasma metabolome across age groups combined with genome-wide genotyping revealed a wide range of genetic influence on diverse chemical species and metabolic pathways. The developmental trajectory of a biological system is shaped by gene-environment interaction especially in early life. Therefore, continuous efforts on generating metabolomics data in diverse human tissue types across age groups are required to understand gene-environment interaction toward healthy aging trajectories.

The Patient Role in a Federal National-Scale Health Information Exchange

The federal Trusted Exchange Framework and Common Agreement (TEFCA) aims to reduce fragmentation of patient records by expanding query-based health information exchange with nationwide connectivity for diverse purposes. TEFCA provides a common agreement and security framework allowing clinicians, and possibly insurance company staff, public health officials, and other authorized users, to query for health information about hundreds of millions of patients. TEFCA presents an opportunity to weave information exchange into the fabric of our national health information economy. We define 3 principles to promote patient autonomy and control within TEFCA: (1) patients can query for data about themselves, (2) patients can know when their data are queried and shared, and (3) patients can configure what is shared about them. We believe TEFCA should address these principles by the time it launches. While health information exchange already occurs on a large scale today, the launch of TEFCA introduces a major, new, and cohesive component of 21st-century US health care information infrastructure. We strongly advocate for a substantive role for the patient in TEFCA, one that will be a model for other systems and policies.

Evaluation of Suicides Among US Adolescents During the COVID-19 Pandemic

This cross-sectional study examines the pattern of suicides from 2015 through 2020 among youth aged 10 to 19 years in 14 US states.

International comparisons of laboratory values from the 4CE collaborative to predict COVID-19 mortality

Given the growing number of prediction algorithms developed to predict COVID-19 mortality, we evaluated the transportability of a mortality prediction algorithm using a multi-national network of healthcare systems. We predicted COVID-19 mortality using baseline commonly measured laboratory values and standard demographic and clinical covariates across healthcare systems, countries, and continents. Specifically, we trained a Cox regression model with nine measured laboratory test values, standard demographics at admission, and comorbidity burden pre-admission. These models were compared at site, country, and continent level. Of the 39,969 hospitalized patients with COVID-19 (68.6% male), 5717 (14.3%) died. In the Cox model, age, albumin, AST, creatine, CRP, and white blood cell count are most predictive of mortality. The baseline covariates are more predictive of mortality during the early days of COVID-19 hospitalization. Models trained at healthcare systems with larger cohort size largely retain good transportability performance when porting to different sites. The combination of routine laboratory test values at admission along with basic demographic features can predict mortality in patients hospitalized with COVID-19. Importantly, this potentially deployable model differs from prior work by demonstrating not only consistent performance but also reliable transportability across healthcare systems in the US and Europe, highlighting the generalizability of this model and the overall approach.

How NFTs could transform health information exchange

[Figure: see text].

Data-driven clustering identifies features distinguishing multisystem inflammatory syndrome from acute COVID-19 in children and adolescents

BACKGROUND

Multisystem inflammatory syndrome in children (MIS-C) consensus criteria were designed for maximal sensitivity and therefore capture patients with acute COVID-19 pneumonia.

METHODS

We performed unsupervised clustering on data from 1,526 patients (684 labeled MIS-C by clinicians) <21 years old hospitalized with COVID-19-related illness admitted between 15 March 2020 and 31 December 2020. We compared prevalence of assigned MIS-C labels and clinical features among clusters, followed by recursive feature elimination to identify characteristics of potentially misclassified MIS-C-labeled patients.

FINDINGS

Of 94 clinical features tested, 46 were retained for clustering. Cluster 1 patients (N = 498; 92% labeled MIS-C) were mostly previously healthy (71%), with mean age 7·2 ± 0·4 years, predominant cardiovascular (77%) and/or mucocutaneous (82%) involvement, high inflammatory biomarkers, and mostly SARS-CoV-2 PCR negative (60%). Cluster 2 patients (N = 445; 27% labeled MIS-C) frequently had pre-existing conditions (79%, with 39% respiratory), were similarly 7·4 ± 2·1 years old, and commonly had chest radiograph infiltrates (79%) and positive PCR testing (90%). Cluster 3 patients (N = 583; 19% labeled MIS-C) were younger (2·8 ± 2·0 y), PCR positive (86%), with less inflammation. Radiographic findings of pulmonary infiltrates and positive SARS-CoV-2 PCR accurately distinguished cluster 2 MIS-C labeled patients from cluster 1 patients.

INTERPRETATION

Using a data driven, unsupervised approach, we identified features that cluster patients into a group with high likelihood of having MIS-C. Other features identified a cluster of patients more likely to have acute severe COVID-19 pulmonary disease, and patients in this cluster labeled by clinicians as MIS-C may be misclassified. These data driven phenotypes may help refine the diagnosis of MIS-C.

Machine-learning-based predictions of direct-acting antiviral therapy duration for patients with hepatitis C

INTRODUCTION

Hepatitis C, which affects 71 million persons worldwide, is the most common blood-borne pathogen in the United States. Chronic infections can be treated effectively thanks to the availability of modern direct-acting antiviral (DAA) therapies. Real-world data on the duration of DAA therapy, which can be used to optimize and guide the course of therapy, may also be useful in determining quality of life enhancements based upon total required supply of medication and long-term improvements to quality of life. We developed a machine learning model to identify patient characteristics associated with prolonged DAA treatment duration.

METHODS

A nationwide U.S. commercial managed care plan with claims data that covers about 60 million beneficiaries from 2009 to 2019 were used in the retrospective study. We examined differences in age, gender, and multiple comorbidities among patients treated with different durations of DAA treatment. We also examined the performance of machine learning models for predicting a prolonged course of DAA based on the area under the receiver operating characteristic curve (AUC).

RESULTS

We identified 3943 cases with hepatitis C who received sofosbuvir/ledipasvir as the first course of DAA and were eligible for the study. Patients receiving prolonged treatment (n = 240, 6.1%) were more likely to have compensated cirrhosis, decompensated cirrhosis, and other comorbidities (P < 0.001). For distinguishing patients who received prolonged DAA treatment for hepatitis C from patients received standard treatment, the optimal predictive model, constructed with XGBoost, had an AUC of 0.745 ± 0.031 (P < 0.001).

CONCLUSIONS

The risk of antiviral resistance and the cost of DAA are strong motivators to ensure that first-round DAA therapy is effective. For the dominant DAA treatment during the course of this analysis, we present a model that identifies factors already captured in established guidelines and adds to those age, comorbidity burden, and type 2 diabetes status; patient characteristics that are predictive of extended treatment.

Increased Prevalence of Familial Autoimmune Disease in Children With Opsoclonus-Myoclonus Syndrome

Background and Objectives

Opsoclonus-myoclonus syndrome (OMS) is a rare autoimmune disorder associated with neuroblastoma in children, although idiopathic and postinfectious etiologies are present in children and adults. Small cohort studies in homogenous populations have revealed elevated rates of autoimmunity in family members of patients with OMS, although no differentiation between paraneoplastic and nonparaneoplastic forms has been performed. The objective of this study was to investigate the prevalence of autoimmune disease in first-degree relatives of pediatric patients with paraneoplastic and nonparaneoplastic OMS.

Methods

A single-center cohort study of consecutively evaluated children with OMS was performed. Parents of patients were prospectively administered surveys on familial autoimmune disease. Rates of autoimmune disease in first-degree relatives of pediatric patients with OMS were compared using Fisher exact t test and χ² analysis: (1) between those with and without a paraneoplastic cause and (2) between healthy and disease (pediatric multiple sclerosis [MS]) controls from the United States Pediatric MS Network.

Results

Thirty-five patients (18 paraneoplastic, median age at onset 19.0 months; 17 idiopathic, median age at onset 25.0 months) and 68 first-degree relatives (median age 41.9 years) were enrolled. One patient developed systemic lupus erythematosus 7 years after OMS onset. Paraneoplastic OMS was associated with a 50% rate of autoimmune disease in a first-degree relative compared with 29% in idiopathic OMS (p = 0.31). The rate of first-degree relative autoimmune disease per OMS case (14/35, 40%) was higher than healthy controls (86/709, 12%; p < 0.001) and children with pediatric MS (101/495, 20%; p = 0.007).

Discussion

In a cohort of pediatric patients with OMS, there were elevated rates of first-degree relative autoimmune disease, with no difference in rates observed between paraneoplastic and idiopathic etiologies, suggesting an autoimmune genetic contribution to the development of OMS in children.

Validation of an internationally derived patient severity phenotype to support COVID-19 analytics from electronic health record data

OBJECTIVE

The Consortium for Clinical Characterization of COVID-19 by EHR (4CE) is an international collaboration addressing coronavirus disease 2019 (COVID-19) with federated analyses of electronic health record (EHR) data. We sought to develop and validate a computable phenotype for COVID-19 severity.

MATERIALS AND METHODS

Twelve 4CE sites participated. First, we developed an EHR-based severity phenotype consisting of 6 code classes, and we validated it on patient hospitalization data from the 12 4CE clinical sites against the outcomes of intensive care unit (ICU) admission and/or death. We also piloted an alternative machine learning approach and compared selected predictors of severity with the 4CE phenotype at 1 site.

RESULTS

The full 4CE severity phenotype had pooled sensitivity of 0.73 and specificity 0.83 for the combined outcome of ICU admission and/or death. The sensitivity of individual code categories for acuity had high variability-up to 0.65 across sites. At one pilot site, the expert-derived phenotype had mean area under the curve of 0.903 (95% confidence interval, 0.886-0.921), compared with an area under the curve of 0.956 (95% confidence interval, 0.952-0.959) for the machine learning approach. Billing codes were poor proxies of ICU admission, with as low as 49% precision and recall compared with chart review.

DISCUSSION

We developed a severity phenotype using 6 code classes that proved resilient to coding variability across international institutions. In contrast, machine learning approaches may overfit hospital-specific orders. Manual chart review revealed discrepancies even in the gold-standard outcomes, possibly owing to heterogeneous pandemic conditions.

CONCLUSIONS

We developed an EHR-based severity phenotype for COVID-19 in hospitalized patients and validated it at 12 international sites.

Patient-led data sharing for clinical bioinformatics research: USCDI and beyond

The 21st Century Cures Act, passed in 2016, and the Final Rules it called for create a roadmap for enabling patient access to their electronic health information. The set of data to be made available, as determined by the Office of the National Coordinator for Health IT through the US Core Data for Interoperability expansion process, will impact the value creation of this improved data liquidity. In this commentary, we look at the potential for significant value creation from USCDI in the context of clinical bioinformatics research and advocate for the research community's involvement in the USCDI process to propel this value creation forward. We also describe 1 mechanism-using existing required APIs for full data export capabilities-that could pragmatically enable this value creation at minimal additional technical lift beyond the current regulatory requirements.

Creative Approaches for Assessing Long-term Outcomes in Children

Advances in new technologies, when incorporated into routine health screening, have tremendous promise to benefit children. The number of health screening tests, many of which have been developed with machine learning or genomics, has exploded. To assess efficacy of health screening, ideally, randomized trials of screening in youth would be conducted; however, these can take years to conduct and may not be feasible. Thus, innovative methods to evaluate the long-term outcomes of screening are needed to help clinicians and policymakers make informed decisions. These methods include using longitudinal and linked-data systems to evaluate screening in clinical and community settings, school data, simulation modeling approaches, and methods that take advantage of data available in the digital and genomic age. Future research is needed to evaluate how longitudinal and linked-data systems drawing on community and clinical settings can enable robust evaluations of the effects of screening on changes in health status. Additionally, future studies are needed to benchmark participating individuals and communities against similar counterparts and to link big data with natural experiments related to variation in screening policies. These novel approaches have great potential for identifying and addressing differences in access to screening and effectiveness of screening across population groups and communities.

Medication utilization in children born preterm in the first two years of life

OBJECTIVE

To compare medications dispensed during the first 2 years in children born preterm and full-term.

STUDY DESIGN

Retrospective analysis of claims data from a commercial national managed care plan 2008-2019. 329,855 beneficiaries were enrolled from birth through 2 years, of which 25,408 (7.7%) were preterm (<37 weeks). Filled prescription claims and paid amount over 2 years were identified.

RESULTS

In preterm children, the number of filled prescriptions was 1.4 times and cost was 3.8 times that of full-term children. Number and cost of medications were inversely related to gestational age. Differences peak at 4-9 months and resolve by 19 months after discharge. Palivizumab, ranitidine, albuterol, lansoprazole, budesonide, and prednisolone had the greatest differences in utilization.

CONCLUSION

Prescription medication utilization among preterm children under 2 years is driven by palivizumab, anti-reflux, and respiratory medications, despite little evidence regarding efficacy for many medications and concern for harm with certain classes.

Characterisation of Pediatric Pulmonary Hypertensive Vascular Disease from the PPHNet Registry

BACKGROUND

There are limited data about the range of diseases, natural history, age-appropriate endpoints and optimal care for children with pulmonary hypertension (PH), including the need for developing high quality patient registries of children with diverse forms of PH to enhance care and research.

OBJECTIVE

To characterise the distribution and clinical features of diseases associated with pediatric PH, including natural history, evaluation, therapeutic interventions and outcomes, as defined by the WSPH Classification.

METHODS

1475 patients were enrolled into a multisite registry across the Pediatric Pulmonary Hypertension Network (PPHNet), comprised of 8 interdisciplinary PH programs.

RESULTS

WSPH Groups 1 (PAH) and 3 (lung disease) were the most common primary classifications (45% and 49% of subjects, respectively). The most common Group 3 conditions were BPD and CDH. Group 1 disease was predominantly associated with congenital heart disease (60%) and idiopathic (23% of Group 1 cases). In comparison with Group 1, Group 3 subjects had better disease resolution (HR=3.1, p<0.001), tended to be younger at diagnosis (0.3 (0.0,0.6) versus 1.6 (0.1,6.9) years (median (IQR); p<0.001), and were more often male (57% versus. 45%, p<0.001). Down syndrome (DS), the most common genetic syndrome in the registry, constituted 11% of the entire PH cohort.

CONCLUSIONS

We find a striking proportion of pediatric PH patients with Group 3 disorders, reflecting the growing recognition of PH in diverse developmental lung diseases. Greater precision of clinical phenotyping based on disease-specific characterization may further enhance care and research of pediatric PH.

A landscape survey of planned SMART/HL7 bulk FHIR data access API implementations and tools

The Office of National Coordinator for Health Information Technology final rule implementing the interoperability and information blocking provisions of the 21st Century Cures Act requires support for two SMART (Substitutable Medical Applications, Reusable Technologies) application programming interfaces (APIs) and instantiates Health Level Seven International (HL7) Fast Healthcare Interoperability Resources (FHIR) as a lingua franca for health data. We sought to assess the current state and near-term plans for the SMART/HL7 Bulk FHIR Access API implementation across organizations including electronic health record vendors, cloud vendors, public health contractors, research institutions, payors, FHIR tooling developers, and other purveyors of health information technology platforms. We learned that many organizations not required through regulation to use standardized bulk data are rapidly implementing the API for a wide array of use cases. This may portend an unprecedented level of standardized population-level health data exchange that will support an apps and analytics ecosystem. Feedback from early adopters on the API's limitations and unsolved problems in the space of population health are highlighted.

International Analysis of Electronic Health Records of Children and Youth Hospitalized With COVID-19 Infection in 6 Countries

IMPORTANCE

Additional sources of pediatric epidemiological and clinical data are needed to efficiently study COVID-19 in children and youth and inform infection prevention and clinical treatment of pediatric patients.

OBJECTIVE

To describe international hospitalization trends and key epidemiological and clinical features of children and youth with COVID-19.

DESIGN, SETTING, AND PARTICIPANTS

This retrospective cohort study included pediatric patients hospitalized between February 2 and October 10, 2020. Patient-level electronic health record (EHR) data were collected across 27 hospitals in France, Germany, Spain, Singapore, the UK, and the US. Patients younger than 21 years who tested positive for COVID-19 and were hospitalized at an institution participating in the Consortium for Clinical Characterization of COVID-19 by EHR were included in the study.

MAIN OUTCOMES AND MEASURES

Patient characteristics, clinical features, and medication use.

RESULTS

There were 347 males (52%; 95% CI, 48.5-55.3) and 324 females (48%; 95% CI, 44.4-51.3) in this study's cohort. There was a bimodal age distribution, with the greatest proportion of patients in the 0- to 2-year (199 patients [30%]) and 12- to 17-year (170 patients [25%]) age range. Trends in hospitalizations for 671 children and youth found discrete surges with variable timing across 6 countries. Data from this cohort mirrored national-level pediatric hospitalization trends for most countries with available data, with peaks in hospitalizations during the initial spring surge occurring within 23 days in the national-level and 4CE data. A total of 27 364 laboratory values for 16 laboratory tests were analyzed, with mean values indicating elevations in markers of inflammation (C-reactive protein, 83 mg/L; 95% CI, 53-112 mg/L; ferritin, 417 ng/mL; 95% CI, 228-607 ng/mL; and procalcitonin, 1.45 ng/mL; 95% CI, 0.13-2.77 ng/mL). Abnormalities in coagulation were also evident (D-dimer, 0.78 ug/mL; 95% CI, 0.35-1.21 ug/mL; and fibrinogen, 477 mg/dL; 95% CI, 385-569 mg/dL). Cardiac troponin, when checked (n = 59), was elevated (0.032 ng/mL; 95% CI, 0.000-0.080 ng/mL). Common complications included cardiac arrhythmias (15.0%; 95% CI, 8.1%-21.7%), viral pneumonia (13.3%; 95% CI, 6.5%-20.1%), and respiratory failure (10.5%; 95% CI, 5.8%-15.3%). Few children were treated with COVID-19-directed medications.

CONCLUSIONS AND RELEVANCE

This study of EHRs of children and youth hospitalized for COVID-19 in 6 countries demonstrated variability in hospitalization trends across countries and identified common complications and laboratory abnormalities in children and youth with COVID-19 infection. Large-scale informatics-based approaches to integrate and analyze data across health care systems complement methods of disease surveillance and advance understanding of epidemiological and clinical features associated with COVID-19 in children and youth.

WEScover: selection between clinical whole exome sequencing and gene panel testing

Background

Whole exome sequencing (WES) is widely adopted in clinical and research settings; however, one of the practical concerns is the potential false negatives due to incomplete breadth and depth of coverage for several exons in clinically implicated genes. In some cases, a targeted gene panel testing may be a dependable option to ascertain true negatives for genomic variants in known disease-associated genes. We developed a web-based tool to quickly gauge whether all genes of interest would be reliably covered by WES or whether targeted gene panel testing should be considered instead to minimize false negatives in candidate genes.

Results

WEScover is a novel web application that provides an intuitive user interface for discovering breadth and depth of coverage across population-scale WES datasets, searching either by phenotype, by targeted gene panel(s) or by gene(s). Moreover, the application shows metrics from the Genome Aggregation Database to provide gene-centric view on breadth of coverage.

Conclusions

WEScover allows users to efficiently query genes and phenotypes for the coverage of associated exons by WES and recommends use of panel tests for the genes with potential incomplete coverage by WES.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12859-021-04178-5.

Adverse drug event rates in pediatric pulmonary hypertension: a comparison of real-world data sources

Objective

Real-world data (RWD) are increasingly used for pharmacoepidemiology and regulatory innovation. Our objective was to compare adverse drug event (ADE) rates determined from two RWD sources, electronic health records and administrative claims data, among children treated with drugs for pulmonary hypertension.

Materials and Methods

Textual mentions of medications and signs/symptoms that may represent ADEs were identified in clinical notes using natural language processing. Diagnostic codes for the same signs/symptoms were identified in our electronic data warehouse for the patients with textual evidence of taking pulmonary hypertension-targeted drugs. We compared rates of ADEs identified in clinical notes to those identified from diagnostic code data. In addition, we compared putative ADE rates from clinical notes to those from a healthcare claims dataset from a large, national insurer.

Results

Analysis of clinical notes identified up to 7-fold higher ADE rates than those ascertained from diagnostic codes. However, certain ADEs (eg, hearing loss) were more often identified in diagnostic code data. Similar results were found when ADE rates ascertained from clinical notes and national claims data were compared.

Discussion

While administrative claims and clinical notes are both increasingly used for RWD-based pharmacovigilance, ADE rates substantially differ depending on data source.

Conclusion

Pharmacovigilance based on RWD may lead to discrepant results depending on the data source analyzed. Further work is needed to confirm the validity of identified ADEs, to distinguish them from disease effects, and to understand tradeoffs in sensitivity and specificity between data sources.

What Every Reader Should Know About Studies Using Electronic Health Record Data but May Be Afraid to Ask

Coincident with the tsunami of COVID-19–related publications, there has been a surge of studies using real-world data, including those obtained from the electronic health record (EHR). Unfortunately, several of these high-profile publications were retracted because of concerns regarding the soundness and quality of the studies and the EHR data they purported to analyze. These retractions highlight that although a small community of EHR informatics experts can readily identify strengths and flaws in EHR-derived studies, many medical editorial teams and otherwise sophisticated medical readers lack the framework to fully critically appraise these studies. In addition, conventional statistical analyses cannot overcome the need for an understanding of the opportunities and limitations of EHR-derived studies. We distill here from the broader informatics literature six key considerations that are crucial for appraising studies utilizing EHR data: data completeness, data collection and handling (eg, transformation), data type (ie, codified, textual), robustness of methods against EHR variability (within and across institutions, countries, and time), transparency of data and analytic code, and the multidisciplinary approach. These considerations will inform researchers, clinicians, and other stakeholders as to the recommended best practices in reviewing manuscripts, grants, and other outputs from EHR-data derived studies, and thereby promote and foster rigor, quality, and reliability of this rapidly growing field.

A proposal for shoring up Federal Trade Commission protections for electronic health record-connected consumer apps under 21st Century Cures

Under the 21st Century Cures Act and the Office of the National Coordinator for Health Information Technology (ONC) rule implementing its interoperability provisions, a patient’s rights to easily request and obtain digital access to portions of their medical records are now supported by both technology and policy. Data, once directed by a patient to leave a Health Insurance Portability and Accountability Act–covered health entity and enter a consumer app, will usually fall under Federal Trade Commission oversight. Because the statutory authority of the ONC does not extend to health data protection, there is not yet regulation to specifically address privacy protections for consumer apps. A technologically feasible workflow that could be widely adopted and permissible under ONC’s rule, involves using the SMART on FHIR OAuth authorization routine to present standardized information about app behavior. This approach would not bias the patient in a way that triggers penalties under information blocking provisions of the rule.

A high-throughput phenotyping algorithm is portable from adult to pediatric populations

OBJECTIVE

Multimodal automated phenotyping (MAP) is a scalable, high-throughput phenotyping method, developed using electronic health record (EHR) data from an adult population. We tested transportability of MAP to a pediatric population.

MATERIALS AND METHODS

Without additional feature engineering or supervised training, we applied MAP to a pediatric population enrolled in a biobank and evaluated performance against physician-reviewed medical records. We also compared performance of MAP at the pediatric institution and the original adult institution where MAP was developed, including for 6 phenotypes validated at both institutions against physician-reviewed medical records.

RESULTS

MAP performed equally well in the pediatric setting (average AUC 0.98) as it did at the general adult hospital system (average AUC 0.96). MAP's performance in the pediatric sample was similar across the 6 specific phenotypes also validated against gold-standard labels in the adult biobank.

CONCLUSIONS

MAP is highly transportable across diverse populations and has potential for wide-scale use.

International Comparisons of Harmonized Laboratory Value Trajectories to Predict Severe COVID-19: Leveraging the 4CE Collaborative Across 342 Hospitals and 6 Countries: A Retrospective Cohort Study

Objectives

To perform an international comparison of the trajectory of laboratory values among hospitalized patients with COVID-19 who develop severe disease and identify optimal timing of laboratory value collection to predict severity across hospitals and regions.

Design

Retrospective cohort study.

Setting

The Consortium for Clinical Characterization of COVID-19 by EHR (4CE), an international multi-site data-sharing collaborative of 342 hospitals in the US and in Europe.

Participants

Patients hospitalized with COVID-19, admitted before or after PCR-confirmed result for SARS-CoV-2.

Primary and secondary outcome measures

Patients were categorized as "ever-severe" or "never-severe" using the validated 4CE severity criteria. Eighteen laboratory tests associated with poor COVID-19-related outcomes were evaluated for predictive accuracy by area under the curve (AUC), compared between the severity categories. Subgroup analysis was performed to validate a subset of laboratory values as predictive of severity against a published algorithm. A subset of laboratory values (CRP, albumin, LDH, neutrophil count, D-dimer, and procalcitonin) was compared between North American and European sites for severity prediction.

Results

Of 36,447 patients with COVID-19, 19,953 (43.7%) were categorized as ever-severe. Most patients (78.7%) were 50 years of age or older and male (60.5%). Longitudinal trajectories of CRP, albumin, LDH, neutrophil count, D-dimer, and procalcitonin showed association with disease severity. Significant differences of laboratory values at admission were found between the two groups. With the exception of D-dimer, predictive discrimination of laboratory values did not improve after admission. Sub-group analysis using age, D-dimer, CRP, and lymphocyte count as predictive of severity at admission showed similar discrimination to a published algorithm (AUC=0.88 and 0.91, respectively). Both models deteriorated in predictive accuracy as the disease progressed. On average, no difference in severity prediction was found between North American and European sites.

Conclusions

Laboratory test values at admission can be used to predict severity in patients with COVID-19. Prediction models show consistency across international sites highlighting the potential generalizability of these models.

GenoPheno: cataloging large-scale phenotypic and next-generation sequencing data within human datasets

Precision medicine promises to revolutionize treatment, shifting therapeutic approaches from the classical one-size-fits-all to those more tailored to the patient's individual genomic profile, lifestyle and environmental exposures. Yet, to advance precision medicine's main objective-ensuring the optimum diagnosis, treatment and prognosis for each individual-investigators need access to large-scale clinical and genomic data repositories. Despite the vast proliferation of these datasets, locating and obtaining access to many remains a challenge. We sought to provide an overview of available patient-level datasets that contain both genotypic data, obtained by next-generation sequencing, and phenotypic data-and to create a dynamic, online catalog for consultation, contribution and revision by the research community. Datasets included in this review conform to six specific inclusion parameters that are: (i) contain data from more than 500 human subjects; (ii) contain both genotypic and phenotypic data from the same subjects; (iii) include whole genome sequencing or whole exome sequencing data; (iv) include at least 100 recorded phenotypic variables per subject; (v) accessible through a website or collaboration with investigators and (vi) make access information available in English. Using these criteria, we identified 30 datasets, reviewed them and provided results in the release version of a catalog, which is publicly available through a dynamic Web application and on GitHub. Users can review as well as contribute new datasets for inclusion (Web: https://avillachlab.shinyapps.io/genophenocatalog/; GitHub: https://github.com/hms-dbmi/GenoPheno-CatalogShiny).

Privacy protections to encourage use of health-relevant digital data in a learning health system

The National Academy of Medicine has long advocated for a "learning healthcare system" that produces constantly updated reference data during the care process. Moving toward a rapid learning system to solve intractable problems in health demands a balance between protecting patients and making data available to improve health and health care. Public concerns in the U.S. about privacy and the potential for unethical or harmful uses of this data, if not proactively addressed, could upset this balance. New federal laws prioritize sharing health data, including with patient digital tools. U.S. health privacy laws do not cover data collected by many consumer digital technologies and have not been updated to address concerns about the entry of large technology companies into health care. Further, there is increasing recognition that many classes of data not traditionally considered to be healthcare-related, for example consumer credit histories, are indeed predictive of health status and outcomes. We propose a multi-pronged approach to protecting health-relevant data while promoting and supporting beneficial uses and disclosures to improve health and health care for individuals and populations. Such protections should apply to entities collecting health-relevant data regardless of whether they are covered by federal health privacy laws. We focus largely on privacy but also address protections against harms as a critical component of a comprehensive approach to governing health-relevant data. U.S. policymakers and regulators should consider these recommendations in crafting privacy bills and rules. However, our recommendations also can inform best practices even in the absence of new federal requirements.

A federated EHR network data completeness tracking system

OBJECTIVE

The study sought to design, pilot, and evaluate a federated data completeness tracking system (CTX) for assessing completeness in research data extracted from electronic health record data across the Accessible Research Commons for Health (ARCH) Clinical Data Research Network.

MATERIALS AND METHODS

The CTX applies a systems-based approach to design workflow and technology for assessing completeness across distributed electronic health record data repositories participating in a queryable, federated network. The CTX invokes 2 positive feedback loops that utilize open source tools (DQe-c and Vue) to integrate technology and human actors in a system geared for increasing capacity and taking action. A pilot implementation of the system involved 6 ARCH partner sites between January 2017 and May 2018.

RESULTS

The ARCH CTX has enabled the network to monitor and, if needed, adjust its data management processes to maintain complete datasets for secondary use. The system allows the network and its partner sites to profile data completeness both at the network and partner site levels. Interactive visualizations presenting the current state of completeness in the context of the entire network as well as changes in completeness across time were valued among the CTX user base.

DISCUSSION

Distributed clinical data networks are complex systems. Top-down approaches that solely rely on technology to report data completeness may be necessary but not sufficient for improving completeness (and quality) of data in large-scale clinical data networks. Improving and maintaining complete (high-quality) data in such complex environments entails sociotechnical systems that exploit technology and empower human actors to engage in the process of high-quality data curating.

CONCLUSIONS

The CTX has increased the network's capacity to rapidly identify data completeness issues and empowered ARCH partner sites to get involved in improving the completeness of respective data in their repositories.

The 21st Century Cures Act: A Competitive Apps Market and the Risk of Innovation Blocking

The 21st Century Cures Act and the recently published "final rule" define standardized methods for obtaining electronic copies of electronic health record (EHR) data through application programming interfaces. The rule is meant to create an ecosystem of reusable, substitutable apps that can be built once but run at any hospital system "without special effort." Yet, despite numerous provisions around information blocking in the final rule, there is concern that the business practices that govern EHR vendors and health care organizations in the United States could still stifle innovation. We describe potential app ecosystems that may form. We caution that misaligned incentives may result in anticompetitive behavior and purposefully limited functionality. Closed proprietary ecosystems may result, limiting the value derived from interoperability. The 21st Century Cures Act and final rule are an exciting step in the direction of improved interoperability. However, realizing the vision of a truly interoperable app ecosystem is not predetermined.

Push Button Population Health: The SMART/HL7 FHIR Bulk Data Access Application Programming Interface

The 21st Century Cures Act requires that certified health information technology have an application programming interface (API) giving access to all data elements of a patient's electronic health record, "without special effort". In the spring of 2020, the Office of the National Coordinator of Health Information Technology (ONC) published a rule-21st Century Cures Act Interoperability, Information Blocking, and the ONC Health IT Certification Program-regulating the API requirement along with protections against information blocking. The rule specifies the SMART/HL7 FHIR Bulk Data Access API, which enables access to patient-level data across a patient population, supporting myriad use cases across healthcare, research, and public health ecosystems. The API enables "push button population health" in that core data elements can readily and standardly be extracted from electronic health records, enabling local, regional, and national-scale data-driven innovation.

Limited Role of Bots in Spreading Vaccine-Critical Information Among Active Twitter Users in the United States: 2017-2019

Objectives. To examine the role that bots play in spreading vaccine information on Twitter by measuring exposure and engagement among active users from the United States.Methods. We sampled 53 188 US Twitter users and examined who they follow and retweet across 21 million vaccine-related tweets (January 12, 2017-December 3, 2019). Our analyses compared bots to human-operated accounts and vaccine-critical tweets to other vaccine-related tweets.Results. The median number of potential exposures to vaccine-related tweets per user was 757 (interquartile range [IQR] = 168-4435), of which 27 (IQR = 6-169) were vaccine critical, and 0 (IQR = 0-12) originated from bots. We found that 36.7% of users retweeted vaccine-related content, 4.5% retweeted vaccine-critical content, and 2.1% retweeted vaccine content from bots. Compared with other users, the 5.8% for whom vaccine-critical tweets made up most exposures more often retweeted vaccine content (62.9%; odds ratio [OR] = 2.9; 95% confidence interval [CI] = 2.7, 3.1), vaccine-critical content (35.0%; OR = 19.0; 95% CI = 17.3, 20.9), and bots (8.8%; OR = 5.4; 95% CI = 4.7, 6.3).Conclusions. A small proportion of vaccine-critical information that reaches active US Twitter users comes from bots.