Causal machine learning for predicting treatment outcomes

Causal machine learning (ML) offers flexible, data-driven methods for predicting treatment outcomes including efficacy and toxicity, thereby supporting the assessment and safety of drugs. A key benefit of causal ML is that it allows for estimating individualized treatment effects, so that clinical decision-making can be personalized to individual patient profiles. Causal ML can be used in combination with both clinical trial data and real-world data, such as clinical registries and electronic health records, but caution is needed to avoid biased or incorrect predictions. In this Perspective, we discuss the benefits of causal ML (relative to traditional statistical or ML approaches) and outline the key components and steps. Finally, we provide recommendations for the reliable use of causal ML and effective translation into the clinic.

Disease progression strikingly differs in research and real-world Parkinson's populations

Characterization of Parkinson's disease (PD) progression using real-world evidence could guide clinical trial design and identify subpopulations. Efforts to curate research populations, the increasing availability of real-world data, and advances in natural language processing, particularly large language models, allow for a more granular comparison of populations than previously possible. This study includes two research populations and two real-world data-derived (RWD) populations. The research populations are the Harvard Biomarkers Study (HBS, N = 935), a longitudinal biomarkers cohort study with in-person structured study visits; and Fox Insights (N = 36,660), an online self-survey-based research study of the Michael J. Fox Foundation. Real-world cohorts are the Optum Integrated Claims-electronic health records (N = 157,475), representing wide-scale linked medical and claims data and de-identified data from Mass General Brigham (MGB, N = 22,949), an academic hospital system. Structured, de-identified electronic health records data at MGB are supplemented using a manually validated natural language processing with a large language model to extract measurements of PD progression. Motor and cognitive progression scores change more rapidly in MGB than HBS (median survival until H&Y 3: 5.6 years vs. >10, p < 0.001; mini-mental state exam median decline 0.28 vs. 0.11, p < 0.001; and clinically recognized cognitive decline, p = 0.001). In real-world populations, patients are diagnosed more than eleven years later (RWD mean of 72.2 vs. research mean of 60.4, p < 0.001). After diagnosis, in real-world cohorts, treatment with PD medications has initiated an average of 2.3 years later (95% CI: [2.1-2.4]; p < 0.001). This study provides a detailed characterization of Parkinson's progression in diverse populations. It delineates systemic divergences in the patient populations enrolled in research settings vs. patients in the real-world. These divergences are likely due to a combination of selection bias and real population differences, but exact attribution of the causes is challenging. This study emphasizes a need to utilize multiple data sources and to diligently consider potential biases when planning, choosing data sources, and performing downstream tasks and analyses.

Disease progression strikingly differs in research and real-world Parkinson's populations

Characterization of Parkinson's disease (PD) progression using real-world evidence could guide clinical trial design and identify subpopulations. Efforts to curate research populations, the increasing availability of real-world data and recent advances in natural language processing, particularly large language models, allow for a more granular comparison of populations and the methods of data collection describing these populations than previously possible. This study includes two research populations and two real-world data derived (RWD) populations. The research populations are the Harvard Biomarkers Study (HBS, N = 935), a longitudinal biomarkers cohort study with in-person structured study visits; and Fox Insights (N = 36,660), an online self-survey-based research study of the Michael J. Fox Foundation. Real-world cohorts are the Optum Integrated Claims-electronic health records (N = 157,475), representing wide-scale linked medical and claims data and de-identified data from Mass General Brigham (MGB, N = 22,949), an academic hospital system. Structured, de-identified electronic health records data at MGB are supplemented using natural language processing with a large language model to extract measurements of PD progression. This extraction process is manually validated for accuracy. Motor and cognitive progression scores change more rapidly in MGB than HBS (median survival until H&Y 3: 5.6 years vs. >10, p<0.001; mini-mental state exam median decline 0.28 vs. 0.11, p<0.001; and clinically recognized cognitive decline, p=0.001). In the real-world populations, patients are diagnosed more than eleven years later (RWD mean of 72.2 vs. research mean of 60.4, p<0.001). After diagnosis, in real-world cohorts, treatment with PD medications is initiated 2.3 years later on average (95% CI: [2.1-2.4]; p<0.001). This study provides a detailed characterization of Parkinson's progression in diverse populations. It delineates systemic divergences in the patient populations enrolled in research settings vs. patients in the real world. These divergences are likely due to a combination of selection bias and real population differences, but exact attribution of the causes is challenging using existing data. This study emphasizes a need to utilize multiple data sources and to diligently consider potential biases when planning, choosing data sources, and performing downstream tasks and analyses.

Joint, multifaceted genomic analysis enables diagnosis of diverse, ultra-rare monogenic presentations

Genomics for rare disease diagnosis has advanced at a rapid pace due to our ability to perform "N-of-1" analyses on individual patients. The increasing sizes of ultra-rare, "N-of-1" disease cohorts internationally newly enables cohort-wide analyses for new discoveries, but well-calibrated statistical genetics approaches for jointly analyzing these patients are still under development.1,2 The Undiagnosed Diseases Network (UDN) brings multiple clinical, research and experimental centers under the same umbrella across the United States to facilitate and scale N-of-1 analyses. Here, we present the first joint analysis of whole genome sequencing data of UDN patients across the network. We apply existing and introduce new, well-calibrated statistical methods for prioritizing disease genes with de novo recurrence and compound heterozygosity. We also detect pathways enriched with candidate and known diagnostic genes. Our computational analysis, coupled with a systematic clinical review, recapitulated known diagnoses and revealed new disease associations. We make our gene-level findings and variant-level information across the cohort available in a public-facing browser (https://dbmi-bgm.github.io/udn-browser/). These results show that N-of-1 efforts should be supplemented by a joint genomic analysis across cohorts.

Simulation of undiagnosed patients with novel genetic conditions

Rare Mendelian disorders pose a major diagnostic challenge and collectively affect 300-400 million patients worldwide. Many automated tools aim to uncover causal genes in patients with suspected genetic disorders, but evaluation of these tools is limited due to the lack of comprehensive benchmark datasets that include previously unpublished conditions. Here, we present a computational pipeline that simulates realistic clinical datasets to address this deficit. Our framework jointly simulates complex phenotypes and challenging candidate genes and produces patients with novel genetic conditions. We demonstrate the similarity of our simulated patients to real patients from the Undiagnosed Diseases Network and evaluate common gene prioritization methods on the simulated cohort. These prioritization methods recover known gene-disease associations but perform poorly on diagnosing patients with novel genetic disorders. Our publicly-available dataset and codebase can be utilized by medical genetics researchers to evaluate, compare, and improve tools that aid in the diagnostic process.

Unsupervised Anomaly Detection to Characterize Heterogeneity in Type 2 Diabetes

Diabetes is associated with heterogeneous behaviors affecting patients' clinical characteristics and trajectories. This study includes 21,288 patients with type 2 diabetes (women, ages 30 to 65). The cohort was filtered through a set of preprocessing heuristics in order to assure the cohort exhibited a similar clinical trajectory. Anomalous characteristics were then identified using dimensionality reduction and anomaly detection methods. Compared to the majority of the cohort, patients classified as anomalous were twice as likely to be admitted into the hospital (7.94[7.59 8.28] versus 3.12[3.06 3.17] times), have a higher incidence of comorbidities (2[1.64 2.36] times more), and be prescribed more insulin and less new and more expensive diabetes medications (such as Sodium glucose co-transporter 2 inhibitors). Patients with these anomalous characteristics may benefit from additional or specialized interventions to avert their risk for adverse outcomes.

SPTSSA variants alter sphingolipid synthesis and cause a complex hereditary spastic paraplegia

Sphingolipids are a diverse family of lipids with critical structural and signalling functions in the mammalian nervous system, where they are abundant in myelin membranes. Serine palmitoyltransferase, the enzyme that catalyses the rate-limiting reaction of sphingolipid synthesis, is composed of multiple subunits including an activating subunit, SPTSSA. Sphingolipids are both essential and cytotoxic and their synthesis must therefore be tightly regulated. Key to the homeostatic regulation are the ORMDL proteins that are bound to serine palmitoyltransferase and mediate feedback inhibition of enzymatic activity when sphingolipid levels become excessive. Exome sequencing identified potential disease-causing variants in SPTSSA in three children presenting with a complex form of hereditary spastic paraplegia. The effect of these variants on the catalytic activity and homeostatic regulation of serine palmitoyltransferase was investigated in human embryonic kidney cells, patient fibroblasts and Drosophila. Our results showed that two different pathogenic variants in SPTSSA caused a hereditary spastic paraplegia resulting in progressive motor disturbance with variable sensorineural hearing loss and language/cognitive dysfunction in three individuals. The variants in SPTSSA impaired the negative regulation of serine palmitoyltransferase by ORMDLs leading to excessive sphingolipid synthesis based on biochemical studies and in vivo studies in Drosophila. These findings support the pathogenicity of the SPTSSA variants and point to excessive sphingolipid synthesis due to impaired homeostatic regulation of serine palmitoyltransferase as responsible for defects in early brain development and function.

An efficient landmark model for prediction of suicide attempts in multiple clinical settings

Growing evidence has shown that applying machine learning models to large clinical data sources may exceed clinician performance in suicide risk stratification. However, many existing prediction models either suffer from "temporal bias" (a bias that stems from using case-control sampling) or require training on all available patient visit data. Here, we adopt a "landmark model" framework that aligns with clinical practice for prediction of suicide-related behaviors (SRBs) using a large electronic health record database. Using the landmark approach, we developed models for SRB prediction (regularized Cox regression and random survival forest) that establish a time-point (e.g., clinical visit) from which predictions are made over user-specified prediction windows using historical information up to that point. We applied this approach to cohorts from three clinical settings: general outpatient, psychiatric emergency department, and psychiatric inpatients, for varying prediction windows and lengths of historical data. Models achieved high discriminative performance (area under the Receiver Operating Characteristic curve 0.74-0.93 for the Cox model) across different prediction windows and settings, even with relatively short periods of historical data. In short, we developed accurate, dynamic SRB risk prediction models with the landmark approach that reduce bias and enhance the reliability and portability of suicide risk prediction models.

Bi-allelic ATG4D variants are associated with a neurodevelopmental disorder characterized by speech and motor impairment

Autophagy regulates the degradation of damaged organelles and protein aggregates, and is critical for neuronal development, homeostasis, and maintenance, yet few neurodevelopmental disorders have been associated with pathogenic variants in genes encoding autophagy-related proteins. We report three individuals from two unrelated families with a neurodevelopmental disorder characterized by speech and motor impairment, and similar facial characteristics. Rare, conserved, bi-allelic variants were identified in ATG4D, encoding one of four ATG4 cysteine proteases important for autophagosome biogenesis, a hallmark of autophagy. Autophagosome biogenesis and induction of autophagy were intact in cells from affected individuals. However, studies evaluating the predominant substrate of ATG4D, GABARAPL1, demonstrated that three of the four ATG4D patient variants functionally impair ATG4D activity. GABARAPL1 is cleaved or "primed" by ATG4D and an in vitro GABARAPL1 priming assay revealed decreased priming activity for three of the four ATG4D variants. Furthermore, a rescue experiment performed in an ATG4 tetra knockout cell line, in which all four ATG4 isoforms were knocked out by gene editing, showed decreased GABARAPL1 priming activity for the two ATG4D missense variants located in the cysteine protease domain required for priming, suggesting that these variants impair the function of ATG4D. The clinical, bioinformatic, and functional data suggest that bi-allelic loss-of-function variants in ATG4D contribute to the pathogenesis of this syndromic neurodevelopmental disorder.

National Projections for Clinical Implications of Race-Free Creatinine-Based GFR Estimating Equations

BACKGROUND

The National Kidney Foundation and American Society of Nephrology Task Force on Reassessing the Inclusion of Race in Diagnosing Kidney Disease recently recommended a new race-free creatinine-based equation for eGFR. The effect on recommended clinical care across race and ethnicity groups is unknown.

METHODS

We analyzed nationally representative cross-sectional questionnaires and medical examinations from 44,360 participants collected between 2001 and 2018 by the National Health and Nutrition Examination Survey. We quantified the number and proportion of Black, White, Hispanic, and Asian/Other adults with guideline-recommended changes in care.

RESULTS

The new equation, if applied nationally, could assign new CKD diagnoses to 434,000 (95% confidence interval [CI], 350,000 to 517,000) Black adults, reclassify 584,000 (95% CI, 508,000 to 667,000) to more advanced stages of CKD, restrict kidney donation eligibility for 246,000 (95% CI, 189,000 to 303,000), expand nephrologist referrals for 41,800 (95% CI, 19,800 to 63,800), and reduce medication dosing for 222,000 (95% CI, 169,000 to 275,000). Among non-Black adults, these changes may undo CKD diagnoses for 5.51 million (95% CI, 4.86 million to 6.16 million), reclassify 4.59 million (95% CI, 4.28 million to 4.92 million) to less advanced stages of CKD, expand kidney donation eligibility for 3.96 million (95% CI, 3.46 million to 4.46 million), reverse nephrologist referral for 75,800 (95% CI, 35,400 to 116,000), and reverse medication dose reductions for 1.47 million (95% CI, 1.22 million to 1.73 million). The racial and ethnic mix of the populations used to develop eGFR equations has a substantial effect on potential care changes.

CONCLUSION

The newly recommended 2021 CKD-EPI creatinine-based eGFR equation may result in substantial changes to recommended care for US patients of all racial and ethnic groups.

Continuing a search for a diagnosis: the impact of adolescence and family dynamics

The "diagnostic odyssey" describes the process those with undiagnosed conditions undergo to identify a diagnosis. Throughout this process, families of children with undiagnosed conditions have multiple opportunities to decide whether to continue or stop their search for a diagnosis and accept the lack of a diagnostic label. Previous studies identified factors motivating a family to begin searching, but there is limited information about the decision-making process in a prolonged search and how the affected child impacts a family's decision. This study aimed to understand how families of children with undiagnosed diseases decide whether to continue to pursue a diagnosis after standard clinical testing has failed. Parents who applied to the Undiagnosed Disease Network (UDN) at the National Institutes of Health (NIH) were recruited to participate in semi-structured interviews. The 2015 Supportive Care Needs model by Pelenstov, which defines critical needs in families with rare/undiagnosed diseases, provided a framework for interview guide development and transcript analysis (Pelentsov et al in Disabil Health J 8(4):475-491, 2015. https://doi.org/10.1016/J.DHJO.2015.03.009 ). A deductive, iterative coding approach was used to identify common unifying themes. Fourteen parents from 13 families were interviewed. The average child's age was 11 years (range 3-18) and an average 63% of their life had been spent searching for a diagnosis. Our analysis found that alignment or misalignment of parent and child needs impact the trajectory of the diagnostic search. When needs and desires align, reevaluation of a decision to pursue a diagnosis is limited. However, when there is conflict between parent and child desires, there is reevaluation, and often a pause, in the search. This tension is exacerbated when children are adolescents and attempting to balance their dependence on parents for medical care with a natural desire for independence. Our results provide novel insights into the roles of adolescents in the diagnostic odyssey. The tension between desired and realistic developmental outcomes for parents and adolescents impacts if, and how, the search for a diagnosis progresses.

Acute respiratory distress syndrome after SARS-CoV-2 infection on young adult population: International observational federated study based on electronic health records through the 4CE consortium

PURPOSE

In young adults (18 to 49 years old), investigation of the acute respiratory distress syndrome (ARDS) after severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection has been limited. We evaluated the risk factors and outcomes of ARDS following infection with SARS-CoV-2 in a young adult population.

METHODS

A retrospective cohort study was conducted between January 1st, 2020 and February 28th, 2021 using patient-level electronic health records (EHR), across 241 United States hospitals and 43 European hospitals participating in the Consortium for Clinical Characterization of COVID-19 by EHR (4CE). To identify the risk factors associated with ARDS, we compared young patients with and without ARDS through a federated analysis. We further compared the outcomes between young and old patients with ARDS.

RESULTS

Among the 75,377 hospitalized patients with positive SARS-CoV-2 PCR, 1001 young adults presented with ARDS (7.8% of young hospitalized adults). Their mortality rate at 90 days was 16.2% and they presented with a similar complication rate for infection than older adults with ARDS. Peptic ulcer disease, paralysis, obesity, congestive heart failure, valvular disease, diabetes, chronic pulmonary disease and liver disease were associated with a higher risk of ARDS. We described a high prevalence of obesity (53%), hypertension (38%- although not significantly associated with ARDS), and diabetes (32%).

CONCLUSION

Trough an innovative method, a large international cohort study of young adults developing ARDS after SARS-CoV-2 infection has been gather. It demonstrated the poor outcomes of this population and associated risk factor.

Long-term kidney function recovery and mortality after COVID-19-associated acute kidney injury: An international multi-centre observational cohort study

Background

While acute kidney injury (AKI) is a common complication in COVID-19, data on post-AKI kidney function recovery and the clinical factors associated with poor kidney function recovery is lacking.

Methods

A retrospective multi-centre observational cohort study comprising 12,891 hospitalized patients aged 18 years or older with a diagnosis of SARS-CoV-2 infection confirmed by polymerase chain reaction from 1 January 2020 to 10 September 2020, and with at least one serum creatinine value 1-365 days prior to admission. Mortality and serum creatinine values were obtained up to 10 September 2021.

Findings

Advanced age (HR 2.77, 95%CI 2.53-3.04, p < 0.0001), severe COVID-19 (HR 2.91, 95%CI 2.03-4.17, p < 0.0001), severe AKI (KDIGO stage 3: HR 4.22, 95%CI 3.55-5.00, p < 0.0001), and ischemic heart disease (HR 1.26, 95%CI 1.14-1.39, p < 0.0001) were associated with worse mortality outcomes. AKI severity (KDIGO stage 3: HR 0.41, 95%CI 0.37-0.46, p < 0.0001) was associated with worse kidney function recovery, whereas remdesivir use (HR 1.34, 95%CI 1.17-1.54, p < 0.0001) was associated with better kidney function recovery. In a subset of patients without chronic kidney disease, advanced age (HR 1.38, 95%CI 1.20-1.58, p < 0.0001), male sex (HR 1.67, 95%CI 1.45-1.93, p < 0.0001), severe AKI (KDIGO stage 3: HR 11.68, 95%CI 9.80-13.91, p < 0.0001), and hypertension (HR 1.22, 95%CI 1.10-1.36, p = 0.0002) were associated with post-AKI kidney function impairment. Furthermore, patients with COVID-19-associated AKI had significant and persistent elevations of baseline serum creatinine 125% or more at 180 days (RR 1.49, 95%CI 1.32-1.67) and 365 days (RR 1.54, 95%CI 1.21-1.96) compared to COVID-19 patients with no AKI.

Interpretation

COVID-19-associated AKI was associated with higher mortality, and severe COVID-19-associated AKI was associated with worse long-term post-AKI kidney function recovery.

Funding

Authors are supported by various funders, with full details stated in the acknowledgement section.

SurvMaximin: Robust federated approach to transporting survival risk prediction models

OBJECTIVE

For multi-center heterogeneous Real-World Data (RWD) with time-to-event outcomes and high-dimensional features, we propose the SurvMaximin algorithm to estimate Cox model feature coefficients for a target population by borrowing summary information from a set of health care centers without sharing patient-level information.

MATERIALS AND METHODS

For each of the centers from which we want to borrow information to improve the prediction performance for the target population, a penalized Cox model is fitted to estimate feature coefficients for the center. Using estimated feature coefficients and the covariance matrix of the target population, we then obtain a SurvMaximin estimated set of feature coefficients for the target population. The target population can be an entire cohort comprised of all centers, corresponding to federated learning, or a single center, corresponding to transfer learning.

RESULTS

Simulation studies and a real-world international electronic health records application study, with 15 participating health care centers across three countries (France, Germany, and the U.S.), show that the proposed SurvMaximin algorithm achieves comparable or higher accuracy compared with the estimator using only the information of the target site and other existing methods. The SurvMaximin estimator is robust to variations in sample sizes and estimated feature coefficients between centers, which amounts to significantly improved estimates for target sites with fewer observations.

CONCLUSIONS

The SurvMaximin method is well suited for both federated and transfer learning in the high-dimensional survival analysis setting. SurvMaximin only requires a one-time summary information exchange from participating centers. Estimated regression vectors can be very heterogeneous. SurvMaximin provides robust Cox feature coefficient estimates without outcome information in the target population and is privacy-preserving.

Association of Race and Socioeconomic Disadvantage With Missed Telemedicine Visits for Pediatric Patients During the COVID-19 Pandemic

This comparative effectiveness study examines whether race and ethnicity and socioeconomic disadvantage are factors associated with missing telemedicine visits during the COVID-19 pandemic among pediatric patients in Massachusetts.

Analytics to monitor local impact of the Protecting Access to Medicare Act's imaging clinical decision support requirements

OBJECTIVE

This study aimed is to: (1) extend the Integrating the Biology and the Bedside (i2b2) data and application models to include medical imaging appropriate use criteria, enabling it to serve as a platform to monitor local impact of the Protecting Access to Medicare Act's (PAMA) imaging clinical decision support (CDS) requirements, and (2) validate the i2b2 extension using data from the Medicare Imaging Demonstration (MID) CDS implementation.

MATERIALS AND METHODS

This study provided a reference implementation and assessed its validity and reliability using data from the MID, the federal government's predecessor to PAMA's imaging CDS program. The Star Schema was extended to describe the interactions of imaging ordering providers with the CDS. New ontologies were added to enable mapping medical imaging appropriateness data to i2b2 schema. z-Ratio for testing the significance of the difference between 2 independent proportions was utilized.

RESULTS

The reference implementation used 26 327 orders for imaging examinations which were persisted to the modified i2b2 schema. As an illustration of the analytical capabilities of the Web Client, we report that 331/1192 or 28.1% of imaging orders were deemed appropriate by the CDS system at the end of the intervention period (September 2013), an increase from 162/1223 or 13.2% for the first month of the baseline period, December 2011 (P = .0212), consistent with previous studies.

CONCLUSIONS

The i2b2 platform can be extended to monitor local impact of PAMA's appropriateness of imaging ordering CDS requirements.

Trends in Medical Management of Moderately to Severely Active Ulcerative Colitis: A Nationwide Retrospective Analysis

BACKGROUND

With an increasing number of therapeutic options available for the management of ulcerative colitis (UC), the variability in treatment and prescribing patterns is not well known. While recent guidelines have provided updates on how these therapeutic options should be used, patterns of long-term use of these drugs over the past 2 decades remain unclear.

METHODS

We analyzed a retrospective, nationwide cohort of more than 1.7 million prescriptions for trends in prescribing behaviors and to evaluate practices suggested in guidelines relating to ordering biologics, step-up therapy, and combination therapy. The primary outcome was 30-day steroid-free remission and secondary outcomes included hospitalization, cost, and additional steroid usage. A pipeline was created to identify cohorts of patients under active UC medical management grouped by prescribing strategies to evaluate comparative outcomes between strategies. Cox proportional hazards and multivariate regression models were utilized to assess postexposure outcomes and adjust for confounders.

RESULTS

Among 6 major drug categories, we noted major baseline differences in patient characteristics at first exposure corresponding to disease activity. We noted earlier use of biologics in patient trajectories (762 days earlier relative to UC diagnosis, 2018 vs 2008; P < .001) and greater overall use of biologics over time (2.53× more in 2018 vs 2008; P < .00001) . Among biologic-naive patients, adalimumab was associated with slightly lower rates of remission compared with infliximab or vedolizumab (odds ratio, 0.92; P < .005). Comparisons of patients with early biologic initiation to patients who transitioned to biologics from 5-aminosalicylic acid suggest lower steroid consumption for early biologic initiation (-761 mg prednisone; P < .001). Combination thiopurine-biologic therapy was associated with higher odds of remission compared with biologic monotherapy (odds ratio, 1.36; P = .01).

CONCLUSIONS

As biologic drugs have become increasingly available for UC management, they have increasingly been used at earlier stages of disease management. Large-scale analyses of prescribing behaviors provide evidence supporting early use of biologics compared with step-up therapy and use of thiopurine and biologic combination therapy.

International electronic health record-derived post-acute sequelae profiles of COVID-19 patients

The risk profiles of post-acute sequelae of COVID-19 (PASC) have not been well characterized in multi-national settings with appropriate controls. We leveraged electronic health record (EHR) data from 277 international hospitals representing 414,602 patients with COVID-19, 2.3 million control patients without COVID-19 in the inpatient and outpatient settings, and over 221 million diagnosis codes to systematically identify new-onset conditions enriched among patients with COVID-19 during the post-acute period. Compared to inpatient controls, inpatient COVID-19 cases were at significant risk for angina pectoris (RR 1.30, 95% CI 1.09–1.55), heart failure (RR 1.22, 95% CI 1.10–1.35), cognitive dysfunctions (RR 1.18, 95% CI 1.07–1.31), and fatigue (RR 1.18, 95% CI 1.07–1.30). Relative to outpatient controls, outpatient COVID-19 cases were at risk for pulmonary embolism (RR 2.10, 95% CI 1.58–2.76), venous embolism (RR 1.34, 95% CI 1.17–1.54), atrial fibrillation (RR 1.30, 95% CI 1.13–1.50), type 2 diabetes (RR 1.26, 95% CI 1.16–1.36) and vitamin D deficiency (RR 1.19, 95% CI 1.09–1.30). Outpatient COVID-19 cases were also at risk for loss of smell and taste (RR 2.42, 95% CI 1.90–3.06), inflammatory neuropathy (RR 1.66, 95% CI 1.21–2.27), and cognitive dysfunction (RR 1.18, 95% CI 1.04–1.33). The incidence of post-acute cardiovascular and pulmonary conditions decreased across time among inpatient cases while the incidence of cardiovascular, digestive, and metabolic conditions increased among outpatient cases. Our study, based on a federated international network, systematically identified robust conditions associated with PASC compared to control groups, underscoring the multifaceted cardiovascular and neurological phenotype profiles of PASC.

International comparisons of laboratory values from the 4CE collaborative to predict COVID-19 mortality

Given the growing number of prediction algorithms developed to predict COVID-19 mortality, we evaluated the transportability of a mortality prediction algorithm using a multi-national network of healthcare systems. We predicted COVID-19 mortality using baseline commonly measured laboratory values and standard demographic and clinical covariates across healthcare systems, countries, and continents. Specifically, we trained a Cox regression model with nine measured laboratory test values, standard demographics at admission, and comorbidity burden pre-admission. These models were compared at site, country, and continent level. Of the 39,969 hospitalized patients with COVID-19 (68.6% male), 5717 (14.3%) died. In the Cox model, age, albumin, AST, creatine, CRP, and white blood cell count are most predictive of mortality. The baseline covariates are more predictive of mortality during the early days of COVID-19 hospitalization. Models trained at healthcare systems with larger cohort size largely retain good transportability performance when porting to different sites. The combination of routine laboratory test values at admission along with basic demographic features can predict mortality in patients hospitalized with COVID-19. Importantly, this potentially deployable model differs from prior work by demonstrating not only consistent performance but also reliable transportability across healthcare systems in the US and Europe, highlighting the generalizability of this model and the overall approach.

Large-scale real-world data analysis identifies comorbidity patterns in schizophrenia

Schizophrenia affects >3.2 million people in the USA. However, its comorbidity patterns have not been systematically characterized in real-world populations. To address this gap, we conducted an observational study using a cohort of 86 million patients in a nationwide health insurance dataset. We identified participants with schizophrenia and those without schizophrenia matched by age, sex, and the first three digits of zip code. For each phenotype encoded in phecodes, we compared their prevalence in schizophrenia patients and the matched non-schizophrenic participants, and we performed subgroup analyses stratified by age and sex. Results show that anxiety, posttraumatic stress disorder, and substance abuse commonly occur in adolescents and young adults prior to schizophrenia diagnoses. Patients aged 60 and above are at higher risks of developing delirium, alcoholism, dementia, pelvic fracture, and osteomyelitis than their matched controls. Type 2 diabetes, sleep apnea, and eating disorders were more prevalent in women prior to schizophrenia diagnosis, whereas acute renal failure, rhabdomyolysis, and developmental delays were found at higher rates in men. Anxiety and obesity are more commonly seen in patients with schizoaffective disorders compared to patients with other types of schizophrenia. Leveraging a large-scale insurance claims dataset, this study identified less-known comorbidity patterns of schizophrenia and confirmed known ones. These comorbidity profiles can guide clinicians and researchers to take heed of early signs of co-occurring diseases.

Identifying the Common Genetic Basis of Antidepressant Response

Background

Antidepressants are a first-line treatment for depression. However, only a third of individuals experience remission after the first treatment. Common genetic variation, in part, likely regulates antidepressant response, yet the success of previous genome-wide association studies has been limited by sample size. This study performs the largest genetic analysis of prospectively assessed antidepressant response in major depressive disorder to gain insight into the underlying biology and enable out-of-sample prediction.

Methods

Genome-wide analysis of remission (n remit = 1852, n nonremit = 3299) and percentage improvement (n = 5218) was performed. Single nucleotide polymorphism-based heritability was estimated using genome-wide complex trait analysis. Genetic covariance with eight mental health phenotypes was estimated using polygenic scores/AVENGEME. Out-of-sample prediction of antidepressant response polygenic scores was assessed. Gene-level association analysis was performed using MAGMA and transcriptome-wide association study. Tissue, pathway, and drug binding enrichment were estimated using MAGMA.

Results

Neither genome-wide association study identified genome-wide significant associations. Single nucleotide polymorphism-based heritability was significantly different from zero for remission (h 2 = 0.132, SE = 0.056) but not for percentage improvement (h 2 = -0.018, SE = 0.032). Better antidepressant response was negatively associated with genetic risk for schizophrenia and positively associated with genetic propensity for educational attainment. Leave-one-out validation of antidepressant response polygenic scores demonstrated significant evidence of out-of-sample prediction, though results varied in external cohorts. Gene-based analyses identified ETV4 and DHX8 as significantly associated with antidepressant response.

Conclusions

This study demonstrates that antidepressant response is influenced by common genetic variation, has a genetic overlap schizophrenia and educational attainment, and provides a useful resource for future research. Larger sample sizes are required to attain the potential of genetics for understanding and predicting antidepressant response.

Effectiveness of a third dose of the BNT162b2 mRNA COVID-19 vaccine for preventing severe outcomes in Israel: an observational study

BACKGROUND

Many countries are experiencing a resurgence of COVID-19, driven predominantly by the delta (B.1.617.2) variant of SARS-CoV-2. In response, these countries are considering the administration of a third dose of mRNA COVID-19 vaccine as a booster dose to address potential waning immunity over time and reduced effectiveness against the delta variant. We aimed to use the data repositories of Israel's largest health-care organisation to evaluate the effectiveness of a third dose of the BNT162b2 mRNA vaccine for preventing severe COVID-19 outcomes.

METHODS

Using data from Clalit Health Services, which provides mandatory health-care coverage for over half of the Israeli population, individuals receiving a third vaccine dose between July 30, 2020, and Sept 23, 2021, were matched (1:1) to demographically and clinically similar controls who did not receive a third dose. Eligible participants had received the second vaccine dose at least 5 months before the recruitment date, had no previous documented SARS-CoV-2 infection, and had no contact with the health-care system in the 3 days before recruitment. Individuals who are health-care workers, live in long-term care facilities, or are medically confined to their homes were excluded. Primary outcomes were COVID-19-related admission to hospital, severe disease, and COVID-19-related death. The third dose effectiveness for each outcome was estimated as 1 - risk ratio using the Kaplan-Meier estimator.

FINDINGS

1 158 269 individuals were eligible to be included in the third dose group. Following matching, the third dose and control groups each included 728 321 individuals. Participants had a median age of 52 years (IQR 37-68) and 51% were female. The median follow-up time was 13 days (IQR 6-21) in both groups. Vaccine effectiveness evaluated at least 7 days after receipt of the third dose, compared with receiving only two doses at least 5 months ago, was estimated to be 93% (231 events for two doses vs 29 events for three doses; 95% CI 88-97) for admission to hospital, 92% (157 vs 17 events; 82-97) for severe disease, and 81% (44 vs seven events; 59-97) for COVID-19-related death.

INTERPRETATION

Our findings suggest that a third dose of the BNT162b2 mRNA vaccine is effective in protecting individuals against severe COVID-19-related outcomes, compared with receiving only two doses at least 5 months ago.

FUNDING

The Ivan and Francesca Berkowitz Family Living Laboratory Collaboration at Harvard Medical School and Clalit Research Institute.

Integrative multiomics-histopathology analysis for breast cancer classification

Histopathologic evaluation of biopsy slides is a critical step in diagnosing and subtyping breast cancers. However, the connections between histology and multi-omics status have never been systematically explored or interpreted. We developed weakly supervised deep learning models over hematoxylin-and-eosin-stained slides to examine the relations between visual morphological signal, clinical subtyping, gene expression, and mutation status in breast cancer. We first designed fully automated models for tumor detection and pathology subtype classification, with the results validated in independent cohorts (area under the receiver operating characteristic curve ≥ 0.950). Using only visual information, our models achieved strong predictive performance in estrogen/progesterone/HER2 receptor status, PAM50 status, and TP53 mutation status. We demonstrated that these models learned lymphocyte-specific morphological signals to identify estrogen receptor status. Examination of the PAM50 cohort revealed a subset of PAM50 genes whose expression reflects cancer morphology. This work demonstrates the utility of deep learning-based image models in both clinical and research regimes, through its ability to uncover connections between visual morphology and genetic statuses.

ATLAS: an automated association test using probabilistically linked health records with application to genetic studies

OBJECTIVE

Large amounts of health data are becoming available for biomedical research. Synthesizing information across databases may capture more comprehensive pictures of patient health and enable novel research studies. When no gold standard mappings between patient records are available, researchers may probabilistically link records from separate databases and analyze the linked data. However, previous linked data inference methods are constrained to certain linkage settings and exhibit low power. Here, we present ATLAS, an automated, flexible, and robust association testing algorithm for probabilistically linked data.

MATERIALS AND METHODS

Missing variables are imputed at various thresholds using a weighted average method that propagates uncertainty from probabilistic linkage. Next, estimated effect sizes are obtained using a generalized linear model. ATLAS then conducts the threshold combination test by optimally combining P values obtained from data imputed at varying thresholds using Fisher's method and perturbation resampling.

RESULTS

In simulations, ATLAS controls for type I error and exhibits high power compared to previous methods. In a real-world genetic association study, meta-analysis of ATLAS-enabled analyses on a linked cohort with analyses using an existing cohort yielded additional significant associations between rheumatoid arthritis genetic risk score and laboratory biomarkers.

DISCUSSION

Weighted average imputation weathers false matches and increases contribution of true matches to mitigate linkage error-induced bias. The threshold combination test avoids arbitrarily choosing a threshold to rule a match, thus automating linked data-enabled analyses and preserving power.

CONCLUSION

ATLAS promises to enable novel and powerful research studies using linked data to capitalize on all available data sources.

Multinational characterization of neurological phenotypes in patients hospitalized with COVID-19

Neurological complications worsen outcomes in COVID-19. To define the prevalence of neurological conditions among hospitalized patients with a positive SARS-CoV-2 reverse transcription polymerase chain reaction test in geographically diverse multinational populations during early pandemic, we used electronic health records (EHR) from 338 participating hospitals across 6 countries and 3 continents (January-September 2020) for a cross-sectional analysis. We assessed the frequency of International Classification of Disease code of neurological conditions by countries, healthcare systems, time before and after admission for COVID-19 and COVID-19 severity. Among 35,177 hospitalized patients with SARS-CoV-2 infection, there was an increase in the proportion with disorders of consciousness (5.8%, 95% confidence interval [CI] 3.7-7.8%, pFDR < 0.001) and unspecified disorders of the brain (8.1%, 5.7-10.5%, pFDR < 0.001) when compared to the pre-admission proportion. During hospitalization, the relative risk of disorders of consciousness (22%, 19-25%), cerebrovascular diseases (24%, 13-35%), nontraumatic intracranial hemorrhage (34%, 20-50%), encephalitis and/or myelitis (37%, 17-60%) and myopathy (72%, 67-77%) were higher for patients with severe COVID-19 when compared to those who never experienced severe COVID-19. Leveraging a multinational network to capture standardized EHR data, we highlighted the increased prevalence of central and peripheral neurological phenotypes in patients hospitalized with COVID-19, particularly among those with severe disease.

International Changes in COVID-19 Clinical Trajectories Across 315 Hospitals and 6 Countries: Retrospective Cohort Study

Background

Many countries have experienced 2 predominant waves of COVID-19–related hospitalizations. Comparing the clinical trajectories of patients hospitalized in separate waves of the pandemic enables further understanding of the evolving epidemiology, pathophysiology, and health care dynamics of the COVID-19 pandemic.

Objective

In this retrospective cohort study, we analyzed electronic health record (EHR) data from patients with SARS-CoV-2 infections hospitalized in participating health care systems representing 315 hospitals across 6 countries. We compared hospitalization rates, severe COVID-19 risk, and mean laboratory values between patients hospitalized during the first and second waves of the pandemic.

Methods

Using a federated approach, each participating health care system extracted patient-level clinical data on their first and second wave cohorts and submitted aggregated data to the central site. Data quality control steps were adopted at the central site to correct for implausible values and harmonize units. Statistical analyses were performed by computing individual health care system effect sizes and synthesizing these using random effect meta-analyses to account for heterogeneity. We focused the laboratory analysis on C-reactive protein (CRP), ferritin, fibrinogen, procalcitonin, D-dimer, and creatinine based on their reported associations with severe COVID-19.

Results

Data were available for 79,613 patients, of which 32,467 were hospitalized in the first wave and 47,146 in the second wave. The prevalence of male patients and patients aged 50 to 69 years decreased significantly between the first and second waves. Patients hospitalized in the second wave had a 9.9% reduction in the risk of severe COVID-19 compared to patients hospitalized in the first wave (95% CI 8.5%-11.3%). Demographic subgroup analyses indicated that patients aged 26 to 49 years and 50 to 69 years; male and female patients; and black patients had significantly lower risk for severe disease in the second wave than in the first wave. At admission, the mean values of CRP were significantly lower in the second wave than in the first wave. On the seventh hospital day, the mean values of CRP, ferritin, fibrinogen, and procalcitonin were significantly lower in the second wave than in the first wave. In general, countries exhibited variable changes in laboratory testing rates from the first to the second wave. At admission, there was a significantly higher testing rate for D-dimer in France, Germany, and Spain.

Conclusions

Patients hospitalized in the second wave were at significantly lower risk for severe COVID-19. This corresponded to mean laboratory values in the second wave that were more likely to be in typical physiological ranges on the seventh hospital day compared to the first wave. Our federated approach demonstrated the feasibility and power of harmonizing heterogeneous EHR data from multiple international health care systems to rapidly conduct large-scale studies to characterize how COVID-19 clinical trajectories evolve.

Real-world data analyses unveiled the immune-related adverse effects of immune checkpoint inhibitors across cancer types

Immune checkpoint inhibitors have demonstrated significant survival benefits in treating many types of cancers. However, their immune-related adverse events (irAEs) have not been systematically evaluated across cancer types in large-scale real-world populations. To address this gap, we conducted real-world data analyses using nationwide insurance claims data with 85.97 million enrollees across 8 years. We identified a significantly increased risk of developing irAEs among patients receiving immunotherapy agents in all seven cancer types commonly treated with immune checkpoint inhibitors. By six months after treatment initialization, those receiving immunotherapy were 1.50-4.00 times (95% CI, lower bound from 1.15 to 2.16, upper bound from 1.69 to 20.36) more likely to develop irAEs in the first 6 months of treatment, compared to matched chemotherapy or targeted therapy groups, with a total of 92,858 patients. The risk of developing irAEs among patients using nivolumab is higher compared to those using pembrolizumab. These results confirmed the need for clinicians to assess irAEs among cancer patients undergoing immunotherapy as part of management. Our methods are extensible to characterizing the effectiveness and adverse effects of novel treatments in large populations in an efficient and economical fashion.

Medication Use in the Management of Comorbidities Among Individuals With Autism Spectrum Disorder From a Large Nationwide Insurance Database

IMPORTANCE

Although there is no pharmacological treatment for autism spectrum disorder (ASD) itself, behavioral and pharmacological therapies have been used to address its symptoms and common comorbidities. A better understanding of the medications used to manage comorbid conditions in this growing population is critical; however, most previous efforts have been limited in size, duration, and lack of broad representation.

OBJECTIVE

To use a nationally representative database to uncover trends in the prevalence of co-occurring conditions and medication use in the management of symptoms and comorbidities over time among US individuals with ASD.

DESIGN, SETTING, AND PARTICIPANTS

This retrospective, population-based cohort study mined a nationwide, managed health plan claims database containing more than 86 million unique members. Data from January 1, 2014, to December 31, 2019, were used to analyze prescription frequency and diagnoses of comorbidities. A total of 26 722 individuals with ASD who had been prescribed at least 1 of 24 medications most commonly prescribed to treat ASD symptoms or comorbidities during the 6-year study period were included in the analysis.

EXPOSURES

Diagnosis codes for ASD based on International Classification of Diseases, Ninth Revision, and International Statistical Classification of Diseases and Related Health Problems, Tenth Revision.

MAIN OUTCOMES AND MEASURES

Quantitative estimates of prescription frequency for the 24 most commonly prescribed medications among the study cohort and the most common comorbidities associated with each medication in this population.

RESULTS

Among the 26 722 individuals with ASD included in the analysis (77.7% male; mean [SD] age, 14.45 [9.40] years), polypharmacy was common, ranging from 28.6% to 31.5%. Individuals' prescription regimens changed frequently within medication classes, rather than between classes. The prescription frequency of a specific medication varied considerably, depending on the coexisting diagnosis of a given comorbidity. Of the 24 medications assessed, 15 were associated with at least a 15% prevalence of a mood disorder, and 11 were associated with at least a 15% prevalence of attention-deficit/hyperactivity disorder. For patients taking antipsychotics, the 2 most common comorbidities were combined type attention-deficit/hyperactivity disorder (11.6%-17.8%) and anxiety disorder (13.1%-30.1%).

CONCLUSIONS AND RELEVANCE

This study demonstrated considerable variability and transiency in the use of prescription medications by US clinicians to manage symptoms and comorbidities associated with ASD. These findings support the importance of early and ongoing surveillance of patients with ASD and co-occurring conditions and offer clinicians insight on the targeted therapies most commonly used to manage co-occurring conditions. Future research and policy efforts are critical to assess the extent to which pharmacological management of comorbidities affects quality of life and functioning in patients with ASD while continuing to optimize clinical guidelines, to ensure effective care for this growing population.

Finding a new balance between a genetics-first or phenotype-first approach to the study of disease

Successes in neuroscience using a genetics-first approach to characterizing disorders such as autism have eclipsed the scientific and clinical value of a comprehensive phenotype-first-clinical or molecular-approach. Recent high-throughput phenotyping techniques using machine learning, electronic medical records, and even administrative databases show the value of a synthesis between the two approaches.

Validation of an internationally derived patient severity phenotype to support COVID-19 analytics from electronic health record data

OBJECTIVE

The Consortium for Clinical Characterization of COVID-19 by EHR (4CE) is an international collaboration addressing coronavirus disease 2019 (COVID-19) with federated analyses of electronic health record (EHR) data. We sought to develop and validate a computable phenotype for COVID-19 severity.

MATERIALS AND METHODS

Twelve 4CE sites participated. First, we developed an EHR-based severity phenotype consisting of 6 code classes, and we validated it on patient hospitalization data from the 12 4CE clinical sites against the outcomes of intensive care unit (ICU) admission and/or death. We also piloted an alternative machine learning approach and compared selected predictors of severity with the 4CE phenotype at 1 site.

RESULTS

The full 4CE severity phenotype had pooled sensitivity of 0.73 and specificity 0.83 for the combined outcome of ICU admission and/or death. The sensitivity of individual code categories for acuity had high variability-up to 0.65 across sites. At one pilot site, the expert-derived phenotype had mean area under the curve of 0.903 (95% confidence interval, 0.886-0.921), compared with an area under the curve of 0.956 (95% confidence interval, 0.952-0.959) for the machine learning approach. Billing codes were poor proxies of ICU admission, with as low as 49% precision and recall compared with chart review.

DISCUSSION

We developed a severity phenotype using 6 code classes that proved resilient to coding variability across international institutions. In contrast, machine learning approaches may overfit hospital-specific orders. Manual chart review revealed discrepancies even in the gold-standard outcomes, possibly owing to heterogeneous pandemic conditions.

CONCLUSIONS

We developed an EHR-based severity phenotype for COVID-19 in hospitalized patients and validated it at 12 international sites.

Patient-led data sharing for clinical bioinformatics research: USCDI and beyond

The 21st Century Cures Act, passed in 2016, and the Final Rules it called for create a roadmap for enabling patient access to their electronic health information. The set of data to be made available, as determined by the Office of the National Coordinator for Health IT through the US Core Data for Interoperability expansion process, will impact the value creation of this improved data liquidity. In this commentary, we look at the potential for significant value creation from USCDI in the context of clinical bioinformatics research and advocate for the research community's involvement in the USCDI process to propel this value creation forward. We also describe 1 mechanism-using existing required APIs for full data export capabilities-that could pragmatically enable this value creation at minimal additional technical lift beyond the current regulatory requirements.

Illustrating potential effects of alternate control populations on real-world evidence-based statistical analyses

Objective

Case-control study designs are commonly used in retrospective analyses of real-world evidence (RWE). Due to the increasingly wide availability of RWE, it can be difficult to determine whether findings are robust or the result of testing multiple hypotheses.

Materials and Methods

We investigate the potential effects of modifying cohort definitions in a case-control association study between depression and type 2 diabetes mellitus. We used a large (>75 million individuals) de-identified administrative claims database to observe the effects of minor changes to the requirements of glucose and hemoglobin A1c tests in the control group.

Results

We found that small permutations to the criteria used to define the control population result in significant shifts in both the demographic structure of the identified cohort as well as the odds ratio of association. These differences remain present when testing against age- and sex-matched controls.

Discussion

Analyses of RWE need to be carefully designed to avoid issues of multiple testing. Minor changes to control cohorts can lead to significantly different results and have the potential to alter even prospective studies through selection bias.

Conclusion

We believe this work offers strong support for the need for robust guidelines, best practices, and regulations around the use of observational RWE for clinical or regulatory decision-making.

Commonalities across computational workflows for uncovering explanatory variants in undiagnosed cases

PURPOSE

Genomic sequencing has become an increasingly powerful and relevant tool to be leveraged for the discovery of genetic aberrations underlying rare, Mendelian conditions. Although the computational tools incorporated into diagnostic workflows for this task are continually evolving and improving, we nevertheless sought to investigate commonalities across sequencing processing workflows to reveal consensus and standard practice tools and highlight exploratory analyses where technical and theoretical method improvements would be most impactful.

METHODS

We collected details regarding the computational approaches used by a genetic testing laboratory and 11 clinical research sites in the United States participating in the Undiagnosed Diseases Network via meetings with bioinformaticians, online survey forms, and analyses of internal protocols.

RESULTS

We found that tools for processing genomic sequencing data can be grouped into four distinct categories. Whereas well-established practices exist for initial variant calling and quality control steps, there is substantial divergence across sites in later stages for variant prioritization and multimodal data integration, demonstrating a diversity of approaches for solving the most mysterious undiagnosed cases.

CONCLUSION

The largest differences across diagnostic workflows suggest that advances in structural variant detection, noncoding variant interpretation, and integration of additional biomedical data may be especially promising for solving chronically undiagnosed cases.

International Analysis of Electronic Health Records of Children and Youth Hospitalized With COVID-19 Infection in 6 Countries

IMPORTANCE

Additional sources of pediatric epidemiological and clinical data are needed to efficiently study COVID-19 in children and youth and inform infection prevention and clinical treatment of pediatric patients.

OBJECTIVE

To describe international hospitalization trends and key epidemiological and clinical features of children and youth with COVID-19.

DESIGN, SETTING, AND PARTICIPANTS

This retrospective cohort study included pediatric patients hospitalized between February 2 and October 10, 2020. Patient-level electronic health record (EHR) data were collected across 27 hospitals in France, Germany, Spain, Singapore, the UK, and the US. Patients younger than 21 years who tested positive for COVID-19 and were hospitalized at an institution participating in the Consortium for Clinical Characterization of COVID-19 by EHR were included in the study.

MAIN OUTCOMES AND MEASURES

Patient characteristics, clinical features, and medication use.

RESULTS

There were 347 males (52%; 95% CI, 48.5-55.3) and 324 females (48%; 95% CI, 44.4-51.3) in this study's cohort. There was a bimodal age distribution, with the greatest proportion of patients in the 0- to 2-year (199 patients [30%]) and 12- to 17-year (170 patients [25%]) age range. Trends in hospitalizations for 671 children and youth found discrete surges with variable timing across 6 countries. Data from this cohort mirrored national-level pediatric hospitalization trends for most countries with available data, with peaks in hospitalizations during the initial spring surge occurring within 23 days in the national-level and 4CE data. A total of 27 364 laboratory values for 16 laboratory tests were analyzed, with mean values indicating elevations in markers of inflammation (C-reactive protein, 83 mg/L; 95% CI, 53-112 mg/L; ferritin, 417 ng/mL; 95% CI, 228-607 ng/mL; and procalcitonin, 1.45 ng/mL; 95% CI, 0.13-2.77 ng/mL). Abnormalities in coagulation were also evident (D-dimer, 0.78 ug/mL; 95% CI, 0.35-1.21 ug/mL; and fibrinogen, 477 mg/dL; 95% CI, 385-569 mg/dL). Cardiac troponin, when checked (n = 59), was elevated (0.032 ng/mL; 95% CI, 0.000-0.080 ng/mL). Common complications included cardiac arrhythmias (15.0%; 95% CI, 8.1%-21.7%), viral pneumonia (13.3%; 95% CI, 6.5%-20.1%), and respiratory failure (10.5%; 95% CI, 5.8%-15.3%). Few children were treated with COVID-19-directed medications.

CONCLUSIONS AND RELEVANCE

This study of EHRs of children and youth hospitalized for COVID-19 in 6 countries demonstrated variability in hospitalization trends across countries and identified common complications and laboratory abnormalities in children and youth with COVID-19 infection. Large-scale informatics-based approaches to integrate and analyze data across health care systems complement methods of disease surveillance and advance understanding of epidemiological and clinical features associated with COVID-19 in children and youth.

Machine learning for patient risk stratification: standing on, or looking over, the shoulders of clinicians?

Machine learning can help clinicians to make individualized patient predictions only if researchers demonstrate models that contribute novel insights, rather than learning the most likely next step in a set of actions a clinician will take. We trained deep learning models using only clinician-initiated, administrative data for 42.9 million admissions using three subsets of data: demographic data only, demographic data and information available at admission, and the previous data plus charges recorded during the first day of admission. Models trained on charges during the first day of admission achieve performance close to published full EMR-based benchmarks for inpatient outcomes: inhospital mortality (0.89 AUC), prolonged length of stay (0.82 AUC), and 30-day readmission rate (0.71 AUC). Similar performance between models trained with only clinician-initiated data and those trained with full EMR data purporting to include information about patient state and physiology should raise concern in the deployment of these models. Furthermore, these models exhibited significant declines in performance when evaluated over only myocardial infarction (MI) patients relative to models trained over MI patients alone, highlighting the importance of physician diagnosis in the prognostic performance of these models. These results provide a benchmark for predictive accuracy trained only on prior clinical actions and indicate that models with similar performance may derive their signal by looking over clinician's shoulders-using clinical behavior as the expression of preexisting intuition and suspicion to generate a prediction. For models to guide clinicians in individual decisions, performance exceeding these benchmarks is necessary.

Development of a Histopathology Informatics Pipeline for Classification and Prediction of Clinical Outcomes in Subtypes of Renal Cell Carcinoma

PURPOSE

Histopathology evaluation is the gold standard for diagnosing clear cell (ccRCC), papillary, and chromophobe renal cell carcinoma (RCC). However, interrater variability has been reported, and the whole-slide histopathology images likely contain underutilized biological signals predictive of genomic profiles.

EXPERIMENTAL DESIGN

To address this knowledge gap, we obtained whole-slide histopathology images and demographic, genomic, and clinical data from The Cancer Genome Atlas, the Clinical Proteomic Tumor Analysis Consortium, and Brigham and Women's Hospital (Boston, MA) to develop computational methods for integrating data analyses. Leveraging these large and diverse datasets, we developed fully automated convolutional neural networks to diagnose renal cancers and connect quantitative pathology patterns with patients' genomic profiles and prognoses.

RESULTS

Our deep convolutional neural networks successfully detected malignancy (AUC in the independent validation cohort: 0.964-0.985), diagnosed RCC histologic subtypes (independent validation AUCs of the best models: 0.953-0.993), and predicted stage I ccRCC patients' survival outcomes (log-rank test P = 0.02). Our machine learning approaches further identified histopathology image features indicative of copy-number alterations (AUC > 0.7 in multiple genes in patients with ccRCC) and tumor mutation burden.

CONCLUSIONS

Our results suggest that convolutional neural networks can extract histologic signals predictive of patients' diagnoses, prognoses, and genomic variations of clinical importance. Our approaches can systematically identify previously unknown relations among diverse data modalities.

What Every Reader Should Know About Studies Using Electronic Health Record Data but May Be Afraid to Ask

Coincident with the tsunami of COVID-19–related publications, there has been a surge of studies using real-world data, including those obtained from the electronic health record (EHR). Unfortunately, several of these high-profile publications were retracted because of concerns regarding the soundness and quality of the studies and the EHR data they purported to analyze. These retractions highlight that although a small community of EHR informatics experts can readily identify strengths and flaws in EHR-derived studies, many medical editorial teams and otherwise sophisticated medical readers lack the framework to fully critically appraise these studies. In addition, conventional statistical analyses cannot overcome the need for an understanding of the opportunities and limitations of EHR-derived studies. We distill here from the broader informatics literature six key considerations that are crucial for appraising studies utilizing EHR data: data completeness, data collection and handling (eg, transformation), data type (ie, codified, textual), robustness of methods against EHR variability (within and across institutions, countries, and time), transparency of data and analytic code, and the multidisciplinary approach. These considerations will inform researchers, clinicians, and other stakeholders as to the recommended best practices in reviewing manuscripts, grants, and other outputs from EHR-data derived studies, and thereby promote and foster rigor, quality, and reliability of this rapidly growing field.

Temporal bias in case-control design: preventing reliable predictions of the future

One of the primary tools that researchers use to predict risk is the case-control study. We identify a flaw, temporal bias, that is specific to and uniquely associated with these studies that occurs when the study period is not representative of the data that clinicians have during the diagnostic process. Temporal bias acts to undermine the validity of predictions by over-emphasizing features close to the outcome of interest. We examine the impact of temporal bias across the medical literature, and highlight examples of exaggerated effect sizes, false-negative predictions, and replication failure. Given the ubiquity and practical advantages of case-control studies, we discuss strategies for estimating the influence of and preventing temporal bias where it exists.

Use of electronic health records to support a public health response to the COVID-19 pandemic in the United States: a perspective from 15 academic medical centers

Our goal is to summarize the collective experience of 15 organizations in dealing with uncoordinated efforts that result in unnecessary delays in understanding, predicting, preparing for, containing, and mitigating the COVID-19 pandemic in the US. Response efforts involve the collection and analysis of data corresponding to healthcare organizations, public health departments, socioeconomic indicators, as well as additional signals collected directly from individuals and communities. We focused on electronic health record (EHR) data, since EHRs can be leveraged and scaled to improve clinical care, research, and to inform public health decision-making. We outline the current challenges in the data ecosystem and the technology infrastructure that are relevant to COVID-19, as witnessed in our 15 institutions. The infrastructure includes registries and clinical data networks to support population-level analyses. We propose a specific set of strategic next steps to increase interoperability, overall organization, and efficiencies.

International Comparisons of Harmonized Laboratory Value Trajectories to Predict Severe COVID-19: Leveraging the 4CE Collaborative Across 342 Hospitals and 6 Countries: A Retrospective Cohort Study

Objectives

To perform an international comparison of the trajectory of laboratory values among hospitalized patients with COVID-19 who develop severe disease and identify optimal timing of laboratory value collection to predict severity across hospitals and regions.

Design

Retrospective cohort study.

Setting

The Consortium for Clinical Characterization of COVID-19 by EHR (4CE), an international multi-site data-sharing collaborative of 342 hospitals in the US and in Europe.

Participants

Patients hospitalized with COVID-19, admitted before or after PCR-confirmed result for SARS-CoV-2.

Primary and secondary outcome measures

Patients were categorized as "ever-severe" or "never-severe" using the validated 4CE severity criteria. Eighteen laboratory tests associated with poor COVID-19-related outcomes were evaluated for predictive accuracy by area under the curve (AUC), compared between the severity categories. Subgroup analysis was performed to validate a subset of laboratory values as predictive of severity against a published algorithm. A subset of laboratory values (CRP, albumin, LDH, neutrophil count, D-dimer, and procalcitonin) was compared between North American and European sites for severity prediction.

Results

Of 36,447 patients with COVID-19, 19,953 (43.7%) were categorized as ever-severe. Most patients (78.7%) were 50 years of age or older and male (60.5%). Longitudinal trajectories of CRP, albumin, LDH, neutrophil count, D-dimer, and procalcitonin showed association with disease severity. Significant differences of laboratory values at admission were found between the two groups. With the exception of D-dimer, predictive discrimination of laboratory values did not improve after admission. Sub-group analysis using age, D-dimer, CRP, and lymphocyte count as predictive of severity at admission showed similar discrimination to a published algorithm (AUC=0.88 and 0.91, respectively). Both models deteriorated in predictive accuracy as the disease progressed. On average, no difference in severity prediction was found between North American and European sites.

Conclusions

Laboratory test values at admission can be used to predict severity in patients with COVID-19. Prediction models show consistency across international sites highlighting the potential generalizability of these models.

Machine Learning of Patient Characteristics to Predict Admission Outcomes in the Undiagnosed Diseases Network

IMPORTANCE

The Undiagnosed Diseases Network (UDN) is a national network that evaluates individual patients whose signs and symptoms have been refractory to diagnosis. Providing reliable estimates of admission outcomes may assist clinical evaluators to distinguish, prioritize, and accelerate admission to the UDN for patients with undiagnosed diseases.

OBJECTIVE

To develop computational models that effectively predict admission outcomes for applicants seeking UDN evaluation and to rank the applications based on the likelihood of patient admission to the UDN.

DESIGN, SETTING, AND PARTICIPANTS

This prognostic study included all applications submitted to the UDN from July 2014 to June 2019, with 1209 applications accepted and 1212 applications not accepted. The main inclusion criterion was an undiagnosed condition despite thorough evaluation by a health care professional; the main exclusion criteria were a diagnosis that explained the objective findings or a review of the records that suggested a diagnosis. A classifier was trained using information extracted from application forms, referral letters from health care professionals, and semantic similarity between referral letters and textual description of known mendelian disorders. The admission labels were provided by the case review committee of the UDN. In addition to retrospective analysis, the classifier was prospectively tested on another 288 applications that were not evaluated at the time of classifier development.

MAIN OUTCOMES AND MEASURES

The primary outcomes were whether a patient was accepted or not accepted to the UDN and application order ranked based on likelihood of admission. The performance of the classifier was assessed by comparing its predictions against the UDN admission outcomes and by measuring improvement in the mean processing time for accepted applications.

RESULTS

The best classifier obtained sensitivity of 0.843, specificity of 0.738, and area under the receiver operating characteristic curve of 0.844 for predicting admission outcomes among 1212 accepted and 1210 not accepted applications. In addition, the classifier can decrease the current mean (SD) UDN processing time for accepted applications from 3.29 (3.17) months to 1.05 (3.82) months (68% improvement) by ordering applications based on their likelihood of acceptance.

CONCLUSIONS AND RELEVANCE

A classification system was developed that may assist clinical evaluators to distinguish, prioritize, and accelerate admission to the UDN for patients with undiagnosed diseases. Accelerating the admission process may improve the diagnostic journeys for these patients and serve as a model for partial automation of triaging or referral for other resource-constrained applications. Such classification models make explicit some of the considerations that currently inform the use of whole-genome sequencing for undiagnosed disease and thereby invite a broader discussion in the clinical genetics community.

High-throughput multimodal automated phenotyping (MAP) with application to PheWAS

OBJECTIVE

Electronic health records linked with biorepositories are a powerful platform for translational studies. A major bottleneck exists in the ability to phenotype patients accurately and efficiently. The objective of this study was to develop an automated high-throughput phenotyping method integrating International Classification of Diseases (ICD) codes and narrative data extracted using natural language processing (NLP).

MATERIALS AND METHODS

We developed a mapping method for automatically identifying relevant ICD and NLP concepts for a specific phenotype leveraging the Unified Medical Language System. Along with health care utilization, aggregated ICD and NLP counts were jointly analyzed by fitting an ensemble of latent mixture models. The multimodal automated phenotyping (MAP) algorithm yields a predicted probability of phenotype for each patient and a threshold for classifying participants with phenotype yes/no. The algorithm was validated using labeled data for 16 phenotypes from a biorepository and further tested in an independent cohort phenome-wide association studies (PheWAS) for 2 single nucleotide polymorphisms with known associations.

RESULTS

The MAP algorithm achieved higher or similar AUC and F-scores compared to the ICD code across all 16 phenotypes. The features assembled via the automated approach had comparable accuracy to those assembled via manual curation (AUCMAP 0.943, AUCmanual 0.941). The PheWAS results suggest that the MAP approach detected previously validated associations with higher power when compared to the standard PheWAS method based on ICD codes.

CONCLUSION

The MAP approach increased the accuracy of phenotype definition while maintaining scalability, thereby facilitating use in studies requiring large-scale phenotyping, such as PheWAS.

Multinational Prevalence of Neurological Phenotypes in Patients Hospitalized with COVID-19

OBJECTIVE:

Neurological complications can worsen outcomes in COVID-19. We defined the prevalence of a wide range of neurological conditions among patients hospitalized with COVID-19 in geographically diverse multinational populations.

METHODS:

Using electronic health record (EHR) data from 348 participating hospitals across 6 countries and 3 continents between January and September 2020, we performed a cross-sectional study of hospitalized adult and pediatric patients with a positive SARS-CoV-2 reverse transcription polymerase chain reaction test, both with and without severe COVID-19. We assessed the frequency of each disease category and 3-character International Classification of Disease (ICD) code of neurological diseases by countries, sites, time before and after admission for COVID-19, and COVID-19 severity.

RESULTS:

Among the 35,177 hospitalized patients with SARS-CoV-2 infection, there was increased prevalence of disorders of consciousness (5.8%, 95% confidence interval [CI]: 3.7%−7.8%, p_FDR<.001) and unspecified disorders of the brain (8.1%, 95%CI: 5.7%−10.5%, p_FDR<.001), compared to pre-admission prevalence. During hospitalization, patients who experienced severe COVID-19 status had 22% (95%CI: 19%−25%) increase in the relative risk (RR) of disorders of consciousness, 24% (95%CI: 13%−35%) increase in other cerebrovascular diseases, 34% (95%CI: 20%−50%) increase in nontraumatic intracranial hemorrhage, 37% (95%CI: 17%−60%) increase in encephalitis and/or myelitis, and 72% (95%CI: 67%−77%) increase in myopathy compared to those who never experienced severe disease.

INTERPRETATION:

Using an international network and common EHR data elements, we highlight an increase in the prevalence of central and peripheral neurological phenotypes in patients hospitalized with SARS-CoV-2 infection, particularly among those with severe disease.