A Framework for Visualizing Study Designs and Data Observability in Electronic Health Record Data

Non-TNFi biologic and targeted synthetic DMARDs in rheumatoid arthritis-associated interstitial lung disease: A propensity score-matched, active-comparator, new-user study

OBJECTIVES

This study aimed to compare treatment outcomes in rheumatoid arthritis-associated interstitial lung disease (RA-ILD) between initiators of rituximab, abatacept, tocilizumab, and tofacitinib using the Target Trial Emulation Framework.

METHODS

We emulated three trials comparing abatacept, tocilizumab, and tofacitinib with rituximab (reference). Patients fulfilling validated RA-ILD algorithms initiating one of these non-TNFi b/tsDMARDs were propensity score (PS)-matched (1:1) using national Veterans Affairs (VA) data from 2006 to 2020. PS models included demographics, comorbidities, general health status indicators, and several RA- and ILD-related severity measures. Composite study outcomes were death and respiratory-related hospitalization, ascertained by VA data and linkages to the National Death Index and Medicare, over three-year (primary) and one-year follow-up periods (secondary). Cox regression models were used to analyze study outcomes adjusting for any unbalanced variables. Several sensitivity and subgroup analyses were performed.

RESULTS

In the primary cohort, we 1:1 matched abatacept (n = 150), tocilizumab (n = 73), and tofacitinib (n = 94) with equal numbers of rituximab initiators (mean age 68.1-69.4 years, 88-92 % male). There were no significant differences in the primary composite outcome among any of the comparisons (abatacept aHR: 1.03 [0.72, 1.47]; tocilizumab aHR: 1.15 [0.68, 1.93]; tofacitinib aHR: 0.89 [0.54, 1.46]). Secondary, subgroup, and sensitivity analyses supported the main findings.

CONCLUSIONS

We did not find significant differences in mortality or respiratory hospitalization between RA-ILD patients initiating different non-TNFi b/tsDMARDs, though estimates were imprecise, and residual confounding may be present. These findings emphasize the need for clinical trials of advanced immunomodulatory therapies in RA-ILD.

Real-world evidence: state-of-the-art and future perspectives

Recent developments in digital infrastructure, advanced analytical approaches, and regulatory settings have facilitated the broadened use of real-world evidence (RWE) in population health management and evaluation of novel health technologies. RWE has uniquely contributed to improving human health by addressing unmet clinical needs, from assessing the external validity of clinical trial data to discovery of new disease phenotypes. In this perspective, we present exemplars across various health areas that have been impacted by real-world data and RWE, and we provide insights into further opportunities afforded by RWE. By deploying robust methodologies and transparently reporting caveats and limitations, real-world data accessed via secure data environments can support proactive healthcare management and accelerate access to novel interventions in England.

Effectiveness of Ampicillin-Sulbactam Versus Ceftriaxone for the Initial Treatment of Community-Acquired Pneumonia in Older Adults: A Target Trial Emulation Study

Background

Current guidelines for community-acquired pneumonia (CAP) include ampicillin-sulbactam as an initial treatment option, though they do not mandate routine coverage of anaerobic organisms. This study aimed to compare the effectiveness of ampicillin-sulbactam with that of ceftriaxone as initial treatment for CAP in older adults.

Methods

This study was conducted using the target trial emulation framework, using a nationwide Japanese database (May 2010-June 2023). The study included patients aged ≥65 years, admitted to Diagnosis Procedure Combination hospitals for CAP, who received either ampicillin-sulbactam or ceftriaxone as the initial treatment. The exposure group received ampicillin-sulbactam, while the control group received ceftriaxone, both on the day of hospitalization. The primary outcome was in-hospital mortality; the secondary outcome was the development of Clostridioides difficile infection during hospitalization.

Results

The study included 26 633 older patients hospitalized with CAP, with 14 906 receiving ampicillin-sulbactam and 11 727 receiving ceftriaxone as initial treatment. After inverse probability of treatment weighting, the ampicillin-sulbactam group was associated with a higher in-hospital mortality rate than the ceftriaxone group (10.5% vs 9.0%, respectively; adjusted risk difference, 1.5% [95% confidence interval, .7%-2.4%]; adjusted odds ratio, 1.19 [1.08-1.31]). The incidence of C difficile infection was numerically higher in the ampicillin-sulbactam group (0.6% vs 0.4%; adjusted risk difference, 0.2% [95% confidence interval, .0%-.4%]; adjusted odds ratio, 1.45 [.99-2.11]). These results were consistent among patients with risk factors for aspiration.

Conclusions

In older patients with CAP, initial treatment with ampicillin-sulbactam was associated with higher mortality compared to treatment with ceftriaxone.

Core Concepts: Self-Controlled Designs in Pharmacoepidemiology

One of the key challenges in pharmacoepidemiological studies is that of uncontrolled confounding, which occurs when confounders are poorly measured, unmeasured or unknown. Self-controlled designs can help address this issue, as their key comparison is not between people, but periods of time within the same person. This controls for all time-stable confounders (genetics) and in the absence of time-varying confounding negates the need for an external control group. However, these benefits come at the cost of strong assumptions, not all of which are verifiable. This review briefly introduces the reader to different types of self-controlled study designs, their terminology and highlights key publications through an annotated reference list. We include a practical description of how these designs can be implemented and visualised using recent examples, and finish by discussing recent developments. We hope this review will serve as a starting point for researchers looking to apply self-controlled designs in their own work.

Effectiveness of Disease-Modifying Treatment on Spinal Cord Lesion Formation in Relapse-Onset Multiple Sclerosis: An MSBase Registry Study

BACKGROUND

Spinal cord lesions in multiple sclerosis (MS) have considerable impact on disability. High-efficacy disease-modifying treatments (hDMTs) are associated with greater reduction of relapses and new brain lesions compared to low-efficacy treatments (lDMTs). Knowledge on the impact of DMTs on cord lesion formation is limited as these outcome measures were not included in MS treatment trials. This study aims to investigate whether hDMTs reduce the formation of cord lesions more effectively than lDMTs.

METHODS

Patients with relapse-onset MS, a cord magnetic resonance imaging (MRI) within 6 months before/after initiation of their first DMT and ≥1 cord MRI at follow-up (interval > 6 months) were extracted from the MSBase registry (ACTRN12605000455662). Patients treated with hDMTs ≥90% or lDMTs ≥90% of follow-up duration were considered the hDMT and lDMT groups, respectively. Matching was performed using propensity scores. Cox proportional hazards models were used to estimate the hazards of new cord lesions, brain lesions and relapses.

RESULTS

Ninety-four and 783 satisfied hDMT and lDMT group criteria, respectively. Seventy-seven hDMT patients were matched to 184 lDMT patients. In the hDMT group there was no evidence of reduction of new cord lesions (hazard ratio [HR] 0.99 [95% CI 0.51, 1.92], p = 0.97), while there were fewer new brain lesions (HR 0.22 [95% CI 0.10, 0.49], p < 0.001) and fewer relapses (HR 0.45 [95% CI 0.28, 0.72], p = 0.004).

CONCLUSION

A potential discrepancy exists in the effect of hDMTs over lDMTs in preventing spinal cord lesions versus brain lesions and relapses. While hDMTs provided a significant reduction for the latter when compared to lDMTs, there was no significant reduction in new spinal cord lesions.

Calibrating Observational Health Record Data Against a Randomized Trial

Importance

The conditions required for health record data sources to accurately assess treatment effectiveness remain unclear. Emulation of randomized clinical trials (RCTs) with health record data and subsequent calibration of the results can help elucidate this.

Objective

To pilot an emulation of the KEYNOTE-189 RCT using a commercially available electronic health record (EHR) data source.

Design, Setting, and Participants

This retrospective cohort study used an EHR database spanning from April 2007 to February 2023. Follow-up began on treatment initiation and proceeded until an outcome event, loss to follow-up, end of data, or end of study period (640 days). The population-based cohort was ascertained from EHRs provided by 52 health systems across the US. Eligibility criteria were defined as closely as possible to the benchmark RCT. Patients with non-small cell lung cancer initiating first-line treatment for metastatic disease were included. Patients with evidence of squamous non-small cell lung cancer, primary nonlung malignant neoplasms, or identified EGFR/ALK variations were excluded. Data were analyzed from June to October 2023.

Exposures

Initiation of first-line pembrolizumab and chemotherapy and chemotherapy alone. Chemotherapy in both groups was defined as a combination of pemetrexed and platinum-based (carboplatin or cisplatin) therapy.

Main Outcomes and Measures

Outcomes of interest were 12-month survival probability and mortality hazard ratio (HR).

Results

A total of 1854 patients (mean [SD] age, 63.7 [9.6] years; 971 [52.4%] men) were eligible, including 589 patients who initiated pembrolizumab and chemotherapy and 1265 patients who initiated chemotherapy only. The cohort included 364 Black patients (19.6%) and 1445 White patients (77.9%). The 12-month survival probabilities were 0.60 (95% CI, 0.54-0.65) in the pembrolizumab group and 0.58 (95% CI, 0.55-0.62) in the chemotherapy-only group, compared with 0.69 (95% CI, 0.64-0.74) in the KEYNOTE-189 pembrolizumab group and 0.49 (95% CI, 0.42-0.56) in the KEYNOTE-189 chemotherapy-only group. The mortality HR was 0.95 (95% CI, 0.78-1.16), compared with 0.49 (95% CI, 0.38-0.64) in the KEYNOTE-189 RCT.

Conclusions and Relevance

In this cohort study piloting an RCT emulation, results were incongruous with the benchmark trial. Differences in patient treatment and data capture between the RCT and EHR populations, confounding by indication, treatment crossover, and accuracy of captured diagnoses may explain these findings. Future feasibility assessments will require data sources to have important oncology-specific measures curated.

Tell me what you want, what you really really want: Estimands in observational pharmacoepidemiologic comparative effectiveness and safety studies

PURPOSE

Ideally, the objectives of a pharmacoepidemiologic comparative effectiveness or safety study should dictate its design and data analysis. This paper discusses how defining an estimand is instrumental to this process.

METHODS

We applied the ICH-E9 (Statistical Principles for Clinical Trials) R1 addendum on estimands - which originally focused on randomized trials - to three examples of observational pharmacoepidemiologic comparative effectiveness and safety studies. Five key elements specify the estimand: the population, contrasted treatments, endpoint, intercurrent events, and population-level summary measure.

RESULTS

Different estimands were defined for case studies representing three types of pharmacological treatments: (1) single-dose treatments using a case study about the effect of influenza vaccination versus no vaccination on mortality risk in an adult population of ≥60 years of age; (2) sustained-treatments using a case study about the effect of dipeptidyl peptidase 4 inhibitor versus glucagon-like peptide-1 agonist on hypoglycemia risk in treatment of uncontrolled diabetes; and (3) as needed treatments using a case study on the effect of nitroglycerin spray as-needed versus no nitroglycerin on syncope risk in treatment of stabile angina pectoris.

CONCLUSIONS

The case studies illustrated that a seemingly clear research question can still be open to multiple interpretations. Defining an estimand ensures that the study targets a treatment effect that aligns with the treatment decision the study aims to inform. Estimand definitions further help to inform choices regarding study design and data-analysis and clarify how to interpret study findings.

The Impact of Longitudinal Data-Completeness of Electronic Health Record Data on the Prediction Performance of Clinical Risk Scores

The impact of electronic health record (EHR) discontinuity (i.e., receiving care outside of a given EHR system) on EHR-based risk prediction is unknown. We aimed to assess the impact of EHR-continuity on the performance of clinical risk scores. The study cohort consisted of patients aged ≥ 65 years with ≥ 1 EHR encounter in the 2 networks in Massachusetts (MA; 2007/1/1-2017/12/31, internal training and validation dataset), and one network in North Carolina (NC; 2007/1/1-2016/12/31, external validation dataset) that were linked with Medicare claims data. Risk scores were calculated using EHR data alone vs. linked EHR-claims data (not subject to misclassification due to EHR-discontinuity): (i) combined comorbidity score (CCS), (ii) claim-based frailty score (CFI), (iii) CHAD2 DS2 -VASc, and (iv) Hypertension, Abnormal renal/liver function, Stroke, Bleeding, Labile, Elderly, and Drugs (HAS-BLED). We assessed the performance of CCS and CFI predicting death, CHAD2 DS2 -VASc predicting ischemic stroke, and HAS-BLED predicting bleeding by area under receiver operating characteristic curve (AUROC), stratified by quartiles of predicted EHR-continuity (Q1-4). There were 319,740 patients in the MA systems and 125,380 in the NC system. In the external validation dataset, AUROC for EHR-based CCS predicting 1-year risk of death was 0.583 in Q1 (lowest) EHR-continuity group, which increased to 0.739 in Q4 (highest) EHR-continuity group. The corresponding improvement in AUROC was 0.539 to 0.647 for CFI, 0.556 to 0.637 for CHAD2 DS2 -VASc, and 0.517 to 0.556 for HAS-BLED. The AUROC in Q4 EHR-continuity group based on EHR alone approximates that based on EHR-claims data. The prediction performance of four clinical risk scores was substantially worse in patients with lower vs. high EHR-continuity.

Worldwide Trends in Registering Real-World Studies at ClinicalTrials.gov: A Cross-Sectional Analysis

Objective

The purpose of this study was to characterize real-world studies (RWSs) registered at ClinicalTrials.gov to help investigators better conduct relevant research in clinical practice.

Methods

A retrospective analysis of 944 studies was performed on February 28, 2023.

Results

A total of 944 studies were included. The included studies involved a total of 48 countries. China was the leading country in terms of the total number of registered studies (37.9%, 358), followed by the United States (19.7%, 186). Regarding intervention type, 42.4% (400) of the studies involved drugs, and only 9.1% (86) of the studies involved devices. Only 8.5% (80) of the studies mentioned both the detailed study design type and data source in the "Brief Summary". A total of 49.4% (466) of studies had a sample size of 500 participants and above. Overall, 63% (595) of the studies were single-center studies. A total of 213 conditions were covered in the included studies. One-third of the studies (32.7%, 309) involved neoplasms (or tumors). China and the United States were very different regarding the study of different conditions.

Conclusion

Although the pandemic has provided new opportunities for RWSs, the rigor of scientific research still needs to be emphasized. Special attention needs to be given to the correct and comprehensive description of the study design in the Brief Summary of registered studies, thereby promoting communication and understanding. In addition, deficiencies in ClinicalTrials.gov registration data remain prominent.

Visualizations throughout pharmacoepidemiology study planning, implementation, and reporting

Transparency is increasingly promoted to instill trust in nonrandomized studies using real-world data. Graphics and data visualizations support transparency by aiding communication and understanding, and can inform study design and analysis decisions. However, other than graphical representation of a study design and flow diagrams (e.g., a Consolidated Standards of Reporting Trials [CONSORT] like diagram), specific standards on how to maximize validity and transparency with visualization are needed. This paper provides guidance on how to use visualizations throughout the life cycle of a pharmacoepidemiology study-from initial study design to final report-to facilitate rationalized and transparent decision-making about study design and implementation, and clear communication of study findings. Our intent is to help researchers align their practices with current consensus statements on transparency.