Reproducibility of real-world evidence studies using clinical practice data to inform regulatory and coverage decisions

Emulating randomized trials by observational database studies: the RCT-DUPLICATE initiative in COPD and asthma

Observational studies are increasingly used to provide real-world evidence in regulatory decision-making. The RCT-DUPLICATE initiative conducted observational studies emulating 2 published randomized trials in patients with asthma and 3 in chronic obstructive pulmonary disease (COPD). For each trial, new-user cohorts were constructed from 2 US healthcare claims databases, comparing initiators of the study and comparator drugs, matched on propensity scores. Proportional hazards models were used to compare the treatments on study outcomes. The observational studies involved more subjects than the corresponding trials, with treatment arms well-matched on baseline characteristics. An asthma example involved emulation of the 26-week FDA-mandated D5896 trial. With 6494 asthma patients per arm, the hazard ratio (HR) of a serious asthma-related event with budesonide-formoterol vs budesonide was 1.29 (95% CI, 0.63-2.65) compared with 1.07 (95% CI, 0.70-1.65) in the trial. A COPD example is the emulation of the one-year IMPACT trial. With 4365 COPD patients per arm, the HR of a COPD exacerbation with triple therapy vs dual bronchodilators was 1.08 (95% CI, 1.00-1.17) compared with 0.84 (95% CI, 0.78-0.91) in the trial. We found mainly discordant results between observational analyses and their emulated randomized trials, likely from the forced discontinuation of treatments prior to randomization in the trials, not mimicable in the observational analyses. This article is part of a Special Collection on Pharmacoepidemiology.

Trends in Utilization of Glucose- and Weight-Lowering Medications After Tirzepatide Approval in the United States : A Population-Based Cohort Study

BACKGROUND

Recent trends in use of tirzepatide, a dual glucagon-like peptide-1 (GLP-1) and glucose-dependent insulinotropic polypeptide receptor agonist (RA), versus other glucose-lowering medications (GLMs) and weight-lowering medications (WLMs) remain unexplored.

OBJECTIVE

To describe trends in insurance claims for GLMs and WLMs after tirzepatide approval.

DESIGN

Population-based cohort study.

SETTING

Claims data from a large U.S. commercial database (January 2021 to December 2023).

PARTICIPANTS

Adults (aged ≥18 years) with type 2 diabetes (T2D) and without diabetes with dispensations for GLMs and WLMs. Any use was defined as medication dispensation regardless of prior use. Incident use was defined as dispensation without use in the preceding year.

MEASUREMENTS

Monthly trends in medication dispensations before and after tirzepatide market entry. Tirzepatide uptake was additionally compared with initial postapproval uptake of other GLMs and WLMs.

RESULTS

Tirzepatide dispensations increased markedly among adults with T2D prescribed GLMs, reaching 12.3% of all GLM dispensations by December 2023. Similar patterns were observed for sodium-glucose cotransporter-2 inhibitors (14.5% to 24.4%) and GLP-1 RAs (19.5% to 28.5%). Dispensations of other GLMs, including metformin, declined. Among adults without diabetes but prescribed WLMs, tirzepatide (0.0% to 40.6%) and semaglutide (2.4 mg) (0.0% to 32.2%) dispensations increased sharply, but semaglutide (2.0 mg) was the most frequently dispensed WLM, increasing from 37.8% to 45.7%. Dispensations of other WLMs declined. Similar trends were observed among incident users. Tirzepatide uptake was more rapid and sustained compared with initial postapproval periods for other medications.

LIMITATION

Generalizability to U.S. adults without commercial health insurance is uncertain.

CONCLUSION

These findings highlight the sharp uptake of tirzepatide after U.S. market entry and enhance understanding of the rapidly shifting landscape of prescribing patterns for GLMs and WLMs.

PRIMARY FUNDING SOURCE

National Institute of Diabetes and Digestive and Kidney Diseases.

A Scalable Approach to Assess the Safety of Recently Marketed Systemic Treatments for Atopic Dermatitis in Clinical Practice: First Analysis Cycle of a Sequential Monitoring System

Targeted systemic immune-modulating drugs to treat atopic dermatitis were highly efficacious in randomized trials. Trials with limited number of subjects leave questions about their safety. We describe a data and analytics structure for the production of timely, high-quality evidence on the comparative safety of recently approved immune-modulating drugs in patients with atopic dermatitis in clinical practice. We established a series of sequential propensity score-balanced cohorts that grow in size with each annual data refresh. Nine health outcomes of interest plus conjunctivitis as a positive tracer outcome were identified. The initial treatment comparison was dupilumab, an IL-4/13 inhibitor, or tralokinumab, an IL-13 inhibitor, versus abrocitinib/upadacitinib, both Jak inhibitors. The first analysis cycle (December 2021-February 2023) compared 269 patients initiating Jak inhibitors and 2650 initiating IL-4/IL-13 inhibitors. Patient characteristics were well-balanced after propensity score matching. Outpatient infections within 180 days occurred in 18% of Jak1 inhibitor initiators versus 12% of dupilumab/tralokinumab initiators (risk ratio = 1.50, 95% confidence interval = 0.96-2.33), whereas acne risks were 7 versus 3%, respectively (risk ratio = 2.29, 95% confidence interval = 0.96-5.46). This sequential monitoring system will produce essential knowledge on the safety of immune-modulating drugs to treat atopic dermatitis on the basis of its growing study size of patients observed in clinical practice.

Perspective review: Will generative AI make common data models obsolete in future analyses of distributed data networks?

Integrating real-world healthcare data is challenging due to diverse formats and terminologies, making standardization resource-intensive. While Common Data Models (CDMs) facilitate interoperability, they often cause information loss, exhibit semantic inconsistencies, and are labor-intensive to implement and update. We explore how generative artificial intelligence (GenAI), especially large language models (LLMs), could make CDMs obsolete in quantitative healthcare data analysis by interpreting natural language queries and generating code, enabling direct interaction with raw data. Knowledge graphs (KGs) standardize relationships and semantics across heterogeneous data, preserving integrity. This perspective review proposes a fourth generation of distributed data network analysis, building on previous generations categorized by their approach to data standardization and utilization. It emphasizes the potential of GenAI to overcome the limitations CDMs with GenAI-enabled access, KGs, and automatic code generation. A data commons may further enhance this capability, and KGs may well be needed to enable effective GenAI. Addressing privacy, security, and governance is critical; any new method must ensure protections comparable to CDM-based models. Our approach would aim to enable efficient, real-time analyses across diverse datasets and enhance patient safety. We recommend prioritizing research to assess how GenAI can transform quantitative healthcare data analysis by overcoming current limitations.

Glucose-Lowering Medications and Risk of Chronic Obstructive Pulmonary Disease Exacerbations in Patients With Type 2 Diabetes

Importance

Recent studies have suggested that sodium-glucose cotransporter-2 inhibitors (SGLT-2is), glucagon-like peptide-1 receptor agonists (GLP-1RAs), and dipeptidyl peptidase 4 inhibitors (DPP-4is) may benefit patients with chronic obstructive pulmonary disease (COPD). However, clinical evidence is lacking on their comparative association with COPD exacerbations in US patients with type 2 diabetes (T2D).

Objective

To compare the risk of moderate or severe COPD exacerbations among SGLT-2is, GLP-1RAs, and DPP-4is.

Design, Setting, and Participants

This comparative effectiveness research study used data from three 1:1 propensity score-matched cohort studies that emulated 3 target trials comparing patients 40 years or older with T2D and active COPD who initiated treatment with SGLT-2is vs DPP-4is, GLP-1RAs vs DPP-4is, and SGLT-2is vs GLP-1RAs. Data were from 3 US insurance claims databases: the Optum deidentified Clinformatics Data Mart Database (2013-2023), IBM Health MarketScan (2013-2021), and Medicare fee for service (2013-2020). The data analysis was conducted from January to June 2024.

Exposures

Initiation of SGLT-2i or DPP-4i, GLP-1RA or DPP-4i, and SGLT-2i or GLP-1RA for the 3 target trials, respectively.

Main Outcomes and Measures

First occurrence of a moderate or severe COPD exacerbation, defined as a filled prescription for oral glucocorticoids in association with an outpatient COPD visit or hospitalization for COPD. Incidence rates, incidence rate differences (IRDs), and hazard ratios (HRs) with 95% CIs were calculated.

Results

There were 27 991, 32 107, and 36 218 pairs in the SGLT-2i vs DPP-4i, GLP-1RA vs DPP-4i, and SGLT-2i vs GLP-1RA propensity score-matched cohorts, respectively (mean [SD] age, 70.8 [8.6] and 70.7 [8.8], 70.4 [8.5] and 70.4 [8.2], and 69.8 [8.7] years, respectively; 13 767 [49.2%] and 13 847 [49.5%], 17 622 [54.9%] and 17 620 [54.9%], and 18 807 [51.9%] and 18 854 [52.1%] female individuals, respectively). During a median follow-up of 145 (IQR, 61-355) days of treatment, the risk of moderate or severe COPD exacerbation was lower among those treated with SGLT-2is vs DPP-4is (9.26 vs 11.4 per 100 person-years [PYs]; HR, 0.81; 95% CI, 0.76-0.86; IRD/100 PYs, -2.20; 95% CI, -2.83 to -1.58) and among those treated with GLP-1RAs vs DPP-4is (9.89 vs 11.49 per 100 PYs; HR, 0.86; 95% CI, 0.81-0.91; IRD/100 PYs, -1.60; 95% CI, -2.18 to -1.02), with minimal differences among those treated with SGLT-2is vs GLP-1RAs (9.47 vs 10.00 per 100 PYs; HR, 0.94; 95% CI, 0.89-1.00; IRD/100 PYs, -0.55; 95% CI, -1.09 to -0.01). Results were consistent across sensitivity and subgroup analyses.

Conclusions and Relevance

The results of this comparative effectiveness research study suggest that SGLT-2is and GLP-1RAs were associated with a reduced risk of moderate or severe COPD exacerbations compared with DPP-4i in adults with T2D and active COPD. This may inform prescribing of glucose-lowering medications among patients with T2D and active COPD.

Transparency of research practices in cardiovascular literature

Background

Several fields have described low reproducibility of scientific research and poor accessibility in research reporting practices. Although previous reports have investigated accessible reporting practices that lead to reproducible research in other fields, to date, no study has explored the extent of accessible and reproducible research practices in cardiovascular science literature.

Methods

To study accessibility and reproducibility in cardiovascular research reporting, we screened 639 randomly selected articles published in 2019 in three top cardiovascular science publications: Circulation, the European Heart Journal, and the Journal of the American College of Cardiology (JACC). Of those 639 articles, 393 were empirical research articles. We screened each paper for accessible and reproducible research practices using a set of accessibility criteria including protocol, materials, data, and analysis script availability, as well as accessibility of the publication itself. We also quantified the consistency of open research practices within and across cardiovascular study types and journal formats.

Results

We identified that fewer than 2% of cardiovascular research publications provide sufficient resources (materials, methods, data, and analysis scripts) to fully reproduce their studies. Of the 639 articles screened, 393 were empirical research studies for which reproducibility could be assessed using our protocol, as opposed to commentaries or reviews. After calculating an accessibility score as a measure of the extent to which an article makes its resources available, we also showed that the level of accessibility varies across study types with a score of 0.08 for case studies or case series and 0.39 for clinical trials (p = 5.500E-5) and across journals (0.19 through 0.34, p = 1.230E-2). We further showed that there are significant differences in which study types share which resources.

Conclusions

Although the degree to which reproducible reporting practices are present in publications varies significantly across journals and study types, current cardiovascular science reports frequently do not provide sufficient materials, protocols, data, or analysis information to reproduce a study. In the future, having higher standards of accessibility mandated by either journals or funding bodies will help increase the reproducibility of cardiovascular research.

Funding

Authors Gabriel Heckerman, Arely Campos-Melendez, and Chisomaga Ekwueme were supported by an NIH R25 grant from the National Heart Lung and Blood Institute (R25HL147666). Eileen Tzng was supported by an AHA Institutional Training Award fellowship (18UFEL33960207).

Real-World Tolerability of Capecitabine and Oxaliplatin in Patients in the United States With Localized Colorectal Cancer Undergoing Curative-Intent Treatment

PURPOSE

The combination of capecitabine and oxaliplatin (CAPOX) is commonly used in patients with localized colorectal cancer (CRC) receiving curative-intent treatment. Our study aimed to assess the real-world tolerability of CAPOX in a single-institution cohort of patients with localized CRC.

METHODS

This is a single-institution retrospective study that included patients with localized CRC receiving neoadjuvant or adjuvant CAPOX. The primary end point was completion rate of intended number (obtained by chart review) of CAPOX cycles irrespective of dose levels. Secondary outcome measures included the rate of grade ≥3 adverse events, hospital admission rate, and dose reductions.

RESULTS

The study included 153 patients with a median age of 61 years; 49% were female and 78.4% had stage III CRC. The proportion of patients (95% CI) who completed all planned CAPOX cycles was 44.4% (36 to 52) in the entire cohort and 34.6% (23 to 45) among female patients. Independent variables associated with treatment completion in multivariable analysis were race, sex, and intended number of cycles. Notably, the therapy completion rates (95% CI) were 55% (43 to 66) and 33% (20 to 45) in patients intended to receive four and eight cycles of CAPOX, respectively. The rate of grade ≥3 adverse events and hospitalization because of CAPOX-related toxicity were 30.7% (95% CI, 23 to 38) and 17.6% (95% CI, 11 to 23), respectively.

CONCLUSION

This study highlights that a substantial number of patients with localized CRC undergoing curative-intent treatment with CAPOX do not complete the planned cycles of chemotherapy because of toxicity. These findings underscore the need for careful patient selection and appropriate supportive care to optimize the therapeutic benefit of CAPOX in this setting.

Transparency in the secondary use of health data: assessing the status quo of guidance and best practices

We evaluated what guidance exists in the literature to improve the transparency of studies that make secondary use of health data. To find peer-reviewed papers, we searched PubMed and Google Scholar. To find institutional documents, we used our personal expertise to draft a list of health organizations and searched their websites. We quantitatively and qualitatively coded different types of research transparency: registration, methods reporting, results reporting, data sharing and code sharing. We found 56 documents that provide recommendations to improve the transparency of studies making secondary use of health data, mainly in relation to study registration (n = 27) and/or methods reporting (n = 39). Only three documents made recommendations on data sharing or code sharing. Recommendations for study registration and methods reporting mainly came in the form of structured documents like registration templates and reporting guidelines. Aside from the recommendations aimed directly at researchers, we also found recommendations aimed at the wider research community, typically on how to improve research infrastructure. Limitations or challenges of improving transparency were rarely mentioned, highlighting the need for more nuance in providing transparency guidance for studies that make secondary use of health data.

Real-world clinical and economic outcomes from rapid start antiretroviral therapy in HIV: systematic review and meta-analysis

OBJECTIVE

We aimed to synthesize clinical and economic outcomes of rapid start versus nonrapid antiretroviral therapy (ART) in people with HIV (PWH) in real-world settings.

METHODS

A search was conducted in PubMed, Embase, Web of Science, and ProQuest from January 2017 to January 2023, supplementing a previous search by Ford et al . in 2018. Observational studies investigating clinical or economic outcomes of rapid start ART versus nonrapid ART in PWH were included. Clinical outcomes were mortality, loss-to-follow-up (LTFU), and viral suppression. Economic outcomes were incremental cost-effectiveness ratio (ICER) values and per patient per month (PPPM) costs. Meta-analyses using random-effects models were performed for clinical outcomes, whereas qualitative syntheses were conducted for economic outcomes. The quality of clinical and economic studies was assessed.

RESULTS

Sixty-two studies were included. The pooled adjusted risk ratio (aRR) for mortality demonstrated a significant reduction in risk of mortality among participants who received rapid start ART compared with nonrapid ART [0.80, 95% confidence interval (CI), 0.65-0.98]. For LTFU at 6 and 12 months, the pooled aRR showed increased LTFU for rapid start ART (1.33, 95% CI, 1.15-1.55 and 1.18, 95% CI, 0.74-1.89 respectively). All cost-effectiveness studies reported cost-saving or cost-effective findings. The PPPM costs of rapid start ART across the first 36 months of treatment were consistently lower than nonrapid ART.

CONCLUSION

Rapid ART is associated with reduced mortality and is cost-effective compared with nonrapid ART in real-world settings. Clinicians and policymakers should consider these findings to facilitate rapid start of ART in PWH. Further research on LTFU in PWH is needed.

Efficacy Versus Effectiveness: The HORIZON-Pivotal Fracture Trial and Its Emulation in Claims Data

OBJECTIVE

The objective of this study is to evaluate the concordance of results between the Health Outcomes and Reduced Incidence with Zoledronic Acid Once Yearly (HORIZON)-Pivotal Fracture Trial (PFT) and a nonrandomized database study designed to emulate the trial.

METHODS

HORIZON-PFT evaluated the efficacy of zoledronic acid versus placebo in reducing the risk of hip fractures and found a 41% risk reduction over a three-year treatment period (hazard ratio [HR] = 0.59; 95% confidence interval [95% CI] 0.42-0.83). Using two US claims databases from August 2007 to December 2020 or June 2021, we applied eligibility criteria from HORIZON-PFT and identified women with osteoporosis who initiated zoledronic acid or raloxifene as a proxy for placebo. The study protocol was registered on ClinicalTrials.gov (NCT04736693) before inferential analyses. We compared HORIZON-PFT and database study results using prespecified metrics.

RESULTS

Because of low adherence in clinical practice, on-treatment follow-up was truncated at 18 months in the database study. The hip fracture risk after 18 months was 9.3 in 1,000 in the trial and 8.3 in 1,000 in the database analysis. In the database study, zoledronic acid was associated with a 28% reduction in hip fractures risk compared with raloxifene (HR = 0.72; 95% CI 0.51-0.92). The attenuated effect of zoledronic acid in the database study may be explained by its shorter follow-up, because the interpolated estimate of the effect in HORIZON-PFT at 18 months was HRRCT (randomized controlled trial), 0.74, nearly identical to the observational estimate HRdatabase 0.72.

CONCLUSION

Real-world emulation of the HORIZON-PFT found that zoledronic acid reduced hip fractures risk over an 18-month follow-up period. Limited adherence in clinical practice diminished the magnitude of its preventive effect and precluded long-term estimation of effectiveness in this setting.

Glucagon-Like Peptide 1 Receptor Agonists and Chronic Lower Respiratory Disease Among Type 2 Diabetes Patients: Replication and Reliability Assessment Across a Research Network

INTRODUCTION

The aim of this study is to use observational methods to evaluate reliability of evidence generated by a study of the effect of glucagon-like peptide 1 receptor agonists (GLP-1RA) on chronic lower respiratory disease (CLRD) outcomes among Type-2 diabetes mellitus (T2DM) patients.

RESEARCH DESIGN AND METHODS

We independently reproduced a study comparing effects of GLP-1RA versus dipeptidyl peptidase-4 inhibitors (DPP4-i) on CLRD outcomes among patients with T2DM and prior CLRD. We reproduced inputs and outputs using the original study data (national administrative claims) and evaluated the robustness of results in comparison to alternate design/analysis decisions. To evaluate generalizability, we applied an analysis protocol and conducted a meta-analysis across a research network that includes a diverse array of populations and data sources. We also produced additional analyses evaluating individual drugs within the GLP-1RA class and CLRD outcomes.

RESULTS

We confirmed alignment of study inputs and outputs and closely reproduced effect estimates and sensitivity analyses. Adjusted effect estimates were robust to empirical calibration. Network meta-analysis confirmed original findings but indicated weaker effects than originally published. Meta-analysis of drugs within the GLP-1RA class against DPP4-i provided evidence that effects vary within the GLP-1RA class, indicating stronger effects for exenatide and weaker effects of dulaglutide.

CONCLUSIONS

This study supports and establishes the reliability of the original study by (1) producing consistent findings in a range of alternate databases and populations, (2) demonstrating effects for multiple drugs within the GLP-1RA class, and (3) independently confirming the reproducibility of the original study and its findings. This reliability evaluation provides the beginnings of a broader framework for using standardized tools and distributed data networks to systematically interrogate the reliability of findings generated using observational data.

Learning from European regulator-initiated studies for regulatory decision making

Drug developers have displayed a growing interest in using real-word evidence (RWE) in the pre-marketing phase. Together with initiatives like DARWIN EU®, this interest encourages regulators to initiate their own observational clinical studies, which could be included in regulatory decision making. The Regulatory Science Network Netherlands (RSNN) organised an expert meeting in 2022 to discuss scenarios and learnings related to these regulator-initiated studies, of which the main points (e.g., transparency, independency and stakeholder interaction) have been published previously. In this review, the authors add their own views, underlining the importance of reproducibility and stakeholder interaction. Stakeholders should collaborate to embrace regulator-initiated studies in a timely and transparent manner to realise an optimal European framework for generating RWE to be included in the regulatory decision-making process.

Project Victoria: A pragmatic data model to automate RWE generation from the national French claims database

Objective: This paper describes Victoria, an empirically built data pipeline for SNDS to: - Build an automated, scalable pipeline supporting changes to the data model inherent to the use of large databases, - Deliver a documented pipeline with clear processes, enabling scientific, epidemiological researches, - Ease access to SNDS data in compliance with regulatory requirements. Methods: This paper describes the 2-steps process of the Victoria pipeline and its final output. The initial cleaning step consists in formatting, deleting empty, error or duplicate records and renaming variables without changing their values, accordingly with the official SNDS documentation. The second step consists in creating 2 linearised data models: every line of each table is an event, and each table is indexed with a unique patient identifier, without the need for a central patient or identifier table. These 2 models are: - the epidemiological model, used for answering most of the research questions requiring population phenotyping (demography, diagnosis, procedures characteristics). - the medico-economic model is used for costs and healthcare consumption analyses. It contains more complex information about reimbursements rates and the data quality assessment is focused on costs rather than medico-administrative information. Results: The pipeline was executed on 2 different datasets representing ∼85 000 and ∼870 000 beneficiaries with the following configuration: one master with 4 cores and 16Go of RAM and respectively 4 and 6 workers. The total execution time for the smaller dataset was 25 h and 96 h for the larger one. The longest part of those times is represented by the format conversion to parquet. The cleaning step took only 4 h in both cases. The epidemiological model took 344 min for the smaller dataset and 1934 min for the larger one. The medico-economic model took the longest time with 704 min and 2145 min, respectively. Conclusion: Victoria pipeline is a successfully implemented SNDS pipeline. Compared to previous pipelines, reviewability is part of its design as unit tests and quality assessments can natively be developed to ensure data and analysis quality. The pipeline has been used for 2 published studies. The recent work toward OMOP conversion will be integrated in upcoming versions and, as Victoria is set to run on a CD platform, the potential evolution if SNDS format can be considered.

Comparative effectiveness and safety of single inhaler triple therapies for chronic obstructive pulmonary disease: new user cohort study

OBJECTIVE

To compare the effectiveness and safety of budesonide-glycopyrrolate-formoterol, a twice daily metered dose inhaler, and fluticasone-umeclidinium-vilanterol, a once daily dry powder inhaler, in patients with chronic obstructive pulmonary disease (COPD) treated in routine clinical practice.

DESIGN

New user cohort study.

SETTING

Longitudinal commercial US claims data.

PARTICIPANTS

New initiators of budesonide-glycopyrrolate-formoterol or fluticasone-umeclidinium-vilanterol between 1 January 2021 and 30 September 2023 who had a diagnosis of COPD and were aged 40 years or older.

MAIN OUTCOME MEASURES

In this 1:1 propensity score matched study, the main outcome measures were first moderate or severe COPD exacerbation (effectiveness) and first admission to hospital with pneumonia (safety) while on treatment. Potential confounders were measured in the 365 days before cohort entry and included in propensity scores. Hazard ratios and 95% confidence intervals (CIs) were estimated using a Cox proportional hazards regression model.

RESULTS

The study cohort included 20 388 propensity score matched pairs of new users initiating single inhaler triple therapy. Patients who received budesonide-glycopyrrolate-formoterol had a 9% higher incidence of first moderate or severe COPD exacerbation (hazard ratio 1.09 (95% CI 1.04 to 1.14); number needed to harm 38) compared with patients receiving fluticasone-umeclidinium-vilanterol and an identical incidence of first admission to hospital with pneumonia (1.00 (0.91 to 1.10)). The hazard of first moderate COPD exacerbation was 7% higher (1.07 (1.02 to 1.12); number needed to harm 54) and the hazard of first severe COPD exacerbation 29% higher (1.29 (1.12 to 1.48); number needed to harm 97) among those receiving budesonide-glycopyrrolate-formoterol compared to fluticasone-umeclidinium-vilanterol. Prespecified sensitivity analyses yielded similar findings to the primary analysis.

CONCLUSIONS

Budesonide-glycopyrrolate-formoterol was not associated with improved clinical outcomes compared with fluticasone-umeclidinium-vilanterol. Given the added climate impact of metered dose inhalers, health systems seeking to decrease use of these products may consider steps to promote further prescribing of fluticasone-umeclidinium-vilanterol compared with budesonide-glycopyrrolate-formoterol in people with COPD.

STUDY REGISTRATION

Center for Open Science Real World Evidence Registry (https://osf.io/6gdyp/).

Assessing methods to ascertain persistence and adherence of oral anticoagulants in patients with atrial fibrillation

BACKGROUND

Persistence and adherence to oral anticoagulants (OACs) is crucial for its effectiveness in stroke prevention in atrial fibrillation (AF). We aimed to assess the impact of different ascertainment methods on estimated persistence rates.

METHODS

We conducted a retrospective cohort study based on the Medicare claims data (01/01/2013-12/31/2019). We built an incident user cohort of OAC (apixaban, dabigatran, edoxaban, rivaroxaban, and warfarin) prescription filling. We measured OAC medication persistence and adherence using the following approaches: (1) treatment-anniversary based persistence: if there is an active prescription overlapping the 180th and 365th day with vs. without a 15-day buffer period (i.e., overlapping with 165th-195th and 350th-380th day); (2) dispensing-gap-based persistence: if there is OAC discontinuation defined as having gap between prescriptions more than a threshold (e.g., 5-60 days) and secondarily, (3) proportion of days covered (PDC) adherence: proportion of days in which patient had filled medication available over the 365-day interval.

RESULTS

We identified 1,398,692 patients who initiated OACs during the study interval. With the treatment-anniversary based approach, only 51.2% to 65.4% of the patients persisted with the medication for either warfarin or DOACs at 180 days, and the number dropped to 43.4% to 60.7% at 1 year. Adding a 15-day buffer period increased the treatment-anniversary based persistence rates by 6.5% to 10.5%. When the allowable gap increased from 5 to 60 days, the persistence rates increased by 36.3% to 42.4% for all OACs. Apixaban users had the highest PDC (74%-75%) over the 365 days, compared to other OACs (60%-69%).

CONCLUSIONS

We found that the estimated persistence rates are sensitive to the choice of ascertainment methods. When reporting and comparing persistence findings using the claims database, definitions of OAC discontinuation must be clearly delineated.

Real-World Evidence BRIDGE: A Tool to Connect Protocol With Code Programming

OBJECTIVE

To enhance documentation on programming decisions in Real World Evidence (RWE) studies.

MATERIALS AND METHODS

We analyzed several statistical analysis plans (SAP) within the Vaccine Monitoring Collaboration for Europe (VAC4EU) to identify study design sections and specifications for programming RWE studies. We designed a machine-readable metadata schema containing study sections, codelists, and time anchoring definitions specified in the SAPs with adaptability and user-friendliness.

RESULTS

We developed the RWE-BRIDGE, a metadata schema in form of relational database divided into four study design sections with 12 tables: Study Variable Definition (two tables), Cohort Definition (two tables), Post-Exposure Outcome Analysis (one table), and Data Retrieval (seven tables). We provide a guide to populate this metadata schema and a Shiny app that checks the tables. RWE-BRIDGE is available on GitHub (github.com/UMC-Utrecht-RWE/RWE-BRIDGE).

DISCUSSION

The RWE-BRIDGE has been designed to support the translation of study design sections from statistical analysis plans into analytical pipelines and to adhere to the FAIR principles, facilitating collaboration and transparency between researcher and programmers. This metadata schema strategy is flexible as it can support different common data models and programming languages, and it is adaptable to the specific needs of each SAP by adding further tables or fields, if necessary. Modified versions of the RWE-BRIGE have been applied in several RWE studies within VAC4EU.

CONCLUSION

RWE-BRIDGE offers a systematic approach to detailing variables, time anchoring, and algorithms for RWE studies. This metadata schema facilitates communication between researcher and programmers.

Health Care Utilization and Costs Associated With Empagliflozin in Older Adults With Type 2 Diabetes

OBJECTIVE

We compared health care resource utilization (HCRU) and costs for inpatient and outpatient services and dispensed medications in older adults with type 2 diabetes initiating empagliflozin versus dipeptidyl peptidase 4 inhibitors (DPP-4is).

RESEARCH DESIGN AND METHODS

The study population included U.S. Medicare fee-for-service beneficiaries with diabetes (age ≥65 years) initiating empagliflozin or DPP-4is (August 2014 to September 2018). We estimated rate ratios (RRs) for HCRU outcomes using zero-inflated negative binomial regression and per-member per-year (PMPY) cost differences using generalized linear model with gamma distributions, overall and stratified by baseline cardiovascular disease (CVD), after adjusting for 143 baseline covariates using 1:1 propensity score matching.

RESULTS

We identified 23,335 matched pairs (mean age 72 years, 51% with baseline CVD). HCRU rates were lower in empagliflozin versus DPP-4i initiators (number of inpatient days, RR 0.89 [95% CI 0.82, 0.97]; number of emergency department [ED] visits, 0.86 [0.82, 0.91]; number of hospitalizations, 0.86 [0.79, 0.93]; number of office visits, 0.96 [0.95, 0.98]). Inpatient cost (-$713 PMPY [95% CI -847, -579), outpatient cost (-$198 PMPY [-272, -124]), and total cost of care (-$1,109 PMPY [-1,478, -739]) were lower for empagliflozin versus DPP-4is, although diabetes medication cost was higher in empagliflozin initiators ($454 PMPY [95% CI 284, 567]). In the CVD subgroup, total cost was lower for empagliflozin initiators (-$2,005 PMPY [-2,451, -1,337]), while the difference was attenuated in the non-CVD subgroup (-$296 PMPY [-740, 148]).

CONCLUSIONS

Among older adults with diabetes, empagliflozin was associated with a lower number of inpatient days, hospitalizations, ED visits, and office visits and with lower costs of care compared with DPP-4is, especially in those with history of CVD.

Characteristics of non-randomised studies of drug treatments: cross sectional study

ABSTRACT

Objective

To examine the characteristics of comparative non-randomised studies that assess the effectiveness or safety, or both, of drug treatments.

Design

Cross sectional study.

Data sources

Medline (Ovid), for reports published from 1 June 2022 to 31 August 2022.

Eligibility criteria for selecting studies

Reports of comparative non-randomised studies that assessed the effectiveness or safety, or both, of drug treatments were included. A randomly ordered sample was screened until 200 eligible reports were found. Data on general characteristics, reporting characteristics, and time point alignment were extracted, and possible related biases, with a piloted form inspired by reporting guidelines and the target trial emulation framework.

Results

Of 462 reports of non-randomised studies identified, 262 studies were excluded (32% had no comparator and 25% did not account for confounding factors). To assess time point alignment and possible related biases, three study time points were considered: eligibility, treatment assignment, and start of follow-up. Of the 200 included reports, 70% had one possible bias, related to: inclusion of prevalent users in 24%, post-treatment eligibility criteria in 32%, immortal time periods in 42%, and classification of treatment in 23%. Reporting was incomplete, and only 2% reported all six of the key elements considered: eligibility criteria (87%), description of treatment (46%), deviations in treatment (27%), causal contrast (11%), primary outcomes (90%), and confounding factors (88%). Most studies used routinely collected data (67%), but only 7% reported using validation studies of the codes or algorithms applied to select the population. Only 7% of reports mentioned registration on a trial registry and 3% had an available protocol.

Conclusions

The findings of the study suggest that although access to real world evidence could be valuable, the robustness and transparency of non-randomised studies need to be improved.

FOUNTAIN: a modular research platform for integrated real-world evidence generation

BACKGROUND

Real-world evidence (RWE) plays a key role in regulatory and healthcare decision-making, but the potentially fragmentated nature of generated evidence may limit its utility for clinical decision-making. Heterogeneity and a lack of reproducibility in RWE resulting from inconsistent application of methodologies across data sources should be minimized through harmonization.

METHODS

This paper's aim is to describe and reflect upon a multidisciplinary research platform (FOUNTAIN; FinerenOne mUlti-database NeTwork for evidence generAtIoN) with coordinated studies using diverse RWE generation approaches and explore the platform's strengths and limitations. With guidance from an executive advisory committee of multidisciplinary experts and patient representatives, the goal of the FOUNTAIN platform is to harmonize RWE generation across a portfolio of research projects, including research partner collaborations and a common data model (CDM)-based program. FOUNTAIN's overarching objectives as a research platform are to establish long-term collaborations among pharmacoepidemiology research partners and experts and to integrate diverse approaches for RWE generation, including global protocol execution by research partners in local data sources and common protocol execution in multiple data sources through federated data networks, while ensuring harmonization of medical definitions, methodology, and reproducible artifacts across all studies. Specifically, the aim of the multiple studies run within the frame of FOUNTAIN is to provide insight into the real-world utilization, effectiveness, and safety of finerenone across its life-cycle.

RESULTS

Currently, the FOUNTAIN platform includes 9 research partner collaborations and 8 CDM-mapped data sources from 7 countries (United States, United Kingdom, China, Japan, The Netherlands, Spain, and Denmark). These databases and research partners were selected after a feasibility fit-for-purpose evaluation. Six multicountry, multidatabase, cohort studies are ongoing to describe patient populations, current standard of care, comorbidity profiles, healthcare resource use, and treatment effectiveness and safety in different patient populations with chronic kidney disease and type 2 diabetes. Strengths and potential limitations of FOUNTAIN are described in the context of valid RWE generation.

CONCLUSION

The establishment of the FOUNTAIN platform has allowed harmonized execution of multiple studies, promoting consistency both within individual studies that employ multiple data sources and across all studies run within the platform's framework. FOUNTAIN presents a proposal to efficiently improve the consistency and generalizability of RWE on finerenone.

Exploring the discrepancies between clinical trials and real-world data: A small-cell lung cancer study

The potential of real-world data to inform clinical trial design and supplement control arms has gained much interest in recent years. The most common approach relies on reproducing control arm outcomes by matching real-world patient cohorts to clinical trial baseline populations. However, recent studies pointed out that there is a lack of replicability, generalisability, and consensus. In this article, we propose a novel approach that aims to explore and examine these discrepancies by concomitantly investigating the impact of selection criteria and operations on the measurements of outcomes from the patient data. We tested the approach on a dataset consisting of small-cell lung cancer patients receiving platinum-based chemotherapy regimens from a real-world data cohort (n = 223) and six clinical trial control arms (n = 1224). The results showed that the discrepancy between real-world and clinical trial data potentially depends on differences in both patient populations and operational conditions (e.g., frequency of assessments, and censoring), for which further investigation is required. Discovering and accounting for confounders, including hidden effects of differences in operations related to the treatment process and clinical trial study protocol, would potentially allow for improved translation between clinical trials and real-world data. Continued development of the method presented here to systematically explore and account for these differences could pave the way for transferring learning across clinical studies and developing mutual translation between the real-world and clinical trials to inform clinical study design.

App Engagement as a Predictor of Weight Loss in Blended-Care Interventions: Retrospective Observational Study Using Large-Scale Real-World Data

BACKGROUND

Early weight loss is an established predictor for treatment outcomes in weight management interventions for people with obesity. However, there is a paucity of additional, reliable, and clinically actionable early predictors in weight management interventions. Novel blended-care weight management interventions combine coach and app support and afford new means of structured, continuous data collection, informing research on treatment adherence and outcome prediction.

OBJECTIVE

Against this backdrop, this study analyzes app engagement as a predictor for weight loss in large-scale, real-world, blended-care interventions. We hypothesize that patients who engage more frequently in app usage in blended-care treatment (eg, higher logging activity) lose more weight than patients who engage comparably less frequently at 3 and 6 months of intervention.

METHODS

Real-world data from 19,211 patients in obesity treatment were analyzed retrospectively. Patients were treated with 3 different blended-care weight management interventions, offered in Switzerland, the United Kingdom, and Germany by a digital behavior change provider. The principal component analysis identified an overarching metric for app engagement based on app usage. A median split informed a distinction in higher and lower engagers among the patients. Both groups were matched through optimal propensity score matching for relevant characteristics (eg, gender, age, and start weight). A linear regression model, combining patient characteristics and app-derived data, was applied to identify predictors for weight loss outcomes.

RESULTS

For the entire sample (N=19,211), mean weight loss was -3.24% (SD 4.58%) at 3 months and -5.22% (SD 6.29%) at 6 months. Across countries, higher app engagement yielded more weight loss than lower engagement after 3 but not after 6 months of intervention (P3 months<.001 and P6 months=.59). Early app engagement within the first 3 months predicted percentage weight loss in Switzerland and Germany, but not in the United Kingdom (PSwitzerland<.001, PUnited Kingdom=.12, and PGermany=.005). Higher age was associated with stronger weight loss in the 3-month period (PSwitzerland=.001, PUnited Kingdom=.002, and PGermany<.001) and, for Germany, also in the 6-month period (PSwitzerland=.09, PUnited Kingdom=.46, and PGermany=.03). In Switzerland, higher numbers of patients' messages to coaches were associated with higher weight loss (P3 months<.001 and P6 months<.001). Messages from coaches were not significantly associated with weight loss (all P>.05).

CONCLUSIONS

Early app engagement is a predictor of weight loss, with higher engagement yielding more weight loss than lower engagement in this analysis. This new predictor lends itself to automated monitoring and as a digital indicator for needed or adapted clinical action. Further research needs to establish the reliability of early app engagement as a predictor for treatment adherence and outcomes. In general, the obtained results testify to the potential of app-derived data to inform clinical monitoring practices and intervention design.

Real-World Evidence to Reinforce Clinical Trial Evidence in Health Technology Assessment: A Critical Review of Real-World Evidence Requirements from Seven Countries and Recommendations to Improve Acceptance

BACKGROUND

Real-world evidence (RWE) can reinforce clinical trial evidence in health technology assessment (HTA).

OBJECTIVES

Review HTA bodies' (HTAbs) requirements for RWE, real uses, and acceptance across seven countries (Brazil, Canada, France, Germany, Italy, Spain, and the United Kingdom) and outline recommendations that may improve acceptance of RWE in efficacy/effectiveness assessments and appraisals processes.

METHODS

RWE requirements were summarized based on HTAbs' guidelines. Acceptance by HTAbs was evaluated based on industry experience and case studies.

RESULTS

As of June 2022, RWE methodological guidelines were in place in three of the seven countries. HTAbs typically requested analyses based on local data sources, but the preferred study design and data sources differed. HTAbs had individual submission, assessment, and appraisal processes; some allowed early meetings for the protocol and/or results validation, though few involved external experts or medical societies to provide input to assessment and appraisal. The extent of submission, assessment, and appraisal requirements did not necessarily reflect the degree of acceptance.

CONCLUSION

All the countries reviewed face common challenges regarding the use of RWE. Our proposals address the need to facilitate collaboration and communication with industry and regulatory agencies and the need for specific guidelines describing RWE design and criteria of acceptance throughout the assessment and appraisal processes.

Value of Real-World Evidence for Treatment Selection: A Case Study in Colon Cancer

PURPOSE

Real-world evidence (RWE)-derived from analysis of real-world data (RWD)-has the potential to guide personalized treatment decisions. However, because of potential confounding, generating valid RWE is challenging. This study demonstrates how to responsibly generate RWE for treatment decisions. We validate our approach by demonstrating that we can uncover an existing adjuvant chemotherapy (ACT) guideline for stage II and III colon cancer (CC)-which came about using both data from randomized controlled trials and expert consensus-solely using RWD.

METHODS

Data from the population-based Netherlands Cancer Registry from a total of 27,056 patients with stage II and III CC who underwent curative surgery were analyzed to estimate the overall survival (OS) benefit of ACT. Focusing on 5-year OS, the benefit of ACT was estimated for each patient using G-computation methods by adjusting for patient and tumor characteristics and estimated propensity score. Subsequently, on the basis of these estimates, an ACT decision tree was constructed.

RESULTS

The constructed decision tree corresponds to the current Dutch guideline: patients with stage III or stage II with T stage 4 should receive surgery and ACT, whereas patients with stage II with T stage 3 should only receive surgery. Interestingly, we do not find sufficient RWE to conclude against ACT for stage II with T stage 4 and microsatellite instability-high (MSI-H), a recent addition to the current guideline.

CONCLUSION

RWE, if used carefully, can provide a valuable addition to our construction of evidence on clinical decision making and therefore ultimately affect treatment guidelines. Next to validating the ACT decisions advised in the current Dutch guideline, this paper suggests additional attention should be paid to MSI-H in future iterations of the guideline.

Describing diversity of real world data sources in pharmacoepidemiologic studies: The DIVERSE scoping review

PURPOSE

Real-world evidence (RWE) is increasingly used for medical regulatory decisions, yet concerns persist regarding its reproducibility and hence validity. This study addresses reproducibility challenges associated with diversity across real-world data sources (RWDS) repurposed for secondary use in pharmacoepidemiologic studies. Our aims were to identify, describe and characterize practices, recommendations and tools for collecting and reporting diversity across RWDSs, and explore how leveraging diversity could improve the quality of evidence.

METHODS

In a preliminary phase, keywords for a literature search and selection tool were designed using a set of documents considered to be key by the coauthors. Next, a systematic search was conducted up to December 2021. The resulting documents were screened based on titles and abstracts, then based on full texts using the selection tool. Selected documents were reviewed to extract information on topics related to collecting and reporting RWDS diversity. A content analysis of the topics identified explicit and latent themes.

RESULTS

Across the 91 selected documents, 12 topics were identified: 9 dimensions used to describe RWDS (organization accessing the data source, data originator, prompt, inclusion of population, content, data dictionary, time span, healthcare system and culture, and data quality), tools to summarize such dimensions, challenges, and opportunities arising from diversity. Thirty-six themes were identified within the dimensions. Opportunities arising from data diversity included multiple imputation and standardization.

CONCLUSIONS

The dimensions identified across a large number of publications lay the foundation for formal guidance on reporting diversity of data sources to facilitate interpretation and enhance replicability and validity of RWE.

Markov modeling for cost-effectiveness using federated health data network

OBJECTIVE

To introduce 2 R-packages that facilitate conducting health economics research on OMOP-based data networks, aiming to standardize and improve the reproducibility, transparency, and transferability of health economic models.

MATERIALS AND METHODS

We developed the software tools and demonstrated their utility by replicating a UK-based heart failure data analysis across 5 different international databases from Estonia, Spain, Serbia, and the United States.

RESULTS

We examined treatment trajectories of 47 163 patients. The overall incremental cost-effectiveness ratio (ICER) for telemonitoring relative to standard of care was 57 472 €/QALY. Country-specific ICERs were 60 312 €/QALY in Estonia, 58 096 €/QALY in Spain, 40 372 €/QALY in Serbia, and 90 893 €/QALY in the US, which surpassed the established willingness-to-pay thresholds.

DISCUSSION

Currently, the cost-effectiveness analysis lacks standard tools, is performed in ad-hoc manner, and relies heavily on published information that might not be specific for local circumstances. Published results often exhibit a narrow focus, central to a single site, and provide only partial decision criteria, limiting their generalizability and comprehensive utility.

CONCLUSION

We created 2 R-packages to pioneer cost-effectiveness analysis in OMOP CDM data networks. The first manages state definitions and database interaction, while the second focuses on Markov model learning and profile synthesis. We demonstrated their utility in a multisite heart failure study, comparing telemonitoring and standard care, finding telemonitoring not cost-effective.

Reliability and Agreement of Free Web-Based 3D Software for Computing Facial Area and Volume Measurements

Background: Facial surgeries require meticulous planning and outcome assessments, where facial analysis plays a critical role. This study introduces a new approach by utilizing three-dimensional (3D) imaging techniques, which are known for their ability to measure facial areas and volumes accurately. The purpose of this study is to introduce and evaluate a free web-based software application designed to take area and volume measurements on 3D models of patient faces. Methods: This study employed the online facial analysis software to conduct ten measurements on 3D models of subjects, including five measurements of area and five measurements of volume. These measurements were then compared with those obtained from the established 3D modeling software called Blender (version 3.2) using the Bland–Altman plot. To ensure accuracy, the intra-rater and inter-rater reliabilities of the web-based software were evaluated using the Intraclass Correlation Coefficient (ICC) method. Additionally, statistical assumptions such as normality and homoscedasticity were rigorously verified before analysis. Results: This study found that the web-based facial analysis software showed high agreement with the 3D software Blender within 95% confidence limits. Moreover, the online application demonstrated excellent intra-rater and inter-rater reliability in most analyses, as indicated by the ICC test. Conclusion: The findings suggest that the free online 3D software is reliable for facial analysis, particularly in measuring areas and volumes. This indicates its potential utility in enhancing surgical planning and evaluation in facial surgeries. This study underscores the software’s capability to improve surgical outcomes by integrating precise area and volume measurements into facial surgery planning and assessment processes.

Prevalence of Pretreatment Testing Recommended for Patients With Chronic Inflammatory Skin Diseases

Importance

Laboratory testing for the presence of tuberculosis, hepatitis, and other conditions before starting most systemic immunomodulatory agents is recommended in patients with chronic inflammatory skin diseases (CISD) but current testing patterns in the US are unclear.

Objective

To determine the prevalence of pretreatment testing that is recommended for patients with CISD (psoriasis, hidradenitis suppurativa, or atopic dermatitis).

Design, Setting, and Participants

This descriptive analysis of US commercial insurance claims databases from December 31, 2002, to December 31, 2020, included adult patients with CISD (psoriasis, hidradenitis suppurativa, or atopic dermatitis) who started an immunomodulatory agent, including methotrexate, tumor necrosis factor α inhibitors, interleukin (IL)-17Ai, ustekinumab, IL-23i, dupilumab, or apremilast.

Main Outcomes and Measures

The proportion of patients who underwent the screening tests as suggested by professional societies-including for tuberculosis, hepatitis, and liver function; complete blood cell counts; and lipid panels-were determined within 6 months before and during 2 years after treatment start.

Results

A total of 122 308 patients with CISDs (median [IQR] age, 49 [38-58] years; 63 663 [52.1%] male) starting systemic immunomodulatory treatment in the US were included. Treatment for patients with CISDs comprised methotrexate (28 684), tumor necrosis factor α inhibitors (40 965), ustekinumab (12 841), IL-23i (6116), IL-17Ai (9799), dupilumab (7787), or apremilast (16 116). Complete blood cell count was the most common test, performed in 41% (3161/7787) to 69% (19 659/28 684) of individuals before initiation across treatments. Between 11% (889/7787) and 59% (3613/6116) of patients had tuberculosis screening within 6 months before treatment, and 3% (149/4577) to 26% (1559/6097) had updated tests 1 year later. Between 13% (1006/7787) and 41% (16 728/40 965) had hepatitis panels before treatment. Low pretreatment testing levels before apremilast (15% [2331/16 116] to 45% [7253/16 116]) persisted a year into treatment (9% [816/8496] to 36% [2999/8496]) and were similar to dupilumab (11% [850/7787] to 41% [3161/7787] vs 3% [149/4577] to 25% [1160/4577]).

Conclusions and Relevance

In this descriptive analysis of patients with CISDs starting systemic immunomodulatory treatment in the US, less than 60% received the recommended pretreatment testing. Additional research is required to understand whether variations in testing affect patient outcomes.

Sodium-Glucose Cotransporter 2 Inhibitors and Nephrolithiasis Risk in Patients With Type 2 Diabetes

Importance

Type 2 diabetes (T2D) is associated with an increased risk of kidney stones. Sodium-glucose cotransporter 2 inhibitors (SGLT2is) might lower the risk of nephrolithiasis by altering urine composition. However, no studies have investigated the association between SGLT2i use and nephrolithiasis risk in patients receiving routine care in the US.

Objective

To investigate the association between SGLT2i use and nephrolithiasis risk in clinical practice.

Design, Setting, and Participants

This new-user, active comparator cohort study used data from commercially insured adults (aged ≥18 years) with T2D who initiated treatment with SGLT2is, glucagon-like peptide 1 receptor agonists (GLP-1RAs), or dipeptidyl peptidase 4 inhibitors (DPP4is) between April 1, 2013, and December 31, 2020. The data were analyzed from July 2021 through June 2023.

Exposure

New initiation of an SGLT2i, GLP-1RA, or DPP4i.

Main Outcomes and Measures

The primary outcome was nephrolithiasis diagnosed by International Classification of Diseases codes in the inpatient or outpatient setting. New SGLT2i users were 1:1 propensity score matched to new users of a GLP-1RA or DPP4i in pairwise comparisons. Incidence rates, rate differences (RDs), and estimated hazard ratios (HRs) with 95% CIs were calculated.

Results

After 1:1 propensity score matching, a total of 716 406 adults with T2D (358 203 pairs) initiating an SGLT2i or a GLP-1RA (mean [SD] age, 61.4 [9.7] years for both groups; 51.4% vs 51.2% female; 48.6% vs 48.5% male) and 662 056 adults (331 028 pairs) initiating an SGLT2i or a DPP4i (mean [SD] age, 61.8 [9.3] vs 61.7 [10.1] years; 47.4% vs 47.3% female; 52.6% vs 52.7% male) were included. Over a median follow-up of 192 (IQR, 88-409) days, the risk of nephrolithiasis was lower in patients initiating an SGLT2i than among those initiating a GLP-1RA (14.9 vs 21.3 events per 1000 person-years; HR, 0.69 [95% CI, 0.67-0.72]; RD, -6.4 [95% CI, -7.1 to -5.7]) or a DPP4i (14.6 vs 19.9 events per 1000 person-years; HR, 0.74 [95% CI, 0.71-0.77]; RD, -5.3 [95% CI, -6.0 to -4.6]). The association between SGLT2i use and nephrolithiasis risk was similar by sex, race and ethnicity, history of chronic kidney disease, and obesity. The magnitude of the risk reduction with SGLT2i use was larger among adults aged younger than 70 years vs aged 70 years or older (HR, 0.85 [95% CI, 0.79-0.91]; RD, -3.46 [95% CI, -4.87 to -2.05] per 1000 person-years; P for interaction <.001).

Conclusions and Relevance

These findings suggest that in adults with T2D, SGLT2i use may lower the risk of nephrolithiasis compared with GLP-1RAs or DPP4is and could help to inform decision-making when prescribing glucose-lowering agents for patients who may be at risk for developing nephrolithiasis.

Cardiorenal effectiveness of empagliflozin vs. glucagon-like peptide-1 receptor agonists: final-year results from the EMPRISE study

BACKGROUND

No randomized clinical trials have directly compared the cardiorenal effectiveness of empagliflozin and GLP-1RA agents with demonstrated cardioprotective effects in patients with a broad spectrum of cardiovascular risk. We reported the final-year results of the EMPRISE study, a monitoring program designed to evaluate the cardiorenal effectiveness of empagliflozin across broad patient subgroups.

METHODS

We identified patients ≥ 18 years old with type 2 diabetes who initiated empagliflozin or GLP-1RA from 2014 to 2019 using US Medicare and commercial claims databases. After 1:1 propensity score matching using 143 baseline characteristics, we evaluated risks of outcomes including myocardial infarction (MI) or stroke, hospitalization for heart failure (HHF), major adverse cardiovascular events (MACE - MI, stroke, or cardiovascular mortality), a composite of HHF or cardiovascular mortality, and progression to end-stage kidney disease (ESKD) (in patients with chronic kidney disease stages 3-4). We estimated hazard ratios (HR) and rate differences (RD) per 1,000 person-years, overall and within subgroups of age, sex, baseline atherosclerotic cardiovascular disease (ASCVD), and heart failure (HF).

RESULTS

We identified 141,541 matched pairs. Compared with GLP-1RA, empagliflozin was associated with similar risks of MI or stroke [HR: 0.99 (0.92, 1.07); RD: -0.23 (-1.25, 0.79)], and lower risks of HHF [HR: 0.50 (0.44, 0.56); RD: -2.28 (-2.98, -1.59)], MACE [HR: 0.90 (0.82, 0.99); RD: -2.54 (-4.76, -0.32)], cardiovascular mortality or HHF [HR: 0.77 (0.69, 0.86); RD: -4.11 (-5.95, -2.29)], and ESKD [0.75 (0.60, 0.94); RD: -6.77 (-11.97, -1.61)]. Absolute risk reductions were larger in older patients and in those with baseline ASCVD/HF. They did not differ by sex.

CONCLUSIONS

The cardiovascular benefits of empagliflozin vs. cardioprotective GLP-1RA agents were larger in older patients and in patients with history of ASCVD or HF, while they did not differ by sex. In patients with advanced CKD, empagliflozin was associated with risk reductions of progression to ESKD.

Implementation of a Taxonomy-Based Framework for the Selection of Appropriate Drugs and Outcomes for Real-World Data Signal Detection Studies

INTRODUCTION

For signal detection studies investigating either drug safety or method evaluation, the choice of drug-outcome pairs needs to be tailored to the planned study design and vice versa. While this is well understood in hypothesis-testing epidemiology, it should be as important in signal detection, but this has not widely been considered. There is a need for a taxonomy framework to provide guidance and a systematic reproducible approach to the selection of appropriate drugs and outcomes for signal detection studies either investigating drug safety or assessing method performance using real-world data.

OBJECTIVE

The aim was to design a general framework for the selection of appropriate drugs and outcomes for signal detection studies given a study design of interest. As a motivating example, we illustrate how the framework is applied to build a reference set for a study aiming to assess the performance of the self-controlled case series with active comparators.

METHODS

We reviewed criteria presented in two published studies which aimed to provide practical advice for choosing the appropriate signal evaluation methodology, and assessed their relevance for signal detection. Further characteristics specific to signal detection were added. The final framework is based on: the application of study design requirements, the database(s) of interest, and the clinical importance of the drug(s) and outcome(s) under consideration. This structure was applied by selecting drug-outcome pairs as a reference set (i.e. list of drug-outcome pairs classified as positive or negative controls) for which the method is expected to work well for a signal detection study aiming to assess the performance of self-controlled case series. Eight criteria were used, related to the application of self-controlled case series assumptions, choice of active comparators, coverage in the database of interest and clinical importance of the outcomes.

RESULTS

After application of the framework, two classes of antibiotics (seven drugs) were selected for the study, and 28 outcomes from all organ classes were chosen from the drug labels, out of the 273 investigated. In total, this corresponds to 104 positive controls (drug-outcome pairs) and 58 negative controls.

CONCLUSIONS

We proposed and applied a framework for the selection of drugs and outcomes for both drug safety signal detection and method assessment used in signal detection to optimise their performance given a study design. This framework will eliminate part of the bias relating to drugs and outcomes not being suited to the method or database. The main difficulty lies in the choice of the criteria and their application to ensure systematic selection, especially as some information remains unknown in signal detection, and clinical judgement was needed on occasions. The same framework could be adapted for other methods.

Quality of systematic reviews with meta-analyses of resveratrol: A methodological systematic review

Recently, several meta-analyses (MAs) have focused on the health effects of resveratrol. However, the methodological and reporting quality of these MAs has not yet been fully evaluated so far. Therefore, the present study evaluated the quality of these MAs through a methodological systematic review. Systematic searches were conducted in PubMed, Embase, Web of Science, and Cochrane Library from inception until May 20, 2022, and PubMed was used to update the search until September 6, 2023. The methodological and reporting quality of the selected MAs was evaluated using AMSTAR-2 and PRISMA 2009. Fifty-one MAs published during 2013-2023 were included. In each review, the number of primary studies ranged from 3 to 37, and the number of participants ranged from 50 to 2114. Among the first-listed primary outcomes, only 23 (45.10%) were "positive." As for the methodological quality, most MAs (44, 86.27%) on resveratrol were rated critically low. Inadequate reporting of the included MAs mainly involved items 2 ("Structured summary"), 5 ("Protocol and registration"), 8 ("Search"), 9 ("Study selection"), 10 ("Data collection process"), 12 ("Risk of bias in individual studies"), and 24 ("Summary of evidence") based on the PRISMA 2009. Additionally, journal's impact factor, number of authors, and funding support were positively associated with the overall methodological quality but were not statistically significant (p > 0.05). Future MAs on resveratrol require better design, implementation, and reporting by following the Cochrane Handbook, AMSTAR-2, and PRISMA.

Stratified analysis in comparative effectiveness studies that emulate randomized trials

PURPOSE

For observational cohort studies that employ matching by propensity scores (PS), preliminary stratification by consequential predictors of outcome better emulates stratified randomization and potentially reduces variance and bias through relaxed dependence on modeling assumptions. We assessed the impact of pre-stratification in two real-life examples. For both, prior evidence from placebo-controlled randomized clinical trials (RCTs) suggested small or no risk reduction, but observational analysis suggested protection, presumably the result of confounding bias.

STUDY DESIGN AND SETTING

The study populations consisted of Medicare beneficiaries (2014-18) with type 2 diabetes initiating either (i) empagliflozin versus dipeptidyl peptidase-4 inhibitors (DPP-4i) or (ii) empagliflozin versus glucagon-like peptide-1 receptor agonists (GLP-1RA). The outcome was myocardial infarction or stroke. We estimated hazard ratios (HR) and rate differences (RD) after controlling for 143 pre-exposure covariates via 1:1 PS matching after (1) PS estimation in the total cohort (total-cohort PS-matching) and (2) PS estimation separately by baseline cardiovascular disease (stratified PS matching).

RESULTS

Stratified PS matching resulted in HRs that exceeded those from total-cohort PS-matching by 13% and 9%, respectively, for the comparisons of empagliflozin to DPP-4i and GLP-1RA. Against both comparators, HRs and RDs after stratified PS matching were closer to the null, with slightly higher variances (2%-3%) than those after total-cohort PS matching.

CONCLUSION

Stratified PS matching produced effect estimates closer to the expected trial findings than total-cohort PS matching. The price paid in increased variance was minimal.

High-Dose vs. Standard-Dose Influenza Vaccine and Cardiopulmonary Hospitalization or Mortality: Emulating the INVESTED Trial Using Insurance Claims Data

The INVESTED trial did not show benefits of high-dose (HD) vaccine vs. standard-dose (SD) for a primary composite outcome of cardiopulmonary hospitalization or all-cause mortality (hazard ratio (HR) = 1.05, 95% confidence interval (CI) = 0.96-1.15) and its components (all-cause mortality HR = 1.01, 95% CI = 0.84-1.21, cardiopulmonary hospitalization HR = 1.05, 95% CI = 0.96-1.16) during three influenza seasons (2016-2019) among participants with recent myocardial infarction or hospitalization for heart failure (HHF). We emulated INVESTED using Medicare claims data to assess whether the real-world evidence (RWE) study reached similar conclusions. We identified 1:1 propensity score (PS)-matched trial-eligible Medicare beneficiaries aged > 65 years and with prior HHF who received an HD or SD vaccine for the 2016-2019 seasons. We also re-analyzed the INVESTED trial data restricting to participants > 65 years with prior HHF to align eligibility criteria more closely with the RWE study. We compared HRs from the trial and RWE study for the main outcomes. Among 53,393 pairs of PS-matched Medicare beneficiaries, the HD vaccine group showed lower risk of the primary composite outcome (HR = 0.96, 95% CI = 0.95-0.98) and all-cause mortality (HR = 0.93, 95% CI = 0.91-0.95), and similar risk of cardiopulmonary hospitalization (HR = 0.98, 95% CI = 0.96-1.00), compared with SD. The RWE and trial results were closely concordant after the trial population was limited to participants > 65 years with prior HHF: trial-based results for the primary composite outcome (HR = 1.02, 95% CI = 0.89-1.17), all-cause mortality (HR = 0.92, 95% CI = 0.72-1.16), and cardiopulmonary hospitalization (HR = 1.02, 95% CI = 0.88-1.18). Although similar to the main trial results, the RWE was closer to the results from trial participants with aligned eligibility criteria. This study affirms the importance of considering different distributions of baseline patient characteristics when comparing trial findings to RWE.

Replicability of simulation studies for the investigation of statistical methods: the RepliSims project

Results of simulation studies evaluating the performance of statistical methods can have a major impact on the way empirical research is implemented. However, so far there is limited evidence of the replicability of simulation studies. Eight highly cited statistical simulation studies were selected, and their replicability was assessed by teams of replicators with formal training in quantitative methodology. The teams used information in the original publications to write simulation code with the aim of replicating the results. The primary outcome was to determine the feasibility of replicability based on reported information in the original publications and supplementary materials. Replicasility varied greatly: some original studies provided detailed information leading to almost perfect replication of results, whereas other studies did not provide enough information to implement any of the reported simulations. Factors facilitating replication included availability of code, detailed reporting or visualization of data-generating procedures and methods, and replicator expertise. Replicability of statistical simulation studies was mainly impeded by lack of information and sustainability of information sources. We encourage researchers publishing simulation studies to transparently report all relevant implementation details either in the research paper itself or in easily accessible supplementary material and to make their simulation code publicly available using permanent links.

Utilization of anonymization techniques to create an external control arm for clinical trial data

BACKGROUND

Subject-level real-world data (RWD) collected during daily healthcare practices are increasingly used in medical research to assess questions that cannot be addressed in the context of a randomized controlled trial (RCT). A novel application of RWD arises from the need to create external control arms (ECAs) for single-arm RCTs. In the analysis of ECAs against RCT data, there is an evident need to manage and analyze RCT data and RWD in the same technical environment. In the Nordic countries, legal requirements may require that the original subject-level data be anonymized, i.e., modified so that the risk to identify any individual is minimal. The aim of this study was to conduct initial exploration on how well pseudonymized and anonymized RWD perform in the creation of an ECA for an RCT.

METHODS

This was a hybrid observational cohort study using clinical data from the control arm of the completed randomized phase II clinical trial (PACIFIC-AF) and RWD cohort from Finnish healthcare data sources. The initial pseudonymized RWD were anonymized within the (k, ε)-anonymity framework (a model for protecting individuals against identification). Propensity score matching and weighting methods were applied to the anonymized and pseudonymized RWD, to balance potential confounders against the RCT data. Descriptive statistics for the potential confounders and overall survival analyses were conducted prior to and after matching and weighting, using both the pseudonymized and anonymized RWD sets.

RESULTS

Anonymization affected the baseline characteristics of potential confounders only marginally. The greatest difference was in the prevalence of chronic obstructive pulmonary disease (4.6% vs. 5.4% in the pseudonymized compared to the anonymized data, respectively). Moreover, the overall survival changed in anonymization by only 8% (95% CI 4-22%). Both the pseudonymized and anonymized RWD were able to produce matched ECAs for the RCT data. Anonymization after matching impacted overall survival analysis by 22% (95% CI -21-87%).

CONCLUSIONS

Anonymization may be a viable technique for cases where flexible data transfer and sharing are required. As anonymization necessarily affects some aspects of the original data, further research and careful consideration of anonymization strategies are needed.

Concepts of Designing and Implementing Pharmacoepidemiology Studies on the Safety of Systemic Treatments in Dermatology Practice

The U.S. Food and Drug Administration and clinical guidelines use evidence from pharmacoepidemiology studies to inform prescribing decisions and fill evidence gaps left by randomized controlled trials (RCTs). The long-term safety and infrequent adverse reactions are not well-understood when RCTs are short and involve few patients, as is the case for most systemic immunomodulating drugs in dermatology. A better understanding of the design and implementation of pharmacoepidemiology studies will help practitioners assess the accuracy of etiologic findings and use them with confidence in clinical practice. Conducting pharmacoepidemiology studies follows a structured approach, which we discuss in this article: (i) a design layer connects the research question with the appropriate study design, and considering which hypothetical RCT one ideally would want to conduct reduces inadvertent investigator errors; (ii) a measurement layer transforms longitudinal patient-level data into variables that identify the study population, patient characteristics, treatment, and outcomes; and (iii) the analysis focuses on the causal treatment effect estimation. The review and interpretation of pharmacoepidemiology studies should consider issues beyond a typical review of RCTs, chiefly the lack of baseline randomization and the use of secondary data. Well-designed and well-conducted pharmacoepidemiologic studies complement dermatology practice with critical information on prescribing systemic medications.

Evidence-Based Surgery: What Can Intra-Operative Images Contribute?

Evidence-based medicine integrates results from randomized controlled trials (RCTs) and meta-analyses, combining the best external evidence with individual clinical expertise and patients' preferences. However, RCTs of surgery differ from those of medicine in that surgical performance is often assumed to be consistent. Yet, evaluating whether each surgery is performed to the same standard is quite challenging. As a primary issue, the novelty of this review is to emphasize-with a focus on orthopedic trauma-the advantage of having complete intra-operative image documentation, allowing the direct evaluation of the quality of the intra-operative technical performance. The absence of complete intra-operative image documentation leads to the inhomogeneity of case series, yielding inconsistent results due to the impossibility of a secondary analysis. Thus, comparisons and the reproduction of studies are difficult. Access to complete intra-operative image data in surgical RCTs allows not only secondary analysis but also comparisons with similar cases. Such complete data can be included in electronic papers. Offering these data to peers-in an accessible link-when presenting papers facilitates the selection process and improves publications for readers. Additionally, having access to the full set of image data for all presented cases serves as a rich resource for learning. It enables the reader to sift through the information and pinpoint the details that are most relevant to their individual needs, allowing them to potentially incorporate this knowledge into daily practice. A broad use of the concept of complete intra-operative image documentation is pivotal for bridging the gap between clinical research findings and real-world applications. Enhancing the quality of surgical RCTs would facilitate the equalization of evidence acquisition in both internal medicine and surgery. Joint effort by surgeons, scientific societies, publishers, and healthcare authorities is needed to support the ideas, implement economic requirements, and overcome the mental obstacles to its realization.

Predicting Treatment Effects of a New-to-Market Drug in Clinical Practice Based on Phase III Randomized Trial Results

Trial results may not be generalizable to target populations treated in clinical practice with different distributions of baseline characteristics that modify the treatment effect. We used outcome models developed with trial data to predict treatment effects in Medicare populations. We used data from the Randomized Evaluation of Long-Term Anticoagulation Therapy trial (RE-LY), which investigated the effect of dabigatran vs. warfarin on stroke or systemic embolism (stroke/SE) among patients with atrial fibrillation. We developed outcome models by fitting proportional hazards models in trial data. Target populations were trial-eligible Medicare beneficiaries who initiated dabigatran or warfarin in 2010-2011 ("early") and 2010-2017 ("extended"). We predicted 2-year risk ratios (RRs) and risk differences (RDs) for stroke/SE, major bleeding, and all-cause death in the Medicare populations using the observed baseline characteristics. The trial and early target populations had similar mean (SD) CHADS2 scores (2.15 (SD 1.13) vs. 2.15 (SD 0.91)) but different mean ages (71 vs. 79 years). Compared with RE-LY, the early Medicare population had similar predicted benefit of dabigatran vs. warfarin for stroke/SE (trial RR = 0.63, 95% confidence interval (CI) = 0.50 to 0.76 and RD = -1.37%, -1.96% to -0.77%, Medicare RR = 0.73, 0.65 to 0.82 and RD = -0.92%, -1.26% to -0.59%) and risks for major bleeding and all-cause death. The time-extended target population showed similar results. Outcome model-based prediction facilitates estimating the average treatment effects of a drug in different target populations when treatment and outcome data are unreliable or unavailable. The predicted effects may inform payers' coverage decisions for patients, especially shortly after a drug's launch when observational data are scarce.

Using public clinical trial reports to probe non-experimental causal inference methods

BACKGROUND

Non-experimental studies (also known as observational studies) are valuable for estimating the effects of various medical interventions, but are notoriously difficult to evaluate because the methods used in non-experimental studies require untestable assumptions. This lack of intrinsic verifiability makes it difficult both to compare different non-experimental study methods and to trust the results of any particular non-experimental study.

METHODS

We introduce TrialProbe, a data resource and statistical framework for the evaluation of non-experimental methods. We first collect a dataset of pseudo "ground truths" about the relative effects of drugs by using empirical Bayesian techniques to analyze adverse events recorded in public clinical trial reports. We then develop a framework for evaluating non-experimental methods against that ground truth by measuring concordance between the non-experimental effect estimates and the estimates derived from clinical trials. As a demonstration of our approach, we also perform an example methods evaluation between propensity score matching, inverse propensity score weighting, and an unadjusted approach on a large national insurance claims dataset.

RESULTS

From the 33,701 clinical trial records in our version of the ClinicalTrials.gov dataset, we are able to extract 12,967 unique drug/drug adverse event comparisons to form a ground truth set. During our corresponding methods evaluation, we are able to use that reference set to demonstrate that both propensity score matching and inverse propensity score weighting can produce estimates that have high concordance with clinical trial results and substantially outperform an unadjusted baseline.

CONCLUSIONS

We find that TrialProbe is an effective approach for probing non-experimental study methods, being able to generate large ground truth sets that are able to distinguish how well non-experimental methods perform in real world observational data.

Venous Thromboembolism in Patients With Bullous Pemphigoid

Importance

Studies have linked bullous pemphigoid (BP) with venous thromboembolism (VTE) across several data sources finding 6-fold to 15-fold increased incidence rates.

Objective

To determine the incidence of VTE in patients with BP compared with similar controls.

Design, Setting, and Participants

This cohort study used insurance claims data from a nationwide US health care database from January 1, 2004, through January 1, 2020. Patients with dermatologist-recorded BP were identified (≥2 diagnoses of BP [International Classification of Diseases, Ninth Revision (ICD-9) 694.5 and ICD-10 L12.0] recorded by dermatologists within 1 year). Risk-set sampling identified comparator patients without BP and free of other chronic inflammatory skin diseases. Patients were followed-up until the first of the following events occurred: VTE, death, disenrollment, or end of data stream.

Exposures

Patients with BP compared with those without BP and free of other chronic inflammatory skin diseases (CISD).

Main outcome

Venous thromboembolism events were identified and incidence rates were computed before and after propensity-score (PS) matching to account for VTE risk factors. Hazard ratios (HRs) compared the incidence of VTE in BP vs non-CISD.

Results

Overall, 2654 patients with BP and 26 814 comparator patients without BP or another CISD were identified. The mean (SD) age in the BP group was 73.0 (12.6) years and 55.0 (18.9) years in the non-CSID group. With a median follow-up time was 2 years, the unadjusted incidence rate (per 1000 person-years) of outpatient or inpatient VTE was 8.5 in the BP group compared with 1.8 in patients without a CISD. Adjusted rates were 6.7 in the BP group compared with 3.0 in the non-CISD group. Age-specific adjusted incidence rates (per 1000 person-years) in patients aged 50 to 74 years was 6.0 (vs 2.9 in the non-CISD group) and in those aged 75 years or older was 7.1 (vs 4.53 in the non-CISD group). After 1:1 propensity-score matching including 60 VTE risk factors and severity markers, BP was associated with a 2-fold increased risk of VTE (2.24 [1.26-3.98]) vs those in the non-CISD group. When restricting to patients aged 50 years or older, the adjusted relative risk of VTE was 1.82 (1.05-3.16) for the BP vs non-CISD groups.

Conclusions

In this nationwide US cohort study, BP was associated with a 2-fold increased incidence of VTE after controlling for VTE risk factors in a dermatology patient population.

Evaluation of reporting quality of cohort studies using real-world data based on RECORD: systematic review

OBJECTIVE

Real-world data (RWD) and real-world evidence (RWE) have been paid more and more attention in recent years. We aimed to evaluate the reporting quality of cohort studies using real-world data (RWD) published between 2013 and 2021 and analyze the possible factors.

METHODS

We conducted a comprehensive search in Medline and Embase through the OVID interface for cohort studies published from 2013 to 2021 on April 29, 2022. Studies aimed at comparing the effectiveness or safety of exposure factors in the real-world setting were included. The evaluation was based on the REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement. Agreement for inclusion and evaluation was calculated using Cohen's kappa. Pearson chi-square test or Fisher's exact test and Mann-Whitney U test were used to analyze the possible factors, including the release of RECORD, journal IFs, and article citations. Bonferroni's correction was conducted for multiple comparisons. Interrupted time series analysis was performed to display the changes in report quality over time.

RESULTS

187 articles were finally included. The mean ± SD of the percentage of adequately reported items in the 187 articles was 44.7 ± 14.3 with a range of 11.1-87%. Of 23 items, the adequate reporting rate of 10 items reached 50%, and the reporting rate of some vital items was inadequate. After Bonferroni's correction, the reporting of only one item significantly improved after the release of RECORD and there was no significant improvement in the overall report quality. For interrupted time series analysis, there were no significant changes in the slope (p = 0.42) and level (p = 0.12) of adequate reporting rate. The journal IFs and citations were respectively related to 2 areas and the former significantly higher in high-reporting quality articles.

CONCLUSION

The endorsement of the RECORD cheklist was generally inadequate in cohort studies using RWD and has not improved in recent years. We encourage researchers to endorse relevant guidelines when utilizing RWD for research.

The IHI Rochester Report 2022 on Healthcare Informatics Research: Resuming After the CoViD-19

In 2020, the CoViD-19 pandemic spread worldwide in an unexpected way and suddenly modified many life issues, including social habits, social relationships, teaching modalities, and more. Such changes were also observable in many different healthcare and medical contexts. Moreover, the CoViD-19 pandemic acted as a stress test for many research endeavors, and revealed some limitations, especially in contexts where research results had an immediate impact on the social and healthcare habits of millions of people. As a result, the research community is called to perform a deep analysis of the steps already taken, and to re-think steps for the near and far future to capitalize on the lessons learned due to the pandemic. In this direction, on June 09th-11th, 2022, a group of twelve healthcare informatics researchers met in Rochester, MN, USA. This meeting was initiated by the Institute for Healthcare Informatics-IHI, and hosted by the Mayo Clinic. The goal of the meeting was to discuss and propose a research agenda for biomedical and health informatics for the next decade, in light of the changes and the lessons learned from the CoViD-19 pandemic. This article reports the main topics discussed and the conclusions reached. The intended readers of this paper, besides the biomedical and health informatics research community, are all those stakeholders in academia, industry, and government, who could benefit from the new research findings in biomedical and health informatics research. Indeed, research directions and social and policy implications are the main focus of the research agenda we propose, according to three levels: the care of individuals, the healthcare system view, and the population view.

Strategies to Address Current Challenges in Real-World Evidence Generation in Japan

The generation of real-world evidence (RWE), which describes patient characteristics or treatment patterns using real-world data (RWD), is rapidly growing more popular as a tool for decision-making in Japan. The aim of this review was to summarize challenges to RWE generation in Japan related to pharmacoepidemiology, and to propose strategies to address some of these challenges. We first focused on data-related issues, including the lack of transparency of RWD sources, linkage across different care settings, definitions of clinical outcomes, and the overall assessment framework of RWD when used for research purposes. Next the study reviewed methodology-related challenges. As lack of design transparency impairs study reproducibility, transparent reporting of study design is critical for stakeholders. For this review, we considered different sources of biases and time-varying confounding, along with potential study design and methodological solutions. Additionally, the implementation of robust assessment of definition uncertainty, misclassification, and unmeasured confounders would enhance RWE credibility in light of RWD source-related limitations, and is being strongly considered by task forces in Japan. Overall, the development of guidance for best practices on data source selection, design transparency, and analytical methods to address different sources of biases and robustness in the process of RWE generation will enhance credibility for stakeholders and local decision-makers.

R WE ready for reimbursement? A round up of developments in real-world evidence relating to health technology assessment: part 11

In this latest update we highlight a study from the REPEAT initiative that evaluates the reproducibility of real-world data studies, the publication of the HARPER Protocol Template developed by a joint ISPE/ISPOR taskforce, and discuss recent US FDA guidance on external control arms.

An overview of methodological flaws of real-world studies investigating drug safety in the post-marketing setting

INTRODUCTION

The evaluation of the post-marketing safety profile of drugs is a continuous monitoring process for approved and marketed medicines and it is crucial for detecting new adverse drug reactions. As such, real-world studies are essential to complement pre-marketing evidence with information concerning drug risk-benefit profile and use in wider patient populations and they have a great potential to support post-marketing drug safety evaluations.

AREAS COVERED

A detailed description of the main limitations of real-world data sources (i.e. claims databases, electronic healthcare records, drug/disease registers and spontaneous reporting system databases) and of the main methodological challenges of real-world studies in generating real-world evidence is provided.

EXPERT OPINION

Real-world evidence biases can be ascribed to both the methodological approach and the specific limitations of the different real-world data sources used to carry out the study. As such, it is crucial to characterize the quality of real-world data, by establishing guidelines and best practices for the assessment of data fitness for purpose. On the other hand, it is important that real-world studies are conducted using a rigorous methodology, aimed at minimizing the risk of bias.