Developments in post-marketing comparative effectiveness research

A Scalable Approach to Assess the Safety of Recently Marketed Systemic Treatments for Atopic Dermatitis in Clinical Practice: First Analysis Cycle of a Sequential Monitoring System

Targeted systemic immune-modulating drugs to treat atopic dermatitis were highly efficacious in randomized trials. Trials with limited number of subjects leave questions about their safety. We describe a data and analytics structure for the production of timely, high-quality evidence on the comparative safety of recently approved immune-modulating drugs in patients with atopic dermatitis in clinical practice. We established a series of sequential propensity score-balanced cohorts that grow in size with each annual data refresh. Nine health outcomes of interest plus conjunctivitis as a positive tracer outcome were identified. The initial treatment comparison was dupilumab, an IL-4/13 inhibitor, or tralokinumab, an IL-13 inhibitor, versus abrocitinib/upadacitinib, both Jak inhibitors. The first analysis cycle (December 2021-February 2023) compared 269 patients initiating Jak inhibitors and 2650 initiating IL-4/IL-13 inhibitors. Patient characteristics were well-balanced after propensity score matching. Outpatient infections within 180 days occurred in 18% of Jak1 inhibitor initiators versus 12% of dupilumab/tralokinumab initiators (risk ratio = 1.50, 95% confidence interval = 0.96-2.33), whereas acne risks were 7 versus 3%, respectively (risk ratio = 2.29, 95% confidence interval = 0.96-5.46). This sequential monitoring system will produce essential knowledge on the safety of immune-modulating drugs to treat atopic dermatitis on the basis of its growing study size of patients observed in clinical practice.

Ranking the Importance of Prognostic Factors for Relapsed/Refractory Multiple Myeloma: International Physician Panel Consensus Following a Systematic Literature Review

PURPOSE

The increasing use of single-arm and nonrandomized trial designs in oncology aims to expedite patient access to novel treatments. To contextualize their results using strategies such as external control arms or indirect treatment comparisons, prespecification of, and subsequent adjustment for, prognostic factors is required to ensure comparability of populations and avoid bias. This study aimed to systematically identify and rank prognostic factors relevant to treatment outcomes in patients with relapsed/refractory multiple myeloma (RRMM).

MATERIALS AND METHODS

To comprehensively identify prognostic factors, a systematic literature review was conducted with databases searched between January 2016 and April 2022. Clinical studies enrolling adult patients with RRMM and assessing prognostic significance using adjusted analyses were included. Subsequently, an international panel of multiple myeloma experts confirmed and ranked these variables by their importance in predicting clinical outcomes. A structured series of expert consultations was conducted from November 2022 to February 2023, including 2 rounds of consensus meetings.

RESULTS

Of 125 studies included in the systematic literature review, 112 described 97 factors significantly associated with at least 1 outcome of interest. A total of 25 factors associated with overall survival and/or objective response and reported in at least 2 studies were included in the ranking process. The physician panel unanimously agreed on the 6 most important prognostic factors: cytogenetic risk, age, refractory status, disease stage, performance status, and extramedullary disease/plasmacytoma.

CONCLUSION

This list of RRMM prognostic factors can be used in comparative analyses to assess treatment effectiveness in the absence of head-to-head trials.

Trends in COPD severe exacerbations, and all-cause and respiratory mortality, before and after implementation of newer long-acting bronchodilators in a large population-based cohort

BACKGROUND

Little is known about the trends in morbidity and mortality at the population level that followed the introduction of newer once-daily long-acting bronchodilators for COPD. The purpose of the study was to evaluate whether the availability of new bronchodilators was associated with changes in the temporal trends in severe COPD exacerbations and mortality between 2007 and 2018 in the older population with COPD; and whether this association was homogeneous across sex and socioeconomic status classes.

METHODS

We used an interrupted time-series and three segments multivariate autoregressive models to evaluate the adjusted changes in slopes (i.e., trend effect) in monthly severe exacerbation and mortality rates after 03/2013 and 02/2015 compared to the tiotropium period (04/2007 to 02/2013). Cohorts of individuals > 65 years with COPD were created from the nationally representative database of the Quebec Integrated Chronic Disease Surveillance System in the province of Quebec, Canada. Whether these trends were similar for men and women and across different socioeconomic status classes was also assessed.

RESULTS

There were 130,750 hospitalizations for severe exacerbation and 104,460 deaths, including 24,457 (23.4%) respiratory-related deaths, over the study period (928,934 person-years). Significant changes in trends were seen after 03/2013 for all-cause mortality (-1.14%/month;95%CI -1.90% to -0.38%), which further decreased after 02/2015 (-1.78%/month;95%CI -2.70% to -0.38%). Decreases in respiratory-related mortality (-2.45%/month;95%CI -4.38% to -0.47%) and severe exacerbation (-1,90%/month;95%CI -3.04% to -0.75%) rates were only observed after 02/2015. These observations tended to be more pronounced in women than in men and in higher socioeconomic status groups (less deprived) than in lower socioeconomic status groups (more deprived).

CONCLUSIONS

The arrival of newer bronchodilators was chronologically associated with reduced trends in severe exacerbation, all-cause and respiratory-related mortality rates among people with COPD > 65 years. Our findings document population benefits on key patient-relevant outcomes in the years following the introduction of newer once-daily long-acting bronchodilators and their combinations, which were likely multifactorial. Public health efforts should focus on closing the gap between lower and higher socioeconomic status groups.

Using real-world evidence in haematology

Most new drug approvals are based on data from large randomized clinical trials (RCTs). However, there are sometimes contradictory conclusions from seemingly similar trials and generalizability of conclusions from these trials is limited. These considerations explain, in part, the gap between conclusions from data of RCTs and those from registries termed real world data (RWD). Recently, real-world evidence (RWE) from RWD processed by artificial intelligence has received increasing attention. We describe the potential of using RWD in haematology concluding RWE from RWD may complement data from RCTs to support regulatory decisions.

Evaluation of VTE, MACE, and Serious Infections Among Patients with RA Treated with Baricitinib Compared to TNFi: A Multi-Database Study of Patients in Routine Care Using Disease Registries and Claims Databases

INTRODUCTION

The aim of this work is to evaluate baricitinib safety with respect to venous thromboembolism (VTE), major adverse cardiovascular events (MACE), and serious infection relative to tumor necrosis factor inhibitors (TNFi) in patients with rheumatoid arthritis (RA).

METHODS

Patients with RA from 14 real-world data sources (three disease registries, eight commercial and three government health insurance claims databases) in the United States (n = 9), Europe (n = 3), and Japan (n = 2) were analyzed using a new user active comparator design. Propensity score matching (1:1) controlled for potential confounding. Meta-analysis of incidence rate ratios (IRR) and incidence rate differences (IRD) for each outcome, from each data source was executed using modified Poisson regression and Cochran-Mantel-Haenszel analysis.

RESULTS

Of 9013 eligible baricitinib-treated patients, 7606 were propensity score-matched with TNFi-treated patients, contributing 5879 and 6512 person-years of baricitinib and TNFi exposure, respectively. Across data sources, 97 patients (56 baricitinib) experienced VTE during follow-up, 93 experienced MACE (54 baricitinib), and 321 experienced serious infection (176 baricitinib). Overall IRRs comparing baricitinib with TNFi treatment were 1.51 (95% CI 1.10, 2.08) for VTE, 1.54 (95% CI 0.93, 2.54) for MACE, and 1.36 (95% CI 0.86, 2.13) for serious infection. IRDs for VTE, MACE, and serious infection, respectively, were 0.26 (95% CI -0.04, 0.57), 0.22 (95% CI -0.07, 0.52), and 0.57 (95% CI -0.07, 1.21) per 100 person-years greater for baricitinib than TNFi.

CONCLUSIONS

Overall results suggest increased risk of VTE with baricitinib versus TNFi, with consistent point estimates from the two largest data sources. A numerically greater risk was observed for MACE and serious infection when comparing baricitinib versus TNFi, with different point estimates from the two largest data sources. Findings from this study and their impact on clinical practice should be considered in context of limitations and other evidence regarding the safety and efficacy of baricitinib and other Janus kinase inhibitors.

TRIAL REGISTRATION

EU PAS Register ( http://encepp.eu ), identifier #32271.

Breaking new frontiers: Assessment and re-evaluation of clinical trial design for nutraceuticals

Despite sophisticated study designs and measurement tools, we have yet to create an innovative space for diet and dietary supplements in the health care system. The path is challenging due to current hierarchies of scientific evidence and regulatory affairs. The role of the randomized, double-blind, placebo-controlled clinical trial (RCT) as a research approach functions well to characterize the benefits and risks of drugs but lacks the sensitivity to capture the efficacy and safety of nutraceuticals. While some facets of RCTs can be relevant and useful when applied to nutraceuticals, other aspects are limiting and potentially misleading when taken in their entirety. A differentiation between guidelines for evidence-based medicine and the evidence required for nutrition spotlight the need to reconceptualize constituents of the RCT and their applicability with relevance to health promotion. This perspective identifies the limitations of the traditional RCT to capture the complexities of nutraceuticals and proposes the N-of-1 as Level 1 evidence better suited for the proof of efficacy of nutraceuticals.

Improved Efficacy of Tafasitamab plus Lenalidomide versus Systemic Therapies for Relapsed/Refractory DLBCL: RE-MIND2, an Observational Retrospective Matched Cohort Study

PURPOSE

In RE-MIND2 (NCT04697160), patient-level outcomes from the L-MIND study (NCT02399085) of tafasitamab plus lenalidomide were retrospectively compared with patient-level matched observational cohorts treated with National Cancer Care Network (NCCN)/European Society for Medical Oncology (ESMO)-listed systemic therapies for relapsed/refractory diffuse large B-cell lymphoma (DLBCL).

PATIENTS AND METHODS

Data were collected from health records of eligible patients aged ≥18 years with histologically confirmed DLBCL who had received ≥2 systemic therapies for DLBCL (including ≥1 anti-CD20 therapy). Patients from L-MIND were matched with patients from the RE-MIND2 observational cohort using estimated propensity score-based 1:1 nearest-neighbor matching, balanced for nine covariates. The primary analysis compared tafasitamab plus lenalidomide with patients who received any systemic therapy for R/R DLBCL (pooled in one cohort) or bendamustine plus rituximab (BR) or rituximab plus gemcitabine and oxaliplatin (R-GemOx; as two distinct cohorts). The primary endpoint was overall survival (OS). Secondary endpoints included treatment response and time-to-event outcomes.

RESULTS

In RE-MIND2, 3,454 patients were enrolled from 200 sites in North America, Europe, and Asia-Pacific. Strictly matched pairs of patients consisted of tafasitamab plus lenalidomide versus systemic therapies pooled (n = 76 pairs), versus BR (n = 75 pairs), and versus R-GemOx (n = 74 pairs). Significantly prolonged OS was reported with tafasitamab plus lenalidomide versus systemic pooled therapies [hazard ratios (HR): 0.55; P = 0.0068], BR (HR: 0.42; P < 0.0001), and R-GemOx (HR: 0.47; P = 0.0003).

CONCLUSIONS

RE-MIND2, a retrospective observational study, met its primary endpoint, demonstrating prolonged OS with tafasitamab plus lenalidomide versus BR and R-GemOx. See related commentary by Cherng and Westin, p. 3908.

Real-World Evidence Prediction of a Phase IV Oncology Trial: Comparative Degarelix vs Leuprolide Safety

BACKGROUND

Medical and regulatory communities are increasingly interested in the utility of real-world evidence (RWE) for answering questions pertaining to drug safety and effectiveness but concerns about validity remain. A principled approach to conducting RWE studies may alleviate concerns and increase confidence in findings. This study sought to predict the findings from the PRONOUNCE trial using a principled approach to generating RWE.

METHODS

This propensity-score (PS) matched observational cohort study utilized 3 claims databases to compare the occurrence of major adverse cardiovascular events (MACE) among initiators of degarelix vs. leuprolide. Patients were included if they had history of prostate cancer and atherosclerotic cardiovascular disease. Subjects were excluded if they didn't have continuous database enrollment in the year prior to treatment initiation, were exposed to androgen deprivation therapy or experienced an acute cardiovascular event within 30 days prior to treatment initiation, or had a history or risk factors of QT prolongation.

RESULTS

There were 12,448 leuprolide and 1,969 degarelix study-eligible patients before matching, with 1,887 in each arm after PS-matching. The results for MACE comparing degarelix to leuprolide in the observational analysis (hazard ratio= 1.35; 95% confidence interval = 0.94-1.93) was consistent with the subsequently released PRONOUNCE result (hazard ratio = 1.28; 95% confidence interval = 0.59-2.79).

CONCLUSIONS

This study successfully predicted the result of a comparative cardiovascular safety trial in the oncology setting. Although the findings are encouraging, limitations of measuring cancer stage and tumor progression are representative of challenges in attempting to generalize whether claims-based RWE can be used as actionable evidence.

The future of research in hematology: Integration of conventional studies with real-world data and artificial intelligence

Most national health-care systems approve new drugs based on data of safety and efficacy from large randomized clinical trials (RCTs). Strict selection biases and study-entry criteria of subjects included in RCTs often do not reflect those of the population where a therapy is intended to be used. Compliance to treatment in RCTs also differs considerably from real world settings and the relatively small size of most RCTs make them unlikely to detect rare but important safety signals. These and other considerations may explain the gap between evidence generated in RCTs and translating conclusions to health-care policies in the real world. Real-world evidence (RWE) derived from real-world data (RWD) is receiving increasing attention from scientists, clinicians, and health-care policy decision-makers - especially when it is processed by artificial intelligence (AI). We describe the potential of using RWD and AI in Hematology to support research and health-care decisions.

Accuracy of identifying diagnosis of moderate to severe chronic kidney disease in administrative claims data

BACKGROUND

Prior validation studies of claims-based definitions of chronic kidney disease (CKD) using ICD-9 codes reported overall low sensitivity, high specificity, and variable but reasonable PPV. No studies to date have evaluated the accuracy of ICD-10 codes to identify a US patient population with CKD.

METHODS

We assessed the accuracy of claims-based algorithms to identify adults with CKD Stages 3-5 compared with laboratory values in a subset (~40%) of a US commercial insurance claims database (Optum's de-identified Clinformatics® Data Mart Database). We calculated the positive predictive value (PPV) of one or two ICD-9 (2012-2014) or ICD-10 (2016-2018) codes for CKD compared with a lab-based estimated glomerular filtration rate (eGFR) occurring within prespecified windows (±90 days, ±180 days, ±365 days) of the ICD-based CKD code(s).

RESULTS

The study population ranged between 104 774 and 161 305 patients (ICD-9 cohorts) and between 285 520 and 373 220 patients (ICD-10 cohorts). The mean age was 74.4 years (ICD-9) and 75.6 years (ICD-10) and the median eGFR was 48 ml/min/1.73 m² . The algorithm of two CKD codes compared with a lab value ±90 days of the first code achieved the highest PPV (PPV 86.36% [ICD-9] and 86.07% [ICD-10]). Overall, ICD-10 based codes had comparable PPVs to ICD-9 based codes and all ICD-10 based algorithms had PPVs >80%. The algorithm of one CKD code compared with laboratory value ±180 days maintained the PPV above 80% but still retained a large number of patients (PPV 80.32% [ICD-9] and 81.56% [ICD-10]).

CONCLUSION

An ICD-10-based definition of CKD identified with sufficient accuracy a patient population with CKD Stages 3-5. Our findings suggest that claims databases could be used for future real-world research studies in patients with CKD Stages 3-5.

Comparing direct acting antivirals for hepatitis C using observational data - Why and how?

The World Health Organisation's goal of hepatitis C virus (HCV) elimination by 2030 will require lower drug prices. Estimates of comparative efficacy promote competition between pharmaceutical companies but direct acting antivirals have been approved for the treatment of HCV without comparative trials. We emulated a randomized trial to answer the question of whether easy to treat patients with genotype 1 HCV could be treated with sofosbuvir/ledipasvir (SOF/LDV) rather than sofosbuvir/velpatasvir (SOF/VEL). Patients without comorbidities or end stage liver disease were selected from the British Colombia Hepatitis Testers Cohort. To create a conceptual trial, we matched each patient starting SOF/VEL (a ‘case’) to the patient starting SOF/LDV with the closest propensity score (a ‘control’). We estimated the probability of treatment failure under a Bayesian logistic model with a random effect for each case‐control set and used that model to give an estimate of a risk difference for the conceptual trial. Treatment failure was recorded for 27 of 825 (3%) cases and for 29 of 602 (5%) matched controls. Estimates from our model were treatment success rates of 97% (95% credible interval, CrI, 95%‐98%) for treatment with SOF/VEL, 95% (95% CrI 93%‐97%) for treatment with SOF/LDV and a risk difference between treatments of 2% (95% CrI 0%‐4%). This risk difference is evidence that SOF/LDV is not inferior to SOF/VEL for easy to treat patients with genotype 1 HCV. The approach is a template for comparing drugs when there are no data from comparative trials.

Matching Methods for Confounder Adjustment: An Addition to the Epidemiologist's Toolbox

Propensity score weighting and outcome regression are popular ways to adjust for observed confounders in epidemiological research. Here, we provide an introduction to matching methods, which serve the same purpose but can offer advantages in robustness and performance. A key difference between matching and weighting methods is that matching methods do not directly rely on the propensity score and so are less sensitive to its misspecification or to the presence of extreme values. Matching methods offer many options for customization, which allow a researcher to incorporate substantive knowledge and carefully manage bias/variance trade-offs in estimating the effects of nonrandomized exposures. We review these options and their implications, providing guidance for their use, and comparison with weighting methods. Because of their potential advantages over other methods, matching methods should have their place in an epidemiologist's methodological toolbox.

A novel data mining application to detect safety signals for newly approved medications in routine care of patients with diabetes

Background

Clinical trials are often underpowered to detect serious but rare adverse events of a new medication. We applied a novel data mining tool to detect potential adverse events of canagliflozin, the first sodium glucose co‐transporter 2 (SGLT2 inhibitor) in the United States, using real‐world data from shortly after its market entry and before public awareness of its potential safety concerns.

Methods

In a U. S. commercial claims dataset (29 March 2013–30 Sept 2015), two pairwise cohorts of patients over 18 years of age with type 2 diabetes (T2D) who were newly dispensed canagliflozin or an active comparator, that is a dipeptidyl peptidase 4 inhibitor (DPP4) or a glucagon‐like peptide 1 receptor agonist (GLP1), were identified and propensity score‐matched. We used variable ratio matching with up to four people receiving a DPP4 or GLP1 for each person receiving canagliflozin. We identified potential safety signals using a hierarchical tree‐based scan statistic data mining method with the hierarchical outcome tree constructed based on international classification of disease coding. We screened for incident adverse events where there were more outcomes observed among canagliflozin vs. comparator initiators than expected by chance, after adjusting for multiple testing.

Results

We identified two pairwise propensity score variable ratio matched cohorts of 44,733 canagliflozin vs. 99,458 DPP4 initiators, and 55,974 canagliflozin vs. 74,727 GLP1 initiators. When we screened inpatient and emergency room diagnoses, diabetic ketoacidosis was the only severe adverse event associated with canagliflozin initiation with p < .05 in both cohorts. When outpatient diagnoses were also considered, signals for female and male genital infections emerged in both cohorts (p < .05).

Conclusions and relevance

In a large population‐based study, we identified known but no other adverse events associated with canagliflozin, providing reassurance on its safety among adult patients with T2D and suggesting the tree‐based scan statistic method is a useful post‐marketing safety monitoring tool for newly approved medications.

Decomposition analysis as a framework for understanding heterogeneity of treatment effects in non-randomized health care studies

This paper uses the decomposition framework from the economics literature to examine the statistical structure of treatment effects estimated with observational data compared to those estimated from randomized studies. It begins with the estimation of treatment effects using a dummy variable in regression models and then presents the decomposition method from economics which estimates separate regression models for the comparison groups and recovers the treatment effect using bootstrapping methods. This method shows that the overall treatment effect is a weighted average of structural relationships of patient features with outcomes within each treatment arm and differences in the distributions of these features across the arms. In large randomized trials, it is assumed that the distribution of features across arms is very similar. Importantly, randomization not only balances observed features but also unobserved. Applying high dimensional balancing methods such as propensity score matching to the observational data causes the distributional terms of the decomposition model to be eliminated but unobserved features may still not be balanced in the observational data. Finally, a correction for non-random selection into the treatment groups is introduced via a switching regime model. Theoretically, the treatment effect estimates obtained from this model should be the same as those from a randomized trial. However, there are significant challenges in identifying instrumental variables that are necessary for estimating such models. At a minimum, decomposition models are useful tools for understanding the relationship between treatment effects estimated from observational versus randomized data.

A systematic approach for introduction of novel treatments to a chronic patient group: sacubitril-valsartan as a case study

Purpose

To develop a model for systematic introduction and to test the feasibility in a chronic disease population. We also investigated how the approach was received by the patients.

Methods and results

The systematic introduction approach is a seven-step procedure: step 1, define a few main criteria; step 2, primary scan patients with the one or two main criteria using computerized medical records/databases/clinical registries; step 3, identify patients applying the other predefined criteria; step 4, evaluate if any examinations/laboratory test updates are required; step 5, summon identified patients to the clinic with an information letter; step 6, discuss treatment with the patient and prescribe if appropriate; and step 7, follow up on initiated therapy and evaluate the applied process. The model was tested in a case study during introduction of the new drug sacubitril-valsartan in a heart failure population. In total, 76 out of 1924 patients were identified to be eligible for sacubitril-valsartan and summoned to the clinic to discuss treatment. Patient experiences with the approach were investigated in an interview study with general inductive approach using qualitative content analysis. This resulted in three final categories: a good approach, role of the information letter, and trust in care.

Conclusions

The systematic introduction approach ensures that strict criteria are used in the selection process and that a treatment can be implemented in eligible patients within a specified population with limited resources and time. The model was effective in our case study and maintained the patient’s confidence in healthcare.

Adjusted treatment COMPArisons between guSelkumab and uStekinumab for treatment of moderate-to-severe plaque psoriasis: the COMPASS analysis

BACKGROUND

Guselkumab is an interleukin-23 inhibitor indicated for the treatment of moderate-to-severe plaque psoriasis in adults. Guselkumab has demonstrated additional benefit in patients with early inadequate response to ustekinumab. Long-term efficacy comparisons of guselkumab and ustekinumab are currently lacking among ustekinumab-naive patients.

OBJECTIVES

To assess the relative efficacy of guselkumab and ustekinumab for maintenance therapy of moderate-to-severe plaque psoriasis, using individual patient data (IPD) from randomized controlled trials.

METHODS

IPD for guselkumab from the VOYAGE 1 and 2 trials were pooled and compared with IPD for ustekinumab from the NAVIGATE trial. Multivariable logistic regression analyses compared guselkumab 100 mg and ustekinumab 45 mg or 90 mg for the achievement and maintenance of Psoriasis Area and Severity Index (PASI) 90, 75 and 100 responses up to 40 weeks. The regression models accounted for a range of clinically relevant covariates (e.g. age, sex, psoriasis duration). Relative efficacy was expressed using odds ratios (ORs) and predicted probability of treatment response associated with each intervention.

RESULTS

Patients receiving guselkumab had significantly higher probabilities of achieving a PASI 90 response than patients receiving ustekinumab, at both week 16 [70·4% vs. 46·0%, OR 2·79, 95% confidence interval (CI) 2·22-3·45] and week 40 (74·2% vs. 54·5%, OR 2·40, 95% CI 1·89-3·13]. Guselkumab was also associated with a significantly increased likelihood of achieving both PASI 75 and PASI 100 responses at weeks 16 and 40, compared with ustekinumab.

CONCLUSIONS

Adjusted analyses leveraging IPD demonstrate that guselkumab has a significantly higher probability of achieving and maintaining PASI treatment responses through week 40 than ustekinumab does.

Improving rheumatoid arthritis comparative effectiveness research through causal inference principles: systematic review using a target trial emulation framework

OBJECTIVES

Target trial emulation is an intuitive design framework that encourages investigators to formulate their comparative effectiveness research (CER) question as a hypothetical randomised controlled trial (RCT). Our aim was to systematically review CER studies in rheumatoid arthritis (RA) to provide examples of design limitations that could be avoided using target trial emulation, and how these limitations might introduce bias.

METHODS

We searched for head-to-head CER studies of biologic disease modifying anti-rheumatic drugs (DMARDs) in RA. Study designs were reviewed for seven components of the target trial emulation framework: eligibility criteria, treatment strategies, assignment procedures, follow-up period, outcome, causal contrasts of interest (ie, intention-to-treat (ITT) or per-protocol effect) and analysis plan. Hypothetical trials corresponding to the reported methods were assessed to identify design limitations that would have been avoided with an explicit target trial protocol. Analysis of the primary effectiveness outcome was chosen where multiple analyses were performed.

RESULTS

We found 31 CER studies, of which 29 (94%) had at least one design limitation belonging to seven components. The most common limitations related to: (1) eligibility criteria: 19/31 (61%) studies used post-baseline information to define baseline eligibility; (2) causal contrasts: 25 (81%) did not define whether ITT or per-protocol effects were estimated and (3) assignment procedures: 13 (42%) studies did not account for confounding by indication or relied solely on statistical confounder selection.

CONCLUSIONS

Design limitations were found in 94% of observational CER studies in RA. Target trial emulation is a structured approach for designing observational CER studies that helps to avoid potential sources of bias.

Emerging opportunities to harness real world data: An introduction to data sources, concepts, and applications

While randomized controlled trials (RCTs) are the gold standard for comparative effectiveness research, they are unable to provide the answers to all pertinent clinical and research questions. Real world evidence (RWE), that is, clinical evidence obtained outside RCTs and often through routine clinical practice, offers the potential to conduct observational studies that accelerate advances in care, improve outcomes for patients, and provide important insights that can answer important questions. Once appropriate information technology is available, real world data can be cost-effective to generate. RWE serves a vital role in the evaluation of treatment strategies for which there are no RCTs and for describing patterns of care. RWE also serves as an important adjunct to RCTs and can be used to determine if benefits seen in RCTs extend to clinical practice, provide insight into the findings of RCTs, generate hypotheses for future RCTs, and inform the design of future RCTs. These potential benefits must be balanced against some of the important limitations of RWE, including variable data quality, lack of granularity for important clinical variables, and the potential for bias and confounding. By using appropriate analytic techniques and study design, these limitations can be minimized but not eliminated. Going forward, RWE studies may be enhanced by using rigorous data quality standards, incorporating randomization, developing more prospective registries, and better leveraging data from electronic health records.

Usability of Clinical Information in Discharge Summary Data in the Diagnosis Procedure Combination Survey for Cancer Patients

Valid data are required to monitor and measure the quality of cancer treatment. This study aims to assess the usability of diagnosis procedure combination (DPC) survey discharge summary data. DPC survey data were analyzed by linking them to the hospital-based cancer registries (HBCR) from 231 hospitals. We focused on patients who were aged 20 years or older and diagnosed in 2013 with stomach, colorectal, liver, lung, or breast cancer. We assessed the percentage of unknown/missing values in supplementary data for patients with five common cancers and compared DPC cancer stage information to that of HBCR. In total, 279,451 discharge data sets for 180,399 patients were analyzed. The percentages of unknown data for smoking index and height/weight were 10.5% and 2.3%, respectively, and varied from 0.0% to 93.0% between hospitals. In the activity of daily living component, the rates of missing data for climbing stairs (3.6%) and bathing (2.9%) at admission were slightly higher than for other elements. Unexpectedly low concordance rate of tumor, node, and metastasis classification between DPC survey and HBCR data was observed as 80.6%, which means 20.4% of the data showed discrepancies. The usability of DPC survey discharge summary data is generally acceptable, but some variables had substantial amounts of missing values.

Ethics and Collateral Findings in Pragmatic Clinical Trials

Pragmatic clinical trials (PCTs) offer important benefits, such as generating evidence that is suited to inform real-world health care decisions and increasing research efficiency. However, PCTs also present ethical challenges. One such challenge involves the management of information that emerges in a PCT that is unrelated to the primary research question(s), yet may have implications for the individual patients, clinicians, or health care systems from whom or within which research data were collected. We term these findings as ?pragmatic clinical trial collateral findings,? or ?PCT-CFs?. In this article, we explore the ethical considerations associated with the identification, assessment, and management of PCT-CFs, and how these considerations may vary based upon the attributes of a specific PCT. Our purpose is to map the terrain of PCT-CFs to serve as a foundation for future scholarship as well as policy-making and to facilitate careful deliberation about actual cases as they occur in practice.

Incidence of suicidal behaviour and violent crime following antidepressant medication: a Danish cohort study

OBJECTIVE

To examine the incidence of suicidal and violent behaviour following initiation of antidepressant medication.

METHOD

Cohorts of 997 911 conscripts and 95 794 patients with a first-time affective disorder were followed for purchase of antidepressant medication, suicide, suicide attempts and conviction for violent crime in Danish registries between 1997 through 2015. Incidence of outcomes was estimated for the first 28 days, 28-365 days or later after initiation of antidepressants or study entry.

RESULTS

Of 16.5% of conscripts and 73.7% of patients with affective disorders initiated antidepressant medication. Incidence of suicide was 3-4 times higher during the first 28 days after initiation compared to the rates in the following year in both cohorts. A similar trend was seen among the untreated patients with affective disorders, whereas suicide incidence was stable at a low level among conscripts not treated with antidepressants. Incidence of attempted suicide was highest during the 28 days before and after initiation of antidepressants, while rates of violent crime were similar before and after initiation. These trends in incidence were independent of class of antidepressant.

CONCLUSION

Higher rates of suicidal behaviour in the weeks following initiation of antidepressant medication probably reflect disease severity and a delay in mood response.

Long-term comparative effectiveness of pegvaliase versus standard of care comparators in adults with phenylketonuria

Phenylketonuria (PKU) is caused by phenylalanine hydroxylase (PAH) deficiency, resulting in high blood and brain Phenylalanine (Phe) concentrations that can lead to impaired brain development and function. Standard treatment involves a Phe-restricted diet alone or in conjunction with sapropterin dihydrochloride in responsive patients. The Food and Drug Administration approved pegvaliase enzyme substitution therapy for adults with blood Phe >600 μmol/L in the US. Recently, the European Commission also approved pegvaliase for treatment of PKU patients aged 16 years or older with blood Phe >600 μmol/L. The analyses presented below were conducted to provide comparative evidence on long-term treatment effectiveness of pegvaliase versus standard of care in adults with PKU. Adult patients (≥18 years) with baseline blood Phe >600 μmol/L who had enrolled in the pegvaliase phase 2 and phase 3 clinical trials were propensity score-matched to historical cohorts of patients treated with "sapropterin + diet" or with "diet alone". These cohorts were derived from the PKU Demographics, Outcome and Safety (PKUDOS) registry and compared for clinical outcomes including blood Phe concentration and natural intact protein intake after 1 and 2 years. Propensity scores were estimated using logistic regression with probability of treatment as outcome (i.e. pegvaliase, "sapropterin + diet", or "diet alone") and patient demographic and disease severity covariates as predictors. An additional analysis in adult PKU patients with baseline blood Phe ≤600 μmol/L comparing non-matched patient groups "sapropterin + diet" to "diet alone" using PKUDOS registry data only was also conducted. The analyses in patients with baseline blood Phe >600 μmol comparing pegvaliase with "sapropterin + diet" (N = 64 matched pairs) showed lower mean blood Phe concentrations after 1 and 2 years with pegvaliase (505 and 427 μmol/L) versus "sapropterin + diet" (807 and 891 μmol/L); mean natural intact protein intake after 1 and 2 years was 49 and 57 g/day respectively with pegvaliase versus 23 and 28 g/day with "sapropterin + diet". The analysis comparing pegvaliase with "diet alone" (N = 120 matched pairs) showed lower mean blood Phe at 1 and 2 years with pegvaliase (473 and 302 μmol/L) versus "diet alone" (1022 and 965 μmol/L); mean natural intact protein intake after 1 and 2 years was 47 and 57 g/day with pegvaliase and 27 and 22 g/day with "diet alone". Considerably more patients achieved blood Phe ≤600, ≤360, and ≤120 μmol/L and reductions from baseline of ≥20%, ≥30%, and ≥50% in blood Phe after 1 and 2 years of pegvaliase versus standard treatments. The analysis in patients with baseline blood Phe ≤600 μmol/L showed lower blood Phe after 1 and 2 years with "sapropterin + diet" (240 and 324 μmol/L) versus "diet alone" (580 and 549 μmol/L) and greater percentages of patients achieving blood Phe targets ≤600, ≤360, and ≤120 μmol/L and reductions from baseline of ≥20%, ≥30%, and ≥50% in blood Phe. These results support pegvaliase as the more effective treatment option to lower Phe levels in adults with PKU who have difficulty keeping blood Phe ≤600 μmol/L with "diet alone". For patients with blood Phe ≤600 μmol/L, adding sapropterin to dietary management is an appropriate treatment option, for those responsive to the treatment.

Comparative effectiveness of warfarin, dabigatran, rivaroxaban and apixaban in non-valvular atrial fibrillation: A nationwide pharmacoepidemiological study

OBJECTIVE

To compare effectiveness and safety of warfarin and the direct oral anticoagulants (DOAC) dabigatran, rivaroxaban and apixaban in non-valvular atrial fibrillation in routine care.

METHODS

From nationwide registries, we identified treatment-naïve patients initiating warfarin, dabigatran, rivaroxaban or apixaban for non-valvular atrial fibrillation from July 2013 to December 2015 in Norway. We assessed prescription duration using reverse waiting time distribution. Adjusting for confounding in a Cox proportional hazards model, we estimated one-year risks for ischemic stroke, transient ischemic attack (TIA) or systemic embolism, major or clinically relevant non-major bleeding; intracranial; gastrointestinal; and other bleeding. We censored at switch of treatment or 365 days of follow-up.

RESULTS

We included 30,820 treatment-naïve patients. Compared to warfarin, the adjusted hazard ratios (HR) for ischemic stroke, TIA or systemic embolism were 0.96 (95% CI 0.71-1.28) for dabigatran, 1.12 (95% CI 0.87-1.45) for rivaroxaban and 0.97 (95% CI 0.75-1.26) for apixaban. Corresponding hazard ratios for major or clinically relevant non-major bleeding were 0.73 (95% CI 0.62-0.86) for dabigatran, 0.97 (95% CI 0.84-1.12) for rivaroxaban and 0.71 (95% CI 0.62-0.82) for apixaban. Statistically significant differences of other safety outcomes compared to warfarin were fewer intracranial bleedings with dabigatran (HR 0.28, 95% CI 0.14-0.56), rivaroxaban (HR 0.40, 95% CI 0.23-0.69) and apixaban (HR 0.56, 95% CI 0.34-0.92); fewer gastrointestinal bleedings with apixaban (HR 0.70, 95% CI 0.52-0.93); and fewer other bleedings with dabigatran (HR 0.67, 95% CI 0.55-0.81) and apixaban (HR 0.70, 95% CI 0.59-0.83).

CONCLUSION

After 1 year follow-up in treatment-naïve patients initiating oral anticoagulation for non-valvular atrial fibrillation, all DOACs were similarly effective as warfarin in prevention of ischemic stroke, TIA or systemic embolism. Safety from bleedings was similar or better, including fewer intracranial bleedings with all direct oral anticoagulants, fewer gastrointestinal bleedings with apixaban and fewer other bleedings with dabigatran and apixaban.

Evaluation of bottom-up interventions targeting community-dwelling frail older people in Belgium: methodological challenges and lessons for future comparative effectiveness studies

Background

Optimizing the organization of care for community-dwelling frail older people is an important issue in many Western countries. In Belgium, a series of complex, innovative, bottom-up interventions was recently designed and implemented to help frail older people live at home longer. As the effectiveness of these interventions may vary between different population groups according to their long-term care needs, they must be evaluated by comparison with a control group that has similar needs.

Methods

The goal was to identify target groups for these interventions and to establish control groups with similar needs and to explore, per group, the extent to which the utilization of long-term care is matched to needs. We merged two databases: a clinical prospective database and the routine administrative database for healthcare reimbursements. Through Principal Component Analysis followed by Clustering, the intervention group was first stratified into disability profiles. Per profile, comparable control groups for clinical variables were established, based on propensity scores. Using chi-squared tests and logistic regression analysis, long-term care utilization at baseline was then compared per profile and group studied.

Results

Stratification highlighted five disability profiles: people with low-level limitations; people with limitations in instrumental activities of daily life and low-level of cognitive impairment; people with functional limitations; people with functional and cognitive impairments; and people with functional, cognitive, and behavioral problems. These profiles made it possible to identify long-term care needs. For instance, at baseline, those who needed more assistance with hygiene tasks also received more personal nursing care (P < 0.05). However, there were some important discrepancies between the need for long-term care and its utilization: while 21% of patients who were totally dependent for hygiene tasks received no personal nursing care, personal nursing care was received by 33% of patients who could perform hygiene tasks.

Conclusions

The disability profiles provide information on long-term care needs but not on the extent to which those needs are met. To assess the effectiveness of interventions, controls at baseline should have similar disability profiles and comparable long-term care utilization. To allow for large comparative effectiveness studies, these dimensions should ideally be available in routine databases.

Electronic supplementary material

The online version of this article (10.1186/s12913-019-4240-9) contains supplementary material, which is available to authorized users.

Time-dependent biases in observational studies of comparative effectiveness research in rheumatology. A methodological review

OBJECTIVE

To assess to what extent time-dependent biases (ie, immortal time bias (ITB) and time-lag bias (TLB)) occur in the latest rheumatology observational studies, describe their main mechanisms and increase the awareness on this topic.

METHODS

We searched PubMed for observational studies on rheumatic diseases published in leading medical journals in the last 5 years. Only studies with a time-to-event analysis exploring the association of one or more interventional strategies with an outcome were included. Each study was labelled as free from bias, at risk of TLB, at risk of misclassified ITB if the period of immortal time was incorrectly attributed to an intervention group, or at risk of excluded ITB if the immortal time was discarded from the analysis.

RESULTS

We included 78 papers. Most studies were performed in Europe or North America (46% each), were not industry funded (62%) and had a safety primary outcome (59%). In total, 13 (17%) studies were considered at risk of time-dependent biases. Among the studies at risk of ITB (n=8; 10%), in 5 (6%), waiting time to receive treatment was wrongly attributed to the treatment exposure group, which indicated misclassified ITB. Five (6%) studies were at risk of TLB: patients on conventional synthetic disease-modifying antirheumatic drugs (DMARD; first-line drugs) were compared with patients on biologic DMARDs (second or third-line drugs) without accounting for disease duration or prior medication use.

CONCLUSIONS

One in six comparative effectiveness observational studies published in leading rheumatology journals is potentially flawed by time-dependent biases.

Data Mining for Adverse Drug Events With a Propensity Score-matched Tree-based Scan Statistic

The tree-based scan statistic is a statistical data mining tool that has been used for signal detection with a self-controlled design in vaccine safety studies. This disproportionality statistic adjusts for multiple testing in evaluation of thousands of potential adverse events. However, many drug safety questions are not well suited for self-controlled analysis. We propose a method that combines tree-based scan statistics with propensity score-matched analysis of new initiator cohorts, a robust design for investigations of drug safety. We conducted plasmode simulations to evaluate performance. In multiple realistic scenarios, tree-based scan statistics in cohorts that were propensity score matched to adjust for confounding outperformed tree-based scan statistics in unmatched cohorts. In scenarios where confounding moved point estimates away from the null, adjusted analyses recovered the prespecified type 1 error while unadjusted analyses inflated type 1 error. In scenarios where confounding moved point estimates toward the null, adjusted analyses preserved power, whereas unadjusted analyses greatly reduced power. Although complete adjustment of true confounders had the best performance, matching on a moderately mis-specified propensity score substantially improved type 1 error and power compared with no adjustment. When there was true elevation in risk of an adverse event, there were often co-occurring signals for clinically related concepts. TreeScan with propensity score matching shows promise as a method for screening and prioritization of potential adverse events. It should be followed by clinical review and safety studies specifically designed to quantify the magnitude of effect, with confounding control targeted to the outcome of interest.

Integrated Care for Crohn's Disease: A Plea for the Development of Clinical Decision Support Systems

Evolution in the management of Crohn's disease [CD] has been characterized by recent paradigm changes. First, new biological therapies induce intestinal healing and full disease control in a substantial number of patients, particularly when introduced early in the disease course. However, they are expensive and associated with potentially severe side effects, raising the question of optimal treatment duration. Secondly, progress in biomarkers and medical imaging performance has enabled better refinement of the definition and prediction of remission or relapse of the disease through monitoring [tight control]. This progress may help to improve tailoring treatment in relation to target ['treat-to target' approach], applying patient-centred and collaborative perspectives, consistent with other chronic disease management. Such an approach requires the integration of a potentially large number of parameters coming from different stakeholders. This integration would be difficult based solely on implementation of classical guidelines and the clinician's intuition. To this end, clinical decision support systems should be developed that integrate a combination of various outcomes to facilitate the treatment decision and to share information between patients, primary care specialists, and health insurance companies or health authorities. This should ease complex therapeutic decisions and serve as a basis for continued research into effectiveness of CD management.

Comparing drug effectiveness in children: A systematic review

PURPOSE

The purpose of the study is to assess the current state of the art in pediatric comparative effectiveness research, potential gaps, and areas for improvement.

METHODS

Relevant articles from inception to February 2015 were retrieved from Embase and Medline. We sequentially screened titles, abstracts, and full texts, with independent validation. Data regarding general information and study methods including statistical analysis were extracted. Study quality was assessed using Newcastle-Ottawa Scale (NOS). Investigated drugs were ranked and compared with data on the prevalence of pediatric drug use.

RESULTS

Three thousand nine hundred twenty-six abstracts were screened for eligibility and inclusion, and 164 articles were included in the review. Most studies were from North America (46.7%). Only 78 studies (47.6%) reported the design: 90.8% were cohort studies. Neonates were least frequently investigated. The drugs that were most often studied included systemic antibacterials (11.4%), psycholeptics (7.9%), and antiepileptics (7.6%). Adjustment for confounding was made using propensity scores in 8.5% of the studies. Studies that did not report the design were of lower quality. Many effectiveness studies were done on antineoplastic agents, which are not frequently used and few studies on analgesics and drugs for obstructive airway diseases which are frequently prescribed.

CONCLUSIONS

There is ample opportunity to improve comparative effectiveness research for drugs used in pediatrics. Routinely prescribed drugs were seldom investigated. Modern methods for confounding adjustment, such as propensity scores, were rarely used.

Association of raltegravir use with long-term health outcomes in HIV-infected patients: an observational post-licensure safety study in a large integrated healthcare system

BACKGROUND

Raltegravir became the first integrase inhibitor to gain FDA approval; but with limited evidence documenting long-term risks in real world care, especially for major health outcomes of interest.

OBJECTIVE

Assess raltegravir safety in clinical practice within an integrated health system.

METHODS

We conducted a cohort study of HIV-infected adults within Kaiser Permanente California from 2005 to 2013. We compared patients initiating raltegravir during the study period with two groups; a historical cohort (started new antiretroviral regimen [ART] 2005-2007) and a concurrent cohort that did not initiate raltegravir (2007-2013). We used multivariate Cox proportional hazard regression to obtain hazard ratios (HR) for pre-specified incident health outcomes, employing propensity scores to adjust for potential confounding.

RESULTS

The population included 8,219 HIV-infected adults (raltegravir cohort N = 1,757; 4,798 patient-years), with greater years known HIV-infected among raltegravir patients. The raltegravir cohort had increased HR for AIDS-defining (HR 2.69 [1.53-4.71]; HR 1.85 [1.21-2.82]) and non-AIDS-defining malignancies (HR 2.26 [1.29-3.94]; HR 1.88 [1.26-2.78]) relative to both comparison cohorts. Compared to the historical cohort we found no significant difference in all-cause mortality; the raltegravir cohort experienced increased HR for all-cause mortality compared to concurrent (HR 1.53 [1.02-2.31]). Raltegravir appeared protective of lipodystrophy when compared to the historical cohort but associated with increased incidence compared to concurrent. There were no significant differences in the incidence of hepatic, skin, or cardiovascular events.

CONCLUSIONS

The potentially elevated risk for malignancy and mortality with raltegravir and residual confounding merits further investigation. We demonstrate the value of observational cohorts for monitoring post-licensure medication safety.

[How to measure real-world effectiveness?]

The new use of medico-administrative databases is a major change in the medical assessment field. Given the limits of the randomized clinical trials when it comes to accurately represent the heterogeneity and complexity of medical care in a real-world context, an increasing demand of observational studies comes from healthcare systems as decision tools. The lack of intervention generates real-world data and assesses both the care and the outcomes of patients in routine practice. Despite this external validity, the bias and confounding factors are a challenge to the internal validity of observational studies. An appropriate study design and statistical methods are necessary to neutralize them and consider a causal relationship between a treatment and a clinical outcome.

Claims-based studies of oral glucose-lowering medications can achieve balance in critical clinical variables only observed in electronic health records

AIM

To evaluate the extent to which balance in unmeasured characteristics of patients with type 2 diabetes (T2DM) was achieved in claims data, by comparing against more detailed information from linked electronic health records (EHR) data.

METHODS

Within a large US commercial insurance database and using a cohort design, we identified patients with T2DM initiating linagliptin or a comparator agent within class (ie, another dipeptidyl peptidase-4 inhibitor) or outside class (ie, pioglitazone or a sulphonylurea) between May 2011 and December 2012. We focused on comparators used at a similar stage of diabetes to linagliptin. For each comparison, 1:1 propensity score (PS) matching was used to balance >100 baseline claims-based characteristics, including proxies of diabetes severity and duration. Additional clinical data from EHR were available for a subset of patients. We assessed representativeness of the claims-EHR-linked subset, evaluated the balance of claims- and EHR-based covariates before and after PS-matching via standardized differences (SDs), and quantified the potential bias associated with observed imbalances.

RESULTS

From a claims-based study population of 166 613 patients with T2DM, 7219 (4.3%) patients were linked to their EHR data. Claims-based characteristics in the EHR-linked and EHR-unlinked patients were similar (SD < 0.1), confirming the representativeness of the EHR-linked subset. The balance of claims-based and EHR-based patient characteristics appeared to be reasonable before PS-matching and generally improved in the PS-matched population, to be SD < 0.1 for most patient characteristics and SD < 0.2 for select laboratory results and body mass index categories, which was not large enough to cause meaningful confounding.

CONCLUSION

In the context of pharmacoepidemiological research on diabetes therapy, choosing appropriate comparison groups paired with a new-user design and 1:1 PS matching on many proxies of diabetes severity and duration improves balance in covariates typically unmeasured in administrative claims datasets, to the extent that residual confounding is unlikely.

Pharmacoepidemiology in pediatrics: Needs, challenges and future directions for research

Despite international initiatives to promote clinical research in pediatrics, there are still many gaps of knowledge in the use of drugs to treat this specific population. When important information cannot be derived only from clinical trials, use of available observational research tools is required. In this paper, we provide an overview of the particular interest of pharmacoepidemiological research into the evaluation of drug effects in children and adolescents. We also sought to underline the unique challenges and specific needs regarding this research. Implementation of innovative methodologies and expansion of database networks to perform necessary studies could further improve performances of observational research.

Use of Health Care Databases to Support Supplemental Indications of Approved Medications

IMPORTANCE

Manufacturers of US Food and Drug Administration-approved prescription drugs often apply for additional indications based on randomized clinical trials. Real-world database analyses on a medication's use and outcomes in routine settings of care might help to inform decision making regarding such supplemental indications.

OBJECTIVE

To examine whether longitudinal data from a health care database can support the results of a randomized clinical trial that led to a supplemental indication for telmisartan.

DESIGN, SETTING, AND PARTICIPANTS

This cohort study of patients newly prescribed telmisartan or ramipril used insurance claims data from a nationwide health care database from January 1, 2003, through September 30, 2009, to compare patient outcomes. This study replicated the inclusion and exclusion criteria used in the Ongoing Telmisartan Alone and in Combination with Ramipril Global End-point Trial (ONTARGET) and used propensity score matching to balance 74 patient characteristics. Data analysis was performed from February 15, 2017, to May 24, 2017.

EXPOSURES

Telmisartan use vs ramipril use.

MAIN OUTCOMES AND MEASURES

The primary outcome was a composite of myocardial infarction, stroke, or hospitalization for congestive heart failure.

RESULTS

Of the 640 951 patients included in the study, 48 053 were newly prescribed ramipril (mean [SD] age, 68.29 [9.52] years; 31 940 male [66.5%]) and 4665 were newly prescribed telmisartan (mean [SD] age, 69.43 [9.60] years; 2413 male [51.7%]). After propensity score matching, a total of 4665 patients were newly prescribed telmisartan (mean [SD] age, 69.43 [9.60] years; 2413 [51.7%]), and 4665 patients were newly prescribed ramipril (mean [SD] age, 69.36 [9.67] years; 2343 male [50.2%]). As seen in ONTARGET, the composite risk of stroke, myocardial infarction, and hospitalization for congestive heart failure was similar for the 2 medications (hazard ratio, 1.0; 95% CI, 0.9-1.1). In addition, the study found that telmisartan was associated with a substantially decreased risk of angioedema (hazard ratio, 0.1; 95% CI, 0.03-0.56) compared with ramipril.

CONCLUSIONS AND RELEVANCE

Real-world data analyses of patients receiving routine care provided findings similar to those found in the randomized clinical trial that established telmisartan's supplemental indication. In certain situations, database studies may support supplemental applications for effectiveness for already approved medications.

Propensity Scores in Pharmacoepidemiology: Beyond the Horizon

Purpose of review

Propensity score methods have become commonplace in pharmacoepidemiology over the past decade. Their adoption has confronted formidable obstacles that arise from pharmacoepidemiology's reliance on large healthcare databases of considerable heterogeneity and complexity. These include identifying clinically meaningful samples, defining treatment comparisons, and measuring covariates in ways that respect sound epidemiologic study design. Additional complexities involve correctly modeling treatment decisions in the face of variation in healthcare practice, and dealing with missing information and unmeasured confounding. In this review, we examine the application of propensity score methods in pharmacoepidemiology with particular attention to these and other issues, with an eye towards standards of practice, recent methodological advances, and opportunities for future progress.

Recent findings

Propensity score methods have matured in ways that can advance comparative effectiveness and safety research in pharmacoepidemiology. These include natural extensions for categorical treatments, matching algorithms that can optimize sample size given design constraints, weighting estimators that asymptotically target matched and overlap samples, and the incorporation of machine learning to aid in covariate selection and model building.

Summary

These recent and encouraging advances should be further evaluated through simulation and empirical studies, but nonetheless represent a bright path ahead for the observational study of treatment benefits and harms.

Assessment of Confounders in Comparative Effectiveness Studies From Secondary Databases

Secondary clinical databases are an important and growing source of data for comparative effectiveness research (CER) studies. However, measurement of confounders, such as biomarker values or patient-reported health status, in secondary clinical databases may not align with the initiation of a new treatment. In many published CER analyses of registry data, investigators assessed confounders based on the first questionnaire in which the new exposure was recorded. However, it is known that adjustment for confounders measured after the start of exposure can lead to biased treatment effect estimates. In the present study, we conducted simulations to compare assessment strategies for a dynamic clinical confounder in a registry-based comparative effectiveness study of 2 therapies. As expected, we found that adjustment for the confounder value at the time of the first questionnaire after the start of exposure creates a biased estimate the total effect of exposure choice on outcome when the confounder mediates part of the effect. However, adjustment for the prior value can also be badly biased when measured long before exposure initiation. Thus, investigators should carefully consider the timing of confounder measurements relative to exposure initiation and the rate of change in the confounder in order to choose the most relevant measure for each patient.

Diffusion of Innovations model helps interpret the comparative uptake of two methodological innovations: co-authorship network analysis and recommendations for the integration of novel methods in practice

OBJECTIVE

The objective of this study was to characterize the diffusion of methodological innovation.

STUDY DESIGN AND SETTING

Comparative case study analysis of the diffusion of two methods that summarize confounder information into a single score: disease risk score (DRS) and high-dimensional propensity score (hdPS). We completed systematic searches to identify DRS and hdPS papers in the field of pharmacoepidemiology through to the end of 2013, plotted the number of papers and unique authors over time, and created sociograms and animations to visualize co-authorship networks. First and last author affiliations were used to ascribe institutional contributions to each paper and network.

RESULTS

We identified 43 DRS papers by 153 authors since 1981, reflecting slow uptake during initial periods of uncertainty and broader diffusion since 2001 linked to early adopters from Vanderbilt. We identified 44 hdPS papers by 147 authors since 2009, reflecting rapid and integrated diffusion, likely facilitated by opinion leaders, early presentation at conferences, easily accessible statistical code, and improvement in funding. Most contributions (87% DRS, 96% hdPS) were from North America.

CONCLUSION

When proposing new methods, authors are encouraged to consider innovation attributes and early evaluation to improve knowledge translation of their innovations for integration into practice, and we provide recommendations for consideration.

"Threshold-crossing": A Useful Way to Establish the Counterfactual in Clinical Trials?

A central question in the assessment of benefit/harm of new treatments is: how does the average outcome on the new treatment (the factual) compare to the average outcome had patients received no treatment or a different treatment known to be effective (the counterfactual)? Randomized controlled trials (RCTs) are the standard for comparing the factual with the counterfactual. Recent developments necessitate and enable a new way of determining the counterfactual for some new medicines. For select situations, we propose a new framework for evidence generation, which we call "threshold-crossing." This framework leverages the wealth of information that is becoming available from completed RCTs and from real world data sources. Relying on formalized procedures, information gleaned from these data is used to estimate the counterfactual, enabling efficacy assessment of new drugs. We propose future (research) activities to enable "threshold-crossing" for carefully selected products and indications in which RCTs are not feasible.

Learning About Missing Data Mechanisms in Electronic Health Records-based Research: A Survey-based Approach

BACKGROUND

Bias due to missing data is a major concern in electronic health record (EHR)-based research. As part of an ongoing EHR-based study of weight change among patients treated for depression, we conducted a survey to investigate determinants of missingness in the available weight information and to evaluate the missing-at-random assumption.

METHODS

We identified 8,345 individuals enrolled in a large EHR-based health care system who had monotherapy treatment for depression from April 2008 to March 2010. A stratified sample of 1,153 individuals completed a detailed survey. Logistic regression was used to investigate determinants of whether a patient (1) had an opportunity to be weighed at treatment initiation (baseline), and (2) had a weight measurement recorded. Parallel analyses were conducted to investigate missingness during follow-up. Throughout, inverse-probability weighting was used to adjust for the design and survey nonresponse. Analyses were also conducted to investigate potential recall bias.

RESULTS

Missingness at baseline and during follow-up was associated with numerous factors not routinely collected in the EHR including whether or not the patient had ever chosen not to be weighed, external weight control activities, and self-reported baseline weight. Patient attitudes about their weight and perceptions regarding the potential impact of their depression treatment on weight were not related to missingness.

CONCLUSION

Adopting a comprehensive strategy to investigate missingness early in the research process gives researchers information necessary to evaluate key assumptions. While the survey presented focuses on outcome data, the overarching strategy can be applied to any and all data elements subject to missingness.

Effect of glucagon-like peptide-1 receptor agonists and dipeptidyl peptidase-4 inhibitors on colorectal cancer incidence and its precursors

AIMS

Incretin-based antihyperglycemic therapies increase intestinal mucosal expansion and polyp growth in mouse models. We aimed to evaluate the effect of dipeptidyl peptidase-4 inhibitors (DPP-4i) or glucagon-like peptide-1 receptor agonists (GLP-1ra) initiation on colorectal cancer incidence.

METHODS

We conducted a cohort study on US Medicare beneficiaries over age 66 from 2007 to 2013 without prevalent cancer. We identified three active-comparator and new-user cohorts: DPP-4i versus thiazolidinediones (TZD), DPP-4i versus sulphonylureas (SU), and GLP-1ra versus long acting insulin (LAI). Follow-up started from 6 months post-second prescription and ended 6 months after stopping (primary as-treated analysis). We estimated hazard ratios (HR) and 95 % confidence intervals (CI) for incident colorectal cancer adjusting for measured confounders using propensity score weighting.

RESULTS

The median duration of treatment ranged 0.7-0.9 years among DPP-4i cohorts. Based on 104 events among 39,334 DPP-4i and 63 events among 25,786 TZD initiators, there was no association between DPP-4i initiation and colorectal cancer (adjusted HR = 1.17 (CI 0.88, 1.71)). There were 73 events among 27,047 DPP-4i and 266 events among 76,012 SU initiators with the adjusted HR 0.98 (CI 0.74, 1.30). We identified 5600 GLP-1ra and 54,767 LAI initiators and the median duration of treatment was 0.8 and 1.2 years, respectively. The adjusted HR was 0.82 (CI 0.42, 1.58) based on <11 events among GLP-1ra versus 276 events among LAI initiators.

CONCLUSION

Although limited by the short duration of treatment, our analyses based on real-world drug utilization patterns provide evidence of no short-term effect of incretin-based agents on colorectal cancer.

Distinguishing Selection Bias and Confounding Bias in Comparative Effectiveness Research

Comparative effectiveness research (CER) aims to provide patients and physicians with evidence-based guidance on treatment decisions. As researchers conduct CER they face myriad challenges. Although inadequate control of confounding is the most-often cited source of potential bias, selection bias that arises when patients are differentially excluded from analyses is a distinct phenomenon with distinct consequences: confounding bias compromises internal validity, whereas selection bias compromises external validity. Despite this distinction, however, the label "treatment-selection bias" is being used in the CER literature to denote the phenomenon of confounding bias. Motivated by an ongoing study of treatment choice for depression on weight change over time, this paper formally distinguishes selection and confounding bias in CER. By formally distinguishing selection and confounding bias, this paper clarifies important scientific, design, and analysis issues relevant to ensuring validity. First is that the 2 types of biases may arise simultaneously in any given study; even if confounding bias is completely controlled, a study may nevertheless suffer from selection bias so that the results are not generalizable to the patient population of interest. Second is that the statistical methods used to mitigate the 2 biases are themselves distinct; methods developed to control one type of bias should not be expected to address the other. Finally, the control of selection and confounding bias will often require distinct covariate information. Consequently, as researchers plan future studies of comparative effectiveness, care must be taken to ensure that all data elements relevant to both confounding and selection bias are collected.