Emulation of Randomized Clinical Trials With Nonrandomized Database Analyses: Results of 32 Clinical Trials

Target trial emulation for comparative effectiveness research with observational data: Promise and challenges for studying medications for opioid use disorder

Medications for opioid use disorder (MOUD) increase retention in care and decrease mortality during active treatment; however, information about the comparative effectiveness of different forms of MOUD is sparse. Observational comparative effectiveness studies are subject to many types of bias; a robust framework to minimize bias would improve the quality of comparative effectiveness evidence. This paper discusses the use of target trial emulation as a framework to conduct comparative effectiveness studies of MOUD with administrative data. Using examples from our planned research project comparing buprenorphine-naloxone and extended-release naltrexone with respect to the rates of MOUD discontinuation, we provide a primer on the challenges and approaches to employing target trial emulation in the study of MOUD.

Emulating randomised clinical trials in relapsing-remitting multiple sclerosis with non-randomised real-world evidence: an application using data from the MSBase Registry

BACKGROUND

To mimic as closely as possible a randomised controlled trial (RCT) and calibrate the real-world evidence (RWE) studies against a known treatment effect would be helpful to understand if RWE can support causal conclusions in selected circumstances. The aim was to emulate the TRANSFORMS trial comparing Fingolimod (FTY) versus intramuscular interferon β-1a (IFN) using observational data.

METHODS

We extracted from the MSBase registry all the patients with relapsing-remitting multiple sclerosis (RRMS) collected in the period 2011-2021 who received IFN or FTY (0.5 mg) and with the same inclusion and exclusion criteria of the TRANSFORMS RCT. The primary endpoint was the annualised relapse rate (ARR) over 12 months. Patients were 1:1 propensity-score (PS) matched. Relapse-rate ratio (RR) was calculated by mean of a negative binomial regression.

RESULTS

A total of 4376 patients with RRMS (1140 in IFN and 3236 in FTY) were selected. After PS, 856 patients in each group were matched. The ARR was 0.45 in IFN and 0.25 in FTY with a significant difference between the two groups (RR: 0.55, 95% CI: 0.45 to 0.68; p<0.001). The result of the emulation was very similar and fell within the 95% CI of that observed in the RCT (RR: 0.49, 95% CI: 0.37 to 0.64; p<0.001) with a standardised difference of 0.66 (p=0.51).

CONCLUSIONS

By applying the same inclusion and exclusion criteria used in the RCT and employing appropriate methodology, we successfully replicated the RCT results with only minor discrepancies. Also, even if the confounding bias cannot be fully eliminated, conducting a rigorous target trial emulation could still yield valuable insights for comparative effectiveness research.

Target Trial Emulation: Improving the Quality of Observational Studies in Inflammatory Bowel Disease Using the Principles of Randomized Trials

The past decade has seen a substantial increase in the number of randomized controlled trials (RCTs) conducted in inflammatory bowel disease (IBD). Randomized controlled trials are the gold standard method for generating robust evidence of drug safety and efficacy but are expensive, time-consuming, and may have ethical implications. Observational studies in IBD are often used to fill the gaps in evidence but are typically hindered by significant bias. There are several approaches for making statistical inferences from observational data with some that focus on study design and others on statistical techniques. Target trial emulation is an emerging methodological process that aims to bridge this gap and improve the quality of observational studies by applying the principles of an ideal, or "target," randomized trial to routinely collected clinical data. There has been a rapid expansion of observational studies that have emulated trials over the past 5 years in other medical fields, but this has yet to be adopted in gastroenterology and IBD. The wealth of nonrandomized clinical data available through electronic health records, patient registries, and administrative health databases afford innumerable hypothesis-generating opportunities for IBD research. This review outlines the principles of target trial emulation, discusses the merits to IBD observational studies in reducing the most common biases and improving confidence in causality, and details the caveats of using this approach.

Why Should the FDA Focus on Pragmatic Clinical Research?

This Viewpoint from the FDA discusses how pragmatic clinical research—assessment that uses real-world data, often in combination with research data, after initial marketing approval—can help in evaluation of new technologies, benefit research sites in underresourced settings, and better inform regulatory decisions and clinical practice.

Effectiveness of nirmatrelvir/ritonavir and molnupiravir in non-hospitalized adults with COVID-19: systematic review and meta-analysis of observational studies

OBJECTIVE

To determine the effectiveness of nirmatrelvir/ritonavir and molnupiravir among vaccinated and unvaccinated non-hospitalized adults with COVID-19.

METHODS

Observational studies of nirmatrelvir/ritonavir or molnupiravir compared to no antiviral drug treatment for COVID-19 in non-hospitalized adults with data on vaccination status were included. We searched MEDLINE, EMBASE, Scopus, Web of Science, WHO COVID-19 Research Database and medRxiv for reports published between 1 January 2022 and 8 November 2023. The primary outcome was a composite of hospitalization or mortality up to 35 days after COVID-19 diagnosis. Risk of bias was assessed with ROBINS-I. Risk ratios (RR), hazard ratios (HR) and risk differences (RD) were separately estimated using random-effects models.

RESULTS

We included 30 cohort studies on adults treated with nirmatrelvir/ritonavir (n = 462 279) and molnupiravir (n = 48 008). Nirmatrelvir/ritonavir probably reduced the composite outcome (RR 0.62, 95%CI 0.55-0.70; I2 = 0%; moderate certainty) with no evidence of effect modification by vaccination status (RR Psubgroup = 0.47). In five studies, RD estimates against the composite outcome for nirmatrelvir/ritonavir were 1.21% (95%CI 0.57% to 1.84%) in vaccinated and 1.72% (95%CI 0.59% to 2.85%) in unvaccinated subgroups.Molnupiravir may slightly reduce the composite outcome (RR 0.75, 95%CI 0.67-0.85; I2 = 32%; low certainty). Evidence of effect modification by vaccination status was inconsistent among studies reporting different effect measures (RR Psubgroup = 0.78; HR Psubgroup = 0.08). In two studies, RD against the composite outcome for molnupiravir were -0.01% (95%CI -1.13% to 1.10%) in vaccinated and 1.73% (95%CI -2.08% to 5.53%) in unvaccinated subgroups.

CONCLUSIONS

Among cohort studies of non-hospitalized adults with COVID-19, nirmatrelvir/ritonavir is effective against the composite outcome of severe COVID-19 independent of vaccination status. Further research and a reassessment of molnupiravir use among vaccinated adults are warranted.

REGISTRATION

PROSPERO CRD42023429232.

Use of the French National Health Data System (SNDS) in pharmacoepidemiology: A systematic review in its maturation phase

AIM OF THE STUDY

The French National Health Data System (SNDS) comprises healthcare data that cover 99% of the population (over 67 million individuals) in France. The aim of this study was to present an overview of published pharmacoepidemiological studies using the SNDS in its maturation phase.

METHODS

We conducted a systematic literature review of original research articles in the Pubmed and EMBASE databases from January 2012 until August 2018.

RESULTS

A total of 316 full-text articles were included, with an annual increase over the study period. Only 16 records were excluded after screening because they did not involve the SNDS but other French healthcare databases. The study design was clearly reported in only 66% of studies of which 57% were retrospective cohorts and 22% cross-sectional studies. The reported study objectives were drug utilization (65%), safety (22%) and effectiveness (9%). Almost all ATC groups were studied but the most frequent ones concerned the nervous system in 149 studies (49%), cardiovascular system drugs in 104 studies (34%) and anti-infectives for systemic use in 50 studies (16%).

CONCLUSION

The SNDS is of growing interest for studies on drug use and safety, which could be conducted more in specific populations, including children, pregnant women and the elderly, as these populations are often not included in clinical trials.

Computational Phenomapping of Randomized Clinical Trials to Enable Assessment of their Real-world Representativeness and Personalized Inference

Importance

Randomized clinical trials (RCTs) are the standard for defining an evidence-based approach to managing disease, but their generalizability to real-world patients remains challenging to quantify.

Objective

To develop a multidimensional patient variable mapping algorithm to quantify the similarity and representation of electronic health record (EHR) patients corresponding to an RCT and estimate the putative treatment effects in real-world settings based on individual treatment effects observed in an RCT.

Design

A retrospective analysis of the Treatment of Preserved Cardiac Function Heart Failure with an Aldosterone Antagonist Trial (TOPCAT; 2006-2012) and a multi-hospital patient cohort from the electronic health record (EHR) in the Yale New Haven Hospital System (YNHHS; 2015-2023).

Setting

A multicenter international RCT (TOPCAT) and multi-hospital patient cohort (YNHHS).

Participants

All TOPCAT participants and patients with heart failure with preserved ejection fraction (HFpEF) and ≥1 hospitalization within YNHHS.

Exposures

63 pre-randomization characteristics measured across the TOPCAT and YNNHS cohorts.

Main Outcomes and Measures

Real-world generalizability of the RCT TOPCAT using a multidimensional phenotypic distance metric between TOPCAT and YNHHS cohorts. Estimation of the individualized treatment effect of spironolactone use on all-cause mortality within the YNHHS cohort based on phenotypic distance from the TOPCAT cohort.

Results

There were 3,445 patients in TOPCAT and 11,712 HFpEF patients across five hospital sites. Across the 63 TOPCAT variables mapped by clinicians to the EHR, there were larger differences between TOPCAT and each of the 5 EHR sites (median SMD 0.200, IQR 0.037-0.410) than between the 5 EHR sites (median SMD 0.062, IQR 0.010-0.130). The synthesis of these differences across covariates using our multidimensional similarity score also suggested substantial phenotypic dissimilarity between the TOPCAT and EHR cohorts. By phenotypic distance, a majority (55%) of TOPCAT participants were closer to each other than any individual EHR patient. Using a TOPCAT-derived model of individualized treatment benefit from spironolactone, those predicted to derive benefit and receiving spironolactone in the EHR cohorts had substantially better outcomes compared with predicted benefit and not receiving the medication (HR 0.74, 95% CI 0.62-0.89).

Conclusions and Relevance

We propose a novel approach to evaluating the real-world representativeness of RCT participants against corresponding patients in the EHR across the full multidimensional spectrum of the represented phenotypes. This enables the evaluation of the implications of RCTs for real-world patients.

KEY POINTS

Question: How can we examine the multi-dimensional generalizability of randomized clinical trials (RCT) to real-world patient populations?Findings: We demonstrate a novel phenotypic distance metric comparing an RCT to real-world populations in a large multicenter RCT of heart failure patients and the corresponding patients in multisite electronic health records (EHRs). Across 63 pre-randomization characteristics, pairwise assessments of members of the RCT and EHR cohorts were more discordant from each other than between members of the EHR cohort (median standardized mean difference 0.200 [0.037-0.410] vs 0.062 [0.010-0.130]), with a majority (55%) of RCT participants closer to each other than any individual EHR patient. The approach also enabled the quantification of expected real world outcomes based on effects observed in the RCT.Meaning: A multidimensional phenotypic distance metric quantifies the generalizability of RCTs to a given population while also offering an avenue to examine expected real-world patient outcomes based on treatment effects observed in the RCT.

Comparative effectiveness of second line oral antidiabetic treatments among people with type 2 diabetes mellitus: emulation of a target trial using routinely collected health data

OBJECTIVE

To compare the effectiveness of three commonly prescribed oral antidiabetic drugs added to metformin for people with type 2 diabetes mellitus requiring second line treatment in routine clinical practice.

DESIGN

Cohort study emulating a comparative effectiveness trial (target trial).

SETTING

Linked primary care, hospital, and death data in England, 2015-21.

PARTICIPANTS

75 739 adults with type 2 diabetes mellitus who initiated second line oral antidiabetic treatment with a sulfonylurea, DPP-4 inhibitor, or SGLT-2 inhibitor added to metformin.

MAIN OUTCOME MEASURES

Primary outcome was absolute change in glycated haemoglobin A1c (HbA1c) between baseline and one year follow-up. Secondary outcomes were change in body mass index (BMI), systolic blood pressure, and estimated glomerular filtration rate (eGFR) at one year and two years, change in HbA1c at two years, and time to ≥40% decline in eGFR, major adverse kidney event, hospital admission for heart failure, major adverse cardiovascular event (MACE), and all cause mortality. Instrumental variable analysis was used to reduce the risk of confounding due to unobserved baseline measures.

RESULTS

75 739 people initiated second line oral antidiabetic treatment with sulfonylureas (n=25 693, 33.9%), DPP-4 inhibitors (n=34 464 ,45.5%), or SGLT-2 inhibitors (n=15 582, 20.6%). SGLT-2 inhibitors were more effective than DPP-4 inhibitors or sulfonylureas in reducing mean HbA1c values between baseline and one year. After the instrumental variable analysis, the mean differences in HbA1c change between baseline and one year were -2.5 mmol/mol (95% confidence interval (CI) -3.7 to -1.3) for SGLT-2 inhibitors versus sulfonylureas and -3.2 mmol/mol (-4.6 to -1.8) for SGLT-2 inhibitors versus DPP-4 inhibitors. SGLT-2 inhibitors were more effective than sulfonylureas or DPP-4 inhibitors in reducing BMI and systolic blood pressure. For some secondary endpoints, evidence for SGLT-2 inhibitors being more effective was lacking-the hazard ratio for MACE, for example, was 0.99 (95% CI 0.61 to 1.62) versus sulfonylureas and 0.91 (0.51 to 1.63) versus DPP-4 inhibitors. SGLT-2 inhibitors had reduced hazards of hospital admission for heart failure compared with DPP-4 inhibitors (0.32, 0.12 to 0.90) and sulfonylureas (0.46, 0.20 to 1.05). The hazard ratio for a ≥40% decline in eGFR indicated a protective effect versus sulfonylureas (0.42, 0.22 to 0.82), with high uncertainty in the estimated hazard ratio versus DPP-4 inhibitors (0.64, 0.29 to 1.43).

CONCLUSIONS

This emulation study of a target trial found that SGLT-2 inhibitors were more effective than sulfonylureas or DPP-4 inhibitors in lowering mean HbA1c, BMI, and systolic blood pressure and in reducing the hazards of hospital admission for heart failure (v DPP-4 inhibitors) and kidney disease progression (v sulfonylureas), with no evidence of differences in other clinical endpoints.

Two Weeks Versus One Week of Maximal Patient-Intensivist Continuity for Adult Medical Intensive Care Patients: A Two-Center Target Trial Emulation

OBJECTIVES

To compare outcomes for 2 weeks vs. 1 week of maximal patient-intensivist continuity in the ICU.

DESIGN

Retrospective cohort study.

SETTING

Two U.S. urban, teaching, medical ICUs where intensivists were scheduled for 2-week service blocks: site A was in the Midwest and site B was in the Northeast.

PATIENTS

Patients 18 years old or older admitted to a study ICU between March 1, 2017, and February 28, 2020.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

We applied target trial emulation to compare admission during an intensivist's first week (as a proxy for 2 wk of maximal continuity) vs. admission during their second week (as a proxy for 1 wk of maximal continuity). Outcomes included hospital mortality, ICU length of stay, and, for mechanically ventilated patients, duration of ventilation. Exploratory outcomes included imaging, echocardiogram, and consultation orders. We used inverse probability weighting to adjust for baseline differences and random-effects meta-analysis to calculate overall effect estimates. Among 2571 patients, 1254 were admitted during an intensivist's first week and 1317 were admitted during a second week. At sites A and B, hospital mortality rates were 25.8% and 24.2%, median ICU length of stay were 4 and 2 days, and median mechanical ventilation durations were 3 and 3 days, respectively. There were no differences in adjusted mortality (odds ratio [OR], 1.01 [95% CI, 0.96-1.06]) or ICU length of stay (-0.25 d [-0.82 d to +0.32 d]) for 2 weeks vs. 1 week of maximal continuity. Among mechanically ventilated patients, there were no differences in adjusted mortality (OR, 1.00 [0.87-1.16]), ICU length of stay (+0.06 d [-0.78 d to +0.91 d]), or duration of mechanical ventilation (+0.37 d [-0.46 d to +1.21 d]) for 2 weeks vs. 1 week of maximal continuity.

CONCLUSIONS

Two weeks of maximal patient-intensivist continuity was not associated with differences in clinical outcomes compared with 1 week in two medical ICUs.

Data Checks Before Registering Study Protocols for Health Care Database Analyses

This Viewpoint discusses the challenges involved with secondary health care data collection vs primary data collection and provides a list of suggested data checks before registration of a study protocol using secondary data.

Emulation of randomized trials of direct oral anticoagulants with claims data and implications for new Factor XI inhibitors

Direct oral anticoagulants (DOACs) revolutionized the management of thromboembolic disorders. Clinical care may be further improved as Factor XIs undergo large-scale outcome trials. What role can non-randomized database studies play in expediting understanding of these drugs in clinical practice? The RCT-DUPLICATIVE Initiative emulated the design of eight DOAC randomized clinical trials (RCT) using non-randomized claims database studies. RCT study design parameters and measurements were closely emulated by the database studies and produced highly concordant results. The results of the single database study that did not meet all agreement metrics with the specific RCT it was emulating were aligned with a meta-analysis of six trials studying similar questions, suggesting the trial result was an outlier. Well-designed database studies using fit-for-purpose data came to the same conclusions as DOAC trials, illustrating how database studies could complement RCTs for Factor XI inhibitors-by accelerating insights in underrepresented populations, demonstrating effectiveness and safety in clinical practice, and testing broader indications.

The Next Horizon of Drug Development: External Control Arms and Innovative Tools to Enrich Clinical Trial Data

Conducting clinical trials (CTs) has become increasingly costly and complex in terms of designing and operationalizing. These challenges exist in running CTs on novel therapies, particularly in oncology and rare diseases, where CTs increasingly target narrower patient groups. In this study, we describe external control arms (ECA) and other relevant tools, such as virtualization and decentralized clinical trials (DCTs), and the ability to follow the clinical trial subjects in the real world using tokenization. ECAs are typically constructed by identifying appropriate external sources of data, then by cleaning and standardizing it to create an analysis-ready data file, and finally, by matching subjects in the external data with the subjects in the CT of interest. In addition, ECA tools also include subject-level meta-analysis and simulated subjects' data for analyses. By implementing the recent advances in digital health technologies and devices, virtualization, and DCTs, realigning of CTs from site-centric designs to virtual, decentralized, and patient-centric designs can be done, which reduces the patient burden to participate in the CTs and encourages diversity. Tokenization technology allows linking the CT data with real-world data (RWD), creating more comprehensive and longitudinal outcome measures. These tools provide robust ways to enrich the CT data for informed decision-making, reduce the burden on subjects and costs of trial operations, and augment the insights gained for the CT data.

Reverse total shoulder replacement versus anatomical total shoulder replacement for osteoarthritis: population based cohort study using data from the National Joint Registry and Hospital Episode Statistics for England

OBJECTIVES

To answer a national research priority by comparing the risk-benefit and costs associated with reverse total shoulder replacement (RTSR) and anatomical total shoulder replacement (TSR) in patients having elective primary shoulder replacement for osteoarthritis.

DESIGN

Population based cohort study using data from the National Joint Registry and Hospital Episode Statistics for England.

SETTING

Public hospitals and publicly funded procedures at private hospitals in England, 2012-20.

PARTICIPANTS

Adults aged 60 years or older who underwent RTSR or TSR for osteoarthritis with intact rotator cuff tendons. Patients were identified from the National Joint Registry and linked to NHS Hospital Episode Statistics and civil registration mortality data. Propensity score matching and inverse probability of treatment weighting were used to balance the study groups.

MAIN OUTCOME MEASURES

The main outcome measure was revision surgery. Secondary outcome measures included serious adverse events within 90 days, reoperations within 12 months, prolonged hospital stay (more than three nights), change in Oxford Shoulder Score (preoperative to six month postoperative), and lifetime costs to the healthcare service.

RESULTS

The propensity score matched population comprised 7124 RTSR or TSR procedures (126 were revised), and the inverse probability of treatment weighted population comprised 12 968 procedures (294 were revised) with a maximum follow-up of 8.75 years. RTSR had a reduced hazard ratio of revision in the first three years (hazard ratio local minimum 0.33, 95% confidence interval 0.18 to 0.59) with no clinically important difference in revision-free restricted mean survival time, and a reduced relative risk of reoperations at 12 months (odds ratio 0.45, 95% confidence interval 0.25 to 0.83) with an absolute risk difference of -0.51% (95% confidence interval -0.89 to -0.13). Serious adverse events and prolonged hospital stay risks, change in Oxford Shoulder Score, and modelled mean lifetime costs were similar. Outcomes remained consistent after weighting.

CONCLUSIONS

This study's findings provide reassurance that RTSR is an acceptable alternative to TSR for patients aged 60 years or older with osteoarthritis and intact rotator cuff tendons. Despite a significant difference in the risk profiles of revision surgery over time, no statistically significant and clinically important differences between RTSR and TSR were found in terms of long term revision surgery, serious adverse events, reoperations, prolonged hospital stay, or lifetime healthcare costs.

The role of COVID-19 vaccines in preventing post-COVID-19 thromboembolic and cardiovascular complications

OBJECTIVE

To study the association between COVID-19 vaccination and the risk of post-COVID-19 cardiac and thromboembolic complications.

METHODS

We conducted a staggered cohort study based on national vaccination campaigns using electronic health records from the UK, Spain and Estonia. Vaccine rollout was grouped into four stages with predefined enrolment periods. Each stage included all individuals eligible for vaccination, with no previous SARS-CoV-2 infection or COVID-19 vaccine at the start date. Vaccination status was used as a time-varying exposure. Outcomes included heart failure (HF), venous thromboembolism (VTE) and arterial thrombosis/thromboembolism (ATE) recorded in four time windows after SARS-CoV-2 infection: 0-30, 31-90, 91-180 and 181-365 days. Propensity score overlap weighting and empirical calibration were used to minimise observed and unobserved confounding, respectively.Fine-Gray models estimated subdistribution hazard ratios (sHR). Random effect meta-analyses were conducted across staggered cohorts and databases.

RESULTS

The study included 10.17 million vaccinated and 10.39 million unvaccinated people. Vaccination was associated with reduced risks of acute (30-day) and post-acute COVID-19 VTE, ATE and HF: for example, meta-analytic sHR of 0.22 (95% CI 0.17 to 0.29), 0.53 (0.44 to 0.63) and 0.45 (0.38 to 0.53), respectively, for 0-30 days after SARS-CoV-2 infection, while in the 91-180 days sHR were 0.53 (0.40 to 0.70), 0.72 (0.58 to 0.88) and 0.61 (0.51 to 0.73), respectively.

CONCLUSIONS

COVID-19 vaccination reduced the risk of post-COVID-19 cardiac and thromboembolic outcomes. These effects were more pronounced for acute COVID-19 outcomes, consistent with known reductions in disease severity following breakthrough versus unvaccinated SARS-CoV-2 infection.

Rise in First-Time ERCP For Benign Indications >1 Year After Cholecystectomy Is Associated With Worse Outcomes

BACKGROUND & AIMS

Greater availability of less invasive biliary imaging to rule out choledocholithiasis should reduce the need for diagnostic endoscopic retrograde cholangiopancreatography (ERCP) in patients who have a remote history of cholecystectomy. The primary aims were to determine the incidence, characteristics, and outcomes of individuals who undergo first-time ERCP >1 year after cholecystectomy (late-ERCP).

METHODS

Data from a commercial insurance claim database (Optum Clinformatics) identified 583,712 adults who underwent cholecystectomy, 4274 of whom underwent late-ERCP, defined as first-time ERCP for nonmalignant indications >1 year after cholecystectomy. Outcomes were exposure and temporal trends in late-ERCP, biliary imaging utilization, and post-ERCP outcomes. Multivariable logistic regression was used to examine patient characteristics associated with undergoing late-ERCP.

RESULTS

Despite a temporal increase in the use of noninvasive biliary imaging (35.9% in 2004 to 65.6% in 2021; P < .001), the rate of late-ERCP increased 8-fold (0.5-4.2/1000 person-years from 2005 to 2021; P < .001). Although only 44% of patients who underwent late-ERCP had gallstone removal, there were high rates of post-ERCP pancreatitis (7.1%), hospitalization (13.1%), and new chronic opioid use (9.7%). Factors associated with late-ERCP included concomitant disorder of gut-brain interaction (odds ratio [OR], 6.48; 95% confidence interval [CI], 5.88-6.91) and metabolic dysfunction steatotic liver disease (OR, 3.27; 95% CI, 2.79-3.55) along with use of anxiolytic (OR, 3.45; 95% CI, 3.19-3.58), antispasmodic (OR, 1.60; 95% CI, 1.53-1.72), and chronic opioids (OR, 6.24; 95% CI, 5.79-6.52).

CONCLUSIONS

The rate of late-ERCP postcholecystectomy is increasing significantly, particularly in patients with comorbidities associated with disorder of gut-brain interaction and mimickers of choledocholithiasis. Late-ERCPs are associated with disproportionately higher rates of adverse events, including initiation of chronic opioid use.

Randomized controlled trials: not always the "gold standard" for evidence in obstetrics and gynecology

Randomized controlled trials are considered the "gold standard" for therapeutic interventions, and it is not uncommon for sweeping changes in medical practice to follow positive results from such trials. However, randomized controlled trials are not without their limitations. Physicians frequently view randomized controlled trials as infallible, whereas they tend to dismiss evidence derived from sources other than randomized controlled trials as less credible or reliable. In several situations in obstetrics and gynecology, there are no randomized controlled trials to help guide the clinician. In these circumstances, it is important to evaluate the entire body of evidence including observational studies, rather than dismiss interventions altogether simply because no randomized controlled trials exist. Randomized controlled trials and observational studies should be viewed as complementary rather than at odds with each other. Some reversals in widely adopted clinical practice have recently been implemented following subsequent studies that contradicted the outcomes of major randomized controlled trials. The most notable of these was the withdrawal from the market of 17-hydroxyprogesterone caproate for preterm birth prevention. Such reversals could potentially have been averted if the inherent limitations of randomized controlled trials were carefully considered before implementing these universal practice changes. This Clinical Opinion underscores the limitations of an exclusive reliance on randomized controlled trials while disregarding other evidence in determining how best to care for patients. Solutions are proposed that advocate that clinicians adopt a more balanced perspective that considers the entirety of the available medical evidence and the individual patient characteristics, needs, and wishes.

Observational studies to emulate randomized trials: Some real-world barriers

The randomized controlled trial (RCT) forms the basis for drug approval by regulatory agencies. Observational studies using existing data from healthcare databases now also provide real-world evidence (RWE) in regulatory decision-making. Several initiatives are assessing the value of RWE by conducting observational studies that emulate published RCTs. While many RCTs are straightforward to emulate, others are challenging. We describe three RCT design aspects that pose challenges for observational studies. First are trials that enrol already treated subjects who must discontinue these treatments at the time of randomization, which can distort the comparison with observational studies. Second is the inclusion of a run-in phase, especially to exclude non-compliant subjects from the trial. Third are trials that evaluate the effect of weaning off treatment. In conclusion, future randomized trials that aim to be emulated by observational studies could consider study designs that allow emulation and thus provide valid and complementary RWE.

Advancing the utilization of real-world data and real-world evidence in clinical pharmacology and translational research-Proceedings from the ASCPT 2023 preconference workshop

Real-world data (RWD) and real-world evidence (RWE) are now being routinely used in epidemiology, clinical practice, and post-approval regulatory decisions. Despite the increasing utility of the methodology and new regulatory guidelines in recent years, there remains a lack of awareness of how this approach can be applied in clinical pharmacology and translational research settings. Therefore, the American Society of Clinical Pharmacology & Therapeutics (ASCPT) held a workshop on March 21st, 2023 entitled "Advancing the Utilization of Real-World Data (RWD) and Real-World Evidence (RWE) in Clinical Pharmacology and Translational Research." The work described herein is a summary of the workshop proceedings.

An evaluation of the replicability of analyses using synthetic health data

Synthetic data generation is being increasingly used as a privacy preserving approach for sharing health data. In addition to protecting privacy, it is important to ensure that generated data has high utility. A common way to assess utility is the ability of synthetic data to replicate results from the real data. Replicability has been defined using two criteria: (a) replicate the results of the analyses on real data, and (b) ensure valid population inferences from the synthetic data. A simulation study using three heterogeneous real-world datasets evaluated the replicability of logistic regression workloads. Eight replicability metrics were evaluated: decision agreement, estimate agreement, standardized difference, confidence interval overlap, bias, confidence interval coverage, statistical power, and precision (empirical SE). The analysis of synthetic data used a multiple imputation approach whereby up to 20 datasets were generated and the fitted logistic regression models were combined using combining rules for fully synthetic datasets. The effects of synthetic data amplification were evaluated, and two types of generative models were used: sequential synthesis using boosted decision trees and a generative adversarial network (GAN). Privacy risk was evaluated using a membership disclosure metric. For sequential synthesis, adjusted model parameters after combining at least ten synthetic datasets gave high decision and estimate agreement, low standardized difference, as well as high confidence interval overlap, low bias, the confidence interval had nominal coverage, and power close to the nominal level. Amplification had only a marginal benefit. Confidence interval coverage from a single synthetic dataset without applying combining rules were erroneous, and statistical power, as expected, was artificially inflated when amplification was used. Sequential synthesis performed considerably better than the GAN across multiple datasets. Membership disclosure risk was low for all datasets and models. For replicable results, the statistical analysis of fully synthetic data should be based on at least ten generated datasets of the same size as the original whose analyses results are combined. Analysis results from synthetic data without applying combining rules can be misleading. Replicability results are dependent on the type of generative model used, with our study suggesting that sequential synthesis has good replicability characteristics for common health research workloads.

Glucocorticoid use in acute respiratory failure from pulmonary causes and association with early changes in the systemic host immune response

BACKGROUND

Glucocorticoids are commonly used in patients with or at-risk for acute respiratory distress syndrome (ARDS), but optimal use remains unclear despite well-conducted clinical trials. We performed a secondary analysis in patients previously enrolled in the Acute Lung Injury and Biospecimen Repository at the University of Pittsburgh. The primary aim of our study was to investigate early changes in host response biomarkers in response to real-world use of glucocorticoids in patients with acute respiratory failure due to ARDS or at-risk due to a pulmonary insult. Participants had baseline plasma samples obtained on study enrollment and on follow-up 3 to 5 days later to measure markers of innate immunity (IL-6, IL-8, IL-10, TNFr1, ST2, fractalkine), epithelial injury (sRAGE), endothelial injury (angiopoietin-2), and host response to bacterial infections (procalcitonin, pentraxin-3). In our primary analyses, we investigated the effect of receiving glucocorticoids between baseline and follow-up samples on host response biomarkers measured at follow-up by doubly robust inverse probability weighting analysis. In exploratory analyses, we examined associations between glucocorticoid use and previously characterized host response subphenotypes (hyperinflammatory and hypoinflammatory).

RESULTS

67 of 148 participants (45%) received glucocorticoids between baseline and follow-up samples. Dose and type of glucocorticoids varied. Regimens that used hydrocortisone alone were most common (37%), and median daily dose was equivalent to 40 mg methylprednisolone (interquartile range: 21, 67). Participants who received glucocorticoids were more likely to be female, to be on immunosuppressive therapy at baseline, and to have higher baseline levels of ST-2, fractalkine, IL-10, pentraxin-3, sRAGE, and TNFr1. Glucocorticoid use was associated with decreases in IL-6 and increases in fractalkine. In exploratory analyses, glucocorticoid use was more frequent in participants in the hyperinflammatory subphenotype (58% vs 40%, p = 0.05), and was not associated with subphenotype classification at the follow-up time point (p = 0.16).

CONCLUSIONS

Glucocorticoid use varied in a cohort of patients with or at-risk for ARDS and was associated with early changes in the systemic host immune response.

The Association Between Metformin Use and New-Onset ICD Coding of Geographic Atrophy

Purpose

Metformin has been suggested to protect against the development of age-related macular degeneration (AMD) in multiple observational studies. However, the association between metformin and geographic atrophy (GA), a debilitating subtype of AMD, has not been analyzed.

Methods

We conducted a case-control study of patients ages 60 years and older with new-onset International Classification of Diseases (ICD) coding of GA in the Merative MarketScan Commercial and Medicare Databases between 2017 and 2021. Cases were matched with propensity scores estimated by age, region, hypertension, and Charlson Comorbidity Index to a control without GA of the same year. Exposure to metformin was assessed for cases and controls in the year prior to their index visit. Conditional multivariable logistic regression, adjusting for AMD risk factors, was used to calculate odd ratios and 95% confidence intervals (CIs). This study design and analysis were repeated in a sample of patients without diabetes.

Results

In the full sample, we identified 10,505 cases with GA and 10,502 matched controls without GA. In total, 1149 (10.9%) cases and 1277 (12.2%) controls were exposed to metformin, and in multivariable regression, metformin decreased the odds of new-onset ICD coding of GA by 12% (95% CI, 0.79-0.99). In the sample of patients without diabetes, we identified 7611 cases with GA and 7608 matched controls without GA. Twenty-nine (0.4%) cases and 63 (0.8%) controls were exposed to metformin, and in multivariable regression, metformin decreased the odds of new-onset ICD coding of GA by 47% (95% CI, 0.33-0.83).

Conclusions

Metformin may hold promise as a noninvasive, alternative agent to prevent the development of GA. This finding is notable due to shortcomings in recently approved therapeutics for GA and metformin's overall ease of use and few adverse effects. Additional studies are required to explore our findings further and motivate a clinical trial.

Staging and clean room: Constructs designed to facilitate transparency and reduce bias in comparative analyses of real-world data

PURPOSE

We describe constructs designed to protect the integrity of the results from comparative analyses using real-world data (RWD): staging and clean room.

METHODS

Staging involves performing sequential preliminary analyses and evaluating the population size available and potential bias before conducting comparative analyses. A clean room involves restricted access to data and preliminary results, policies governing exploratory analyses and protocol deviations, and audit trail. These constructs are intended to allow decisions about protocol deviations, such as changes to design or model specification, to be made without knowledge of how they might affect subsequent analyses. We describe an example for implementing staging with a clean room.

RESULTS

Stage 1 may involve selecting a data source, developing and registering a protocol, establishing a clean room, and applying inclusion/exclusion criteria. Stage 2 may involve attempting to achieve covariate balance, often through propensity score models. Stage 3 may involve evaluating the presence of residual confounding using negative control outcomes. After each stage, check points may be implemented when a team of statisticians, epidemiologists and clinicians masked to how their decisions may affect study outcomes, reviews the results. This review team may be tasked with making recommendations for protocol deviations to address study precision or bias. They may recommend proceeding to the next stage, conducting additional analyses to address bias, or terminating the study. Stage 4 may involve conducting the comparative analyses.

CONCLUSIONS

The staging and clean room constructs are intended to protect the integrity and enhance confidence in the results of analyses of RWD.

Relevance of Protein Intake for Weaning in the Mechanically Ventilated Critically Ill: Analysis of a Large International Database

OBJECTIVES

The association between protein intake and the need for mechanical ventilation (MV) is controversial. We aimed to investigate the associations between protein intake and outcomes in ventilated critically ill patients.

DESIGN

Analysis of a subset of a large international point prevalence survey of nutritional practice in ICUs.

SETTING

A total of 785 international ICUs.

PATIENTS

A total of 12,930 patients had been in the ICU for at least 96 hours and required MV by the fourth day after ICU admission at the latest.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

We modeled associations between the adjusted hazard rate (aHR) of death in patients requiring MV and successful weaning (competing risks), and three categories of protein intake (low: < 0.8 g/kg/d, standard: 0.8-1.2 g/kg/d, high: > 1.2 g/kg/d). We compared five different hypothetical protein diets (an exclusively low protein intake, a standard protein intake given early (days 1-4) or late (days 5-11) after ICU admission, and an early or late high protein intake). There was no evidence that the level of protein intake was associated with time to weaning. However, compared with an exclusively low protein intake, a standard protein intake was associated with a lower hazard of death in MV: minimum aHR 0.60 (95% CI, 0.45-0.80). With an early high intake, there was a trend to a higher risk of death in patients requiring MV: maximum aHR 1.35 (95% CI, 0.99-1.85) compared with a standard diet.

CONCLUSIONS

The duration of MV does not appear to depend on protein intake, whereas mortality in patients requiring MV may be improved by a standard protein intake. Adverse effects of a high protein intake cannot be excluded.

Adapting Health Technology Assessment agency standards for surrogate outcomes in early stage cancer trials: what needs to happen?

INTRODUCTION

An avalanche of early stage cancer clinical trials is coming. The majority of these solely use surrogate outcomes that have not been validated against a target outcome of interest (e.g. overall survival). Current HTA guidance on surrogate outcome validation are not methodologically or practically conducive to this scenario.

AREAS COVERED

We provide a high-level overview of methods, approaches, and conceptual thinking for making better use of limited evidence within early stage cancer HTA submissions. We outline regulatory and HTA issues and emphasize how evidence transitions from one to another, what major gaps currently exist, and how these may be bridged. We summarize current methodologies and practices, their pros and cons. We outline how complementary measurements strengthen evaluations and address fallacies and biases of conventional statistical methods for surrogate outcomes validation. The value of real-world data to support some of the necessary validity components is discussed. Lastly, we address the importance of the patient voice for better understanding which surrogate outcomes may appropriately inform HTA.

EXPERT OPINION

Conventional surrogate outcome validation represents a fraught and sub-optimal framework for HTA purposes, particularly for early stage cancer. Tools for optimizing use of limited evidence exist. Education of stakeholders is highly needed.

Prevalence of Pretreatment Testing Recommended for Patients With Chronic Inflammatory Skin Diseases

Importance

Laboratory testing for the presence of tuberculosis, hepatitis, and other conditions before starting most systemic immunomodulatory agents is recommended in patients with chronic inflammatory skin diseases (CISD) but current testing patterns in the US are unclear.

Objective

To determine the prevalence of pretreatment testing that is recommended for patients with CISD (psoriasis, hidradenitis suppurativa, or atopic dermatitis).

Design, Setting, and Participants

This descriptive analysis of US commercial insurance claims databases from December 31, 2002, to December 31, 2020, included adult patients with CISD (psoriasis, hidradenitis suppurativa, or atopic dermatitis) who started an immunomodulatory agent, including methotrexate, tumor necrosis factor α inhibitors, interleukin (IL)-17Ai, ustekinumab, IL-23i, dupilumab, or apremilast.

Main Outcomes and Measures

The proportion of patients who underwent the screening tests as suggested by professional societies-including for tuberculosis, hepatitis, and liver function; complete blood cell counts; and lipid panels-were determined within 6 months before and during 2 years after treatment start.

Results

A total of 122 308 patients with CISDs (median [IQR] age, 49 [38-58] years; 63 663 [52.1%] male) starting systemic immunomodulatory treatment in the US were included. Treatment for patients with CISDs comprised methotrexate (28 684), tumor necrosis factor α inhibitors (40 965), ustekinumab (12 841), IL-23i (6116), IL-17Ai (9799), dupilumab (7787), or apremilast (16 116). Complete blood cell count was the most common test, performed in 41% (3161/7787) to 69% (19 659/28 684) of individuals before initiation across treatments. Between 11% (889/7787) and 59% (3613/6116) of patients had tuberculosis screening within 6 months before treatment, and 3% (149/4577) to 26% (1559/6097) had updated tests 1 year later. Between 13% (1006/7787) and 41% (16 728/40 965) had hepatitis panels before treatment. Low pretreatment testing levels before apremilast (15% [2331/16 116] to 45% [7253/16 116]) persisted a year into treatment (9% [816/8496] to 36% [2999/8496]) and were similar to dupilumab (11% [850/7787] to 41% [3161/7787] vs 3% [149/4577] to 25% [1160/4577]).

Conclusions and Relevance

In this descriptive analysis of patients with CISDs starting systemic immunomodulatory treatment in the US, less than 60% received the recommended pretreatment testing. Additional research is required to understand whether variations in testing affect patient outcomes.

Synthesis methods used to combine observational studies and randomised trials in published meta-analyses

BACKGROUND

This study examined the synthesis methods used in meta-analyses pooling data from observational studies (OSs) and randomised controlled trials (RCTs) from various medical disciplines.

METHODS

We searched Medline via PubMed to identify reports of systematic reviews of interventions, including and pooling data from RCTs and OSs published in 110 high-impact factor general and specialised journals between 2015 and 2019. Screening and data extraction were performed in duplicate. To describe the synthesis methods used in the meta-analyses, we considered the first meta-analysis presented in each article.

RESULTS

Overall, 132 reports were identified with a median number of included studies of 14 [9-26]. The median number of OSs was 6.5 [3-12] and that of RCTs was 3 [1-6]. The effect estimates recorded from OSs (i.e., adjusted or unadjusted) were not specified in 82% (n = 108) of the meta-analyses. An inverse-variance common-effect model was used in 2% (n = 3) of the meta-analyses, a random-effects model was used in 55% (n = 73), and both models were used in 40% (n = 53). A Poisson regression model was used in 1 meta-analysis, and 2 meta-analyses did not report the model they used. The mean total weight of OSs in the studied meta-analyses was 57.3% (standard deviation, ± 30.3%). Only 44 (33%) meta-analyses reported results stratified by study design. Of them, the results between OSs and RCTs had a consistent direction of effect in 70% (n = 31). Study design was explored as a potential source of heterogeneity in 79% of the meta-analyses, and confounding factors were investigated in only 10% (n = 13). Publication bias was assessed in 70% (n = 92) of the meta-analyses. Tau-square was reported in 32 meta-analyses with a median of 0.07 [0-0.30].

CONCLUSION

The inclusion of OSs in a meta-analysis on interventions could provide useful information. However, considerations of several methodological and conceptual aspects of OSs, that are required to avoid misleading findings, were often absent or insufficiently reported in our sample.

Clinical data mining: challenges, opportunities, and recommendations for translational applications

Clinical data mining of predictive models offers significant advantages for re-evaluating and leveraging large amounts of complex clinical real-world data and experimental comparison data for tasks such as risk stratification, diagnosis, classification, and survival prediction. However, its translational application is still limited. One challenge is that the proposed clinical requirements and data mining are not synchronized. Additionally, the exotic predictions of data mining are difficult to apply directly in local medical institutions. Hence, it is necessary to incisively review the translational application of clinical data mining, providing an analytical workflow for developing and validating prediction models to ensure the scientific validity of analytic workflows in response to clinical questions. This review systematically revisits the purpose, process, and principles of clinical data mining and discusses the key causes contributing to the detachment from practice and the misuse of model verification in developing predictive models for research. Based on this, we propose a niche-targeting framework of four principles: Clinical Contextual, Subgroup-Oriented, Confounder- and False Positive-Controlled (CSCF), to provide guidance for clinical data mining prior to the model's development in clinical settings. Eventually, it is hoped that this review can help guide future research and develop personalized predictive models to achieve the goal of discovering subgroups with varied remedial benefits or risks and ensuring that precision medicine can deliver its full potential.

Design differences and variation in results between randomised trials and non-randomised emulations: meta-analysis of RCT-DUPLICATE data

Objective

To explore how design emulation and population differences relate to variation in results between randomised controlled trials (RCT) and non-randomised real world evidence (RWE) studies, based on the RCT-DUPLICATE initiative (Randomised, Controlled Trials Duplicated Using Prospective Longitudinal Insurance Claims: Applying Techniques of Epidemiology).

Design

Meta-analysis of RCT-DUPLICATE data.

Data sources

Trials included in RCT-DUPLICATE, a demonstration project that emulated 32 randomised controlled trials using three real world data sources: Optum Clinformatics Data Mart, 2004-19; IBM MarketScan, 2003-17; and subsets of Medicare parts A, B, and D, 2009-17.

Eligibility criteria for selecting studies

Trials where the primary analysis resulted in a hazard ratio; 29 RCT-RWE study pairs from RCT-DUPLICATE.

Results

Differences and variation in effect sizes between the results from randomised controlled trials and real world evidence studies were investigated. Most of the heterogeneity in effect estimates between the RCT-RWE study pairs in this sample could be explained by three emulation differences in the meta-regression model: treatment started in hospital (which does not appear in health insurance claims data), discontinuation of some baseline treatments at randomisation (which would have been an unusual care decision in clinical practice), and delayed onset of drug effects (which would be under-reported in real world clinical practice because of the relatively short persistence of the treatment). Adding the three emulation differences to the meta-regression reduced heterogeneity from 1.9 to almost 1 (absence of heterogeneity).

Conclusions

This analysis suggests that a substantial proportion of the observed variation between results from randomised controlled trials and real world evidence studies can be attributed to differences in design emulation.

Association of Renin-Angiotensin System Inhibition With Liver-Related Events and Mortality in Compensated Cirrhosis

BACKGROUND & AIMS

While renin-angiotensin system inhibition lowers the hepatic venous gradient, the effect on more clinically meaningful endpoints is less studied. We aimed to quantify the relationship between renin-angiotensin system inhibition and liver-related events (LREs) among adults with compensated cirrhosis.

METHODS

In this national cohort study using the Optum database, we quantified the association between angiotensin-converting enzyme (ACE) inhibitor or angiotensin-receptor blocker (ARB) use and LREs (hepatocellular carcinoma, liver transplantation, ascites, hepatic encephalopathy, or variceal bleeding) among patients with cirrhosis between 2009 and 2019. Selective beta-blocker (SBB) users served as the comparator group. We used demographic and clinical features to calculate inverse-probability treatment weighting-weighted cumulative incidences, absolute risk differences, and Cox proportional hazard ratios.

RESULTS

Among 4214 adults with cirrhosis, 3155 were ACE inhibitor/ARB users and 1059 were SBB users. In inverse probability treatment weighting-weighted analyses, ACE inhibitor/ARB (vs SBB) users had lower 5-year cumulative incidence (30.6% [95% confidence interval (CI), 27.8% to 33.2%] vs 41.3% [95% CI, 34.0% to 47.7%]; absolute risk difference, -10.7% [95% CI, -18.1% to -3.6%]) and lower risk of LREs (adjusted hazard ratio [aHR], 0.69; 95% CI, 0.60 to 0.80). There was a dose-response relationship: compared with SBB use, ACE inhibitor/ARB prescriptions ≥1 defined daily dose (aHR, 0.65; 95% CI, 0.56 to 0.76) were associated with a greater risk reduction compared with <1 defined daily dose (aHR, 0.87; 95% CI, 0.71 to 1.07). Results were robust across sensitivity analyses such as comparing ACE inhibitor/ARB users with nonusers and as-treated analysis.

CONCLUSIONS

In this national cohort study, ACE inhibitor/ARB use was associated with significantly lower risk of LREs in patients with compensated cirrhosis. These results provide support for a randomized clinical trial to confirm clinical benefit.

Survival impact of post-operative immunotherapy in resected stage III cutaneous melanomas in the checkpoint era

BACKGROUND

Checkpoint inhibitors have shown improvement in recurrence-free survival in the post-operative setting for node-positive melanoma and were first approved in late 2015. However, single-agent checkpoint therapies have yet to show benefit to overall survival (OS) for lower-risk stage III cancers. We evaluated the OS benefit of post-operative immunotherapy in the National Cancer Database (NCDB).

PATIENTS AND METHODS

Patient cases were selected from the NCDB 2020 Participant Use File. Patients diagnosed with stage III cutaneous melanoma between 2016 and 2019 who underwent definitive resection for their melanoma were included. OS between those who received post-operative immunotherapy within 84 days of surgery and those who did not was analyzed by the Kaplan-Meier method. Demographic and clinical characteristics between the two groups were compared via Cox proportional hazard models.

RESULTS

14 978 patients with stage III melanoma were included. Of those, 34.9% (n = 5234) received post-operative immunotherapy and 65.1% (n = 9744) did not. Using the American Joint Committee on Cancer version 8 (AJCCv8) staging, 36-month survival was significantly higher in patients who received post-operative immunotherapy compared to no post-operative systemic therapy in those diagnosed with stage IIIB (88.0% versus 84.7%, P = 0.011), IIIC (75.6% versus 68.1%, P < 0.001), or IIID (59.2% versus 48.4%, P = 0.002). No significant improvement in 36-month survival was seen in patients who received post-operative immunotherapy in patients with stage IIIA disease (93.0% versus 92.2%, P = 0.218).

CONCLUSIONS

Post-operative immunotherapy had an OS benefit in patients with AJCCv8 stage IIIB, IIIC, and IIID disease, but had no significant survival benefit for patients with stage IIIA melanomas.

Surrogacy and the Valuation of ATMPs: Taking Our Place in the Evidence Generation/Assessment Continuum

Medical technology is advancing rapidly, but established methods for health technology assessment are struggling to keep up. This challenge is particularly stark for the assessment of advanced therapy medicinal products-therapies often launched on the basis of single-arm studies powered to a surrogate primary endpoint. The most robust surrogacy methods investigate trial-level correlations between the treatment effect on the surrogate and the outcome of ultimate interest. However, these methods are often impossible with the evidence usually available for advanced therapy medicinal products at the time of the launch (randomized controlled trials are necessary for these advanced methods). Additionally, these surrogacy relationships are usually considered to be technology specific, adding uncertainty for any approach that primarily relies on historic data to estimate the surrogacy relationship for novel interventions such as advanced therapy medicinal products. The literature has already highlighted the need for early dialogue, staged assessment processes, and pricing arrangements that responsibly share the risk between the manufacturer and payer. However, it is our view that in addition to these critical developments, the modeling methods employed could also improve. Currently, health technology assessment practitioners typically either ignore the surrogate and simply extrapolate the endpoint of greatest patient relevance irrespective of the degree of maturity or assume historic surrogate relationships apply to the novel technology. In this opinion piece, we outline an additional avenue. By drawing on the understanding of the mechanism of action and insights generated earlier in the evidence generation/assessment continuum, cost-effectiveness modelers can make better use of the wider data available. These efforts are expected to reduce uncertainty at the time of the initial launch of pharmaceutical products and increase the value of subsequent data collection efforts.

Is the price right? Paying for value today to get more value tomorrow

BACKGROUND

Contemporary debates about drug pricing feature several widely held misconceptions, including the relationship between incentives and innovation, the proportion of total healthcare spending on pharmaceuticals, and whether the economic evaluation of a medicine can be influenced by things other than clinical efficacy.

MAIN BODY

All citizens should have access to timely, equitable, and cost-effective care covered by public funds, private insurance, or a combination of both. Better managing the collective burden of diseases borne by today's and future generations depends in part on developing better technologies, including better medicines. As in any innovative industry, the expectation of adequate financial returns incentivizes innovators and their investors to develop new medicines. Estimating expected returns requires that they forecast revenues, based on the future price trajectory and volume of use over time. How market participants decide what price to set or accept can be complicated, and some observers and stakeholders want to confirm whether the net prices society pays for novel medicines, whether as a reward for past innovation or an incentive for future innovation, are commensurate with those medicines' incremental value. But we must also ask "value to whom?"; medicines not only bring immediate clinical benefits to patients treated today, but also can provide a broad spectrum of short- and long-term benefits to patients, their families, and society. Spending across all facets of healthcare has grown over the last 25 years, but both inpatient and outpatient spending has outpaced drug spending growth even as our drug armamentarium is constantly improving with safer and more effective medicines. In large part, this is because, unlike hospitals, drugs typically go generic, thus making room in our budgets for new and better ones, even as they often keep patients out of hospitals, driving further savings.

CONCLUSION

A thorough evaluation of drug spending and value can help to promote a better allocation of healthcare resources for both the healthy and the sick, both of whom must pay for healthcare. Taking a holistic approach to assessing drug value makes it clear that a branded drug's value to a patient is often only a small fraction of the drug's total value to society. Societal value merits consideration when determining whether and how to make a medicine affordable and accessible to patients: a drug that is worth its price to society should not be rendered inaccessible to ill patients by imposing high out-of-pocket costs or restricting coverage based on narrow health technology assessments (HTAs). Furthermore, recognizing the total societal cost of un- or undertreated conditions is crucial to gaining a thorough understanding of what guides the biomedical innovation ecosystem to create value for society. It would be unwise to discourage the development of new solutions without first appreciating the cost of leaving the problems unsolved.

Platelet versus fresh frozen plasma transfusion for coagulopathy in cardiac surgery patients

BACKGROUND

Platelets (PLTS) and fresh frozen plasma (FFP) are often transfused in cardiac surgery patients for perioperative bleeding. Their relative effectiveness is unknown.

METHODS

We conducted an entropy-weighted retrospective cohort study using the Australian and New Zealand Society of Cardiac and Thoracic Surgeons National Cardiac Surgery Database. All adults undergoing cardiac surgery between 2005-2021 across 58 sites were included. The primary outcome was operative mortality.

RESULTS

Of 174,796 eligible patients, 15,360 (8.79%) received PLTS in the absence of FFP and 6,189 (3.54%) patients received FFP in the absence of PLTS. The median cumulative dose was 1 unit of pooled platelets (IQR 1 to 3) and 2 units of FFP (IQR 0 to 4) respectively. After entropy weighting to achieve balanced cohorts, FFP was associated with increased perioperative (Risk Ratio [RR], 1.63; 95% Confidence Interval [CI], 1.40 to 1.91; P<0.001) and 1-year (RR, 1.50; 95% CI, 1.32 to 1.71; P<0.001) mortality. FFP was associated with increased rates of 4-hour chest drain tube output (Adjusted mean difference in ml, 28.37; 95% CI, 19.35 to 37.38; P<0.001), AKI (RR, 1.13; 95% CI, 1.01 to 1.27; P = 0.033) and readmission to ICU (RR, 1.24; 95% CI, 1.09 to 1.42; P = 0.001).

CONCLUSION

In perioperative bleeding in cardiac surgery patient, platelets are associated with a relative mortality benefit over FFP. This information can be used by clinicians in their choice of procoagulant therapy in this setting.

A Retrospective Observational Real-Word Analysis of the Adherence, Healthcare Resource Consumption and Costs in Patients Treated with Bisoprolol/Perindopril as Single-Pill or Free Combination

INTRODUCTION

The present real-world analysis aims to compare the drug utilization, hospitalizations and direct healthcare costs related to the use of single-pill combination (SPC) or free-equivalent combination (FEC) of perindopril and bisoprolol (PER/BIS) in a large Italian population.

METHODS

This observational retrospective analysis was based on administrative databases covering approximately 7 million subjects across Italy. All adult subjects receiving PER/BIS as SPC or FEC between January 2017-June 2020 were included. Subjects were followed for 1 year after the first prescription of PER/BIS as FEC (± 1 month) or SPC. Before comparing the SPC and FEC cohorts, propensity score matching (PSM) was applied to balance the baseline characteristics. Drug utilization was investigated as adherence (defined by the proportion of days covered, PDC) and persistence (evaluated by Kaplan-Meier curves). Hospitalizations and mean annual direct healthcare costs (due to drug prescriptions, hospitalizations and use of outpatient services) were analyzed during follow-up.

RESULTS

The original cohort included 11,440 and 6521 patients taking the SPC and FEC PER/BIS combination, respectively. After PSM, two balanced SPC and FEC cohorts of 4688 patients were obtained (mean age 70 years, approximately 50% male, 24% in secondary prevention). The proportion of adherent patients (PDC ≥ 80%) was higher for those on SPC (45.5%) than those on FEC (38.6%), p < 0.001. The PER/BIS combination was discontinued by 35.8% of patients in the SPC cohort and 41.7% in the FEC cohort (p < 0.001). The SPC cohort had fewer cardiovascular (CV) hospitalizations (5.3%) than the free-combination cohort (7.4%), p < 0.001. Mean annual total healthcare costs were lower in the SPC (1999€) than in the FEC (2359€) cohort (p < 0.001).

CONCLUSION

In a real-world setting, patients treated with PER/BIS SPC showed higher adherence, lower risk of drug discontinuation, reduced risk of CV hospitalization, and lower healthcare costs than those on FEC of the same drugs.

Stratified analysis in comparative effectiveness studies that emulate randomized trials

PURPOSE

For observational cohort studies that employ matching by propensity scores (PS), preliminary stratification by consequential predictors of outcome better emulates stratified randomization and potentially reduces variance and bias through relaxed dependence on modeling assumptions. We assessed the impact of pre-stratification in two real-life examples. For both, prior evidence from placebo-controlled randomized clinical trials (RCTs) suggested small or no risk reduction, but observational analysis suggested protection, presumably the result of confounding bias.

STUDY DESIGN AND SETTING

The study populations consisted of Medicare beneficiaries (2014-18) with type 2 diabetes initiating either (i) empagliflozin versus dipeptidyl peptidase-4 inhibitors (DPP-4i) or (ii) empagliflozin versus glucagon-like peptide-1 receptor agonists (GLP-1RA). The outcome was myocardial infarction or stroke. We estimated hazard ratios (HR) and rate differences (RD) after controlling for 143 pre-exposure covariates via 1:1 PS matching after (1) PS estimation in the total cohort (total-cohort PS-matching) and (2) PS estimation separately by baseline cardiovascular disease (stratified PS matching).

RESULTS

Stratified PS matching resulted in HRs that exceeded those from total-cohort PS-matching by 13% and 9%, respectively, for the comparisons of empagliflozin to DPP-4i and GLP-1RA. Against both comparators, HRs and RDs after stratified PS matching were closer to the null, with slightly higher variances (2%-3%) than those after total-cohort PS matching.

CONCLUSION

Stratified PS matching produced effect estimates closer to the expected trial findings than total-cohort PS matching. The price paid in increased variance was minimal.

Real world evidence for public health decision-making on vaccination policies: perspectives from an expert roundtable

INTRODUCTION

Influenza causes significant morbidity and mortality, but influenza vaccine uptake remains below most countries' targets. Vaccine policy recommendations vary, as do procedures for reviewing and appraising the evidence.

AREAS COVERED

During a series of roundtable discussions, we reviewed procedures and methodologies used by health ministries in four European countries to inform vaccine recommendations. We review the type of evidence currently recommended by each health ministry and the range of approaches toward considering randomized controlled trials (RCTs) and real-world evidence (RWE) studies when setting influenza vaccine recommendations.

EXPERT OPINION

Influenza vaccine recommendations should be based on data from both RCTs and RWE studies of efficacy, effectiveness, and safety. Such data should be considered alongside health-economic, cost-effectiveness, and budgetary factors. Although RCT data are more robust and less prone to bias, well-designed RWE studies permit timely evaluation of vaccine benefits, effectiveness comparisons over multiple seasons in large populations, and detection of rare adverse events, under real-world conditions. Given the variability of vaccine effectiveness due to influenza virus mutations and increasing diversification of influenza vaccines, we argue that consideration of both RWE and RCT evidence is the best approach to more nuanced and timely updates of influenza vaccine recommendations.

Building trust in real world evidence (RWE): moving transparency in RWE towards the randomized controlled trial standard

The increasing use of RWE in regulatory and reimbursement decision-making indicates the significant progress that has been made in building trust in RWE through greater transparency. This review of the published literature and key online sources was conducted to provide an update on progress towards greater transparency in RWE, based on four key barriers to trust identified in a 2016 paper and applying learnings from transparency initiatives established for RCTs, such as the US FDA Amendments Act (FDAAA) 2007 Final Rule. Multiple initiatives from the US FDA, EMA and organizations such as the ISPOR-ISPE Joint Task Force have yielded new guidance documents and tools that enable greater transparency in RWE study methodology (STaRT-RWE and HARPER templates), data source selection and quality (SPIFD2 framework, REQueST tool), strategy (the Center for Open Science RWE Study Registry), and will therefore improve transparency in RWE study reporting. Programs such as the REPEAT Initiative and RWE DUPLICATE are investigating reproducibility of RWE studies and improving understanding of the circumstances when valid inference on treatment effects can be obtained from RWE studies. Further work is needed to embed and to implement the new tools and guidance that are available, and thus raise standards for RWE transparency towards the levels expected for RCTs.