Review of pragmatic trials found that multiple primary outcomes are common but so too are discrepancies between protocols and final reports

Integrating Implementation Outcomes into Effectiveness Studies: A Practical Guide for Clinical Interventionists

Designing effectiveness studies with implementation in mind can allow interventionists to translate their research into real-world practice. Implementation outcomes measure how much and how well a particular intervention was implemented and can provide valuable insights into any heterogeneity in effectiveness outcomes. As part of the National Institutes of Health's (NIH) Helping to End Addiction Long-term® (HEAL) Initiative's Data2Action program, a workgroup of the Research Adoption Support Center (RASC) was tasked with creating a resource guide to aid clinical interventionists in integrating implementation outcomes into their research plans. This paper aims to provide a plain-language, pragmatic guide to implementation outcomes for clinical interventionists, including key considerations for each outcome and examples of implementation-effectiveness studies that reported implementation outcomes. We conclude by discussing the limitations of our guide and implementation outcome reporting more broadly. We offer suggestions for implementation scientists and clinical interventionists to work toward a common goal of improved implementation outcome reporting.

Estimating relative risks and risk differences in randomised controlled trials: a systematic review of current practice

BACKGROUND

Guidelines for randomised controlled trials (RCTs) recommend reporting relative and absolute measures of effect for binary outcomes while adjusting for covariates. There are a number of different ways covariate-adjusted relative risks and risk differences can be estimated.

OBJECTIVES

Our goal was to identify methods used to estimate covariate-adjusted relative risk and risk differences in RCTs published in high-impact journals with binary outcomes. Other secondary objectives included the identification of how covariates are chosen for adjustment and whether covariate adjustment results in an increase in statistical precision in practice.

METHODS

We included two-arm parallel RCTs published in JAMA, NEJM, Lancet, or the BMJ between January 1, 2018, and March 11, 2023, reporting relative risks or risk differences as a summary measure for a binary primary outcome. The search was conducted in Ovid-MEDLINE.

RESULTS

Of the 308 RCTs identified, around half (49%; 95% CI: 43-54%) reported a covariate-adjusted relative risk or risk difference. Of these, 82 reported an adjusted relative risk. When the reporting was clear (n = 65, 79%), the log-binomial model (used in 65% of studies; 95% CI: 52-76%) and modified Poisson (29%; 95% CI: 19-42%) were most commonly used. Of the 92 studies that reported an adjusted risk difference, when the reporting was clear (n = 56, 61%), the binomial model (used in 48% of studies; 95% CI: 35-62%) and marginal standardisation (21%; 95% CI: 12-35%) were the common approaches used.

CONCLUSIONS

Approximately half of the RCTs report either a covariate-adjusted relative risk or risk difference. Many RCTs lack adequate details on the methods used to estimate covariate-adjusted effects. Of those that do report the approaches used, the binomial model, modified Poisson and to a lesser extent marginal standardisation are the approaches used.

Power analysis for cluster randomized trials with continuous coprimary endpoints

Pragmatic trials evaluating health care interventions often adopt cluster randomization due to scientific or logistical considerations. Systematic reviews have shown that coprimary endpoints are not uncommon in pragmatic trials but are seldom recognized in sample size or power calculations. While methods for power analysis based on K (K ≥ 2 $K\ge 2$ ) binary coprimary endpoints are available for cluster randomized trials (CRTs), to our knowledge, methods for continuous coprimary endpoints are not yet available. Assuming a multivariate linear mixed model (MLMM) that accounts for multiple types of intraclass correlation coefficients among the observations in each cluster, we derive the closed-form joint distribution of K treatment effect estimators to facilitate sample size and power determination with different types of null hypotheses under equal cluster sizes. We characterize the relationship between the power of each test and different types of correlation parameters. We further relax the equal cluster size assumption and approximate the joint distribution of the K treatment effect estimators through the mean and coefficient of variation of cluster sizes. Our simulation studies with a finite number of clusters indicate that the predicted power by our method agrees well with the empirical power, when the parameters in the MLMM are estimated via the expectation-maximization algorithm. An application to a real CRT is presented to illustrate the proposed method.

Reporting of and explanations for under-recruitment and over-recruitment in pragmatic trials: a secondary analysis of a database of primary trial reports published from 2014 to 2019

OBJECTIVES

To describe the extent to which pragmatic trials underachieved or overachieved their target sample sizes, examine explanations and identify characteristics associated with under-recruitment and over-recruitment.

STUDY DESIGN AND SETTING

Secondary analysis of an existing database of primary trial reports published during 2014-2019, registered in ClinicalTrials.gov, self-labelled as pragmatic and with target and achieved sample sizes available.

RESULTS

Of 372 eligible trials, the prevalence of under-recruitment (achieving <90% of target sample size) was 71 (19.1%) and of over-recruitment (>110% of target) was 87 (23.4%). Under-recruiting trials commonly acknowledged that they did not achieve their targets (51, 71.8%), with the majority providing an explanation, but only 11 (12.6%) over-recruiting trials acknowledged recruitment excess. The prevalence of under-recruitment in individually randomised versus cluster randomised trials was 41 (17.0%) and 30 (22.9%), respectively; prevalence of over-recruitment was 39 (16.2%) vs 48 (36.7%), respectively. Overall, 101 025 participants were recruited to trials that did not achieve at least 90% of their target sample size. When considering trials with over-recruitment, the total number of participants recruited in excess of the target was a median (Q1-Q3) 319 (75-1478) per trial for an overall total of 555 309 more participants than targeted. In multinomial logistic regression, cluster randomisation and lower journal impact factor were significantly associated with both under-recruitment and over-recruitment, while using exclusively routinely collected data and educational/behavioural interventions were significantly associated with over-recruitment; we were unable to detect significant associations with obtaining consent, publication year, country of recruitment or public engagement.

CONCLUSIONS

A clear explanation for under-recruitment or over-recruitment in pragmatic trials should be provided to encourage transparency in research, and to inform recruitment to future trials with comparable designs. The issues and ethical implications of over-recruitment should be more widely recognised by trialists, particularly when designing cluster randomised trials.