Analytic Challenges Arising from the STOP CRC Trial: Pragmatic Solutions for Pragmatic Problems

Addressing identification bias in the design and analysis of cluster-randomized pragmatic trials: a case study

Background

Pragmatic trials provide the opportunity to study the effectiveness of health interventions to improve care in real-world settings. However, use of open-cohort designs with patients becoming eligible after randomization and reliance on electronic health records (EHRs) to identify participants may lead to a form of selection bias referred to as identification bias. This bias can occur when individuals identified as a result of the treatment group assignment are included in analyses.

Methods

To demonstrate the importance of identification bias and how it can be addressed, we consider a motivating case study, the PRimary care Opioid Use Disorders treatment (PROUD) Trial. PROUD is an ongoing pragmatic, cluster-randomized implementation trial in six health systems to evaluate a program for increasing medication treatment of opioid use disorders (OUDs). A main study objective is to evaluate whether the PROUD intervention decreases acute care utilization among patients with OUD (effectiveness aim). Identification bias is a particular concern, because OUD is underdiagnosed in the EHR at baseline, and because the intervention is expected to increase OUD diagnosis among current patients and attract new patients with OUD to the intervention site. We propose a framework for addressing this source of bias in the statistical design and analysis.

Results

The statistical design sought to balance the competing goals of fully capturing intervention effects and mitigating identification bias, while maximizing power. For the primary analysis of the effectiveness aim, identification bias was avoided by defining the study sample using pre-randomization data (pre-trial modeling demonstrated that the optimal approach was to use individuals with a prior OUD diagnosis). To expand generalizability of study findings, secondary analyses were planned that also included patients newly diagnosed post-randomization, with analytic methods to account for identification bias.

Conclusion

As more studies seek to leverage existing data sources, such as EHRs, to make clinical trials more affordable and generalizable and to apply novel open-cohort study designs, the potential for identification bias is likely to become increasingly common. This case study highlights how this bias can be addressed in the statistical study design and analysis.

Trial registration

ClinicalTrials.gov, NCT03407638. Registered on 23 January 2018.

Moderators of the effectiveness of an intervention to increase colorectal cancer screening through mailed fecal immunochemical test kits: results from a pragmatic randomized trial

Background

Colorectal cancer (CRC) screening rates remain suboptimal, particularly in low-income and underserved populations. Mailed fecal immunochemical testing (FIT) may overcome common barriers to screening; however, the effect of mailed FIT kits may differ across important subpopulations. The goal of the current study was to examine sociodemographic and health-related factors that moderate the effect of an intervention of automated direct mail of FIT kits at health clinics serving low-income populations.

Methods

This study is a secondary analysis of the Strategies and Opportunities to Stop Colon Cancer in Priority Populations (STOP CRC) study, a cluster-randomized pragmatic trial to increase uptake of CRC screening in patients seen at federally qualified health centers. The intervention involved tools embedded in the electronic medical records to enable participating clinics to mail FIT kits and related materials to eligible participants. We examined the rate of FIT completion by potential moderating characteristics using electronic health record data supplemented by the American Community Survey and the Centers for Medicare & Medicaid Services Geographic Variation datasets, linked via geocoding to patients’ addresses. All patients aged 50–75 seen in participating health clinics who were eligible for CRC screening were included.

Results

Although not always statistically significant, we saw a consistent pattern of increased FIT return rates among intervention participants compared to control participants across all subgroups studied, with incidence rate ratios (IRRs) generally ranging from 1.25 to 1.50. FIT completion in the intervention group ranged from 15 and 20% across subpopulations, typically three to six percentage points higher than the control group participants. The only moderator with a statistically significant interaction was race: persons of Asian descent showed a twofold response to the intervention (adjusted incidence rate ratio [aIRR] = 2.06, 95% confidence interval 1.41 to 3.00).

Conclusions

Response to a mailed FIT intervention was generally consistent across a wide range of individual and neighborhood-level patient characteristics, including typically underserved patients and those in low-resource communities.

Trial registration

ClinicalTrials.gov, NCT01742065. Registered on 5 December 2012.