A review of the use of covariates in cluster randomized trials uncovers marked discrepancies between guidance and practice

Guidelines for the content of statistical analysis plans in clinical trials: protocol for an extension to cluster randomized trials

BACKGROUND

Guidance exists to inform the content of statistical analysis plans in clinical trials. Though not explicitly stated, this guidance is generally focused on clinical trials in which the randomization units are individual patients and not groups of patients. There are critical considerations for the analysis of cluster randomized trials, such as accounting for clustering, the risk of imbalances between the arms due to post-randomization recruitment, and the need to use small sample corrections when the number of clusters is small.

METHODS

This paper outlines the protocol for the development of a set of reporting guidelines for the content of statistical analysis plans for cluster randomized trials (including variations such as the stepped wedge cluster randomized trial and other cluster cross-over designs) by extending the minimum reporting analysis requirements as previously defined for individually randomized trials to cluster randomized trials. The guideline will be developed using a consensus-based approach, modifying existing reporting items from the guideline for individually randomized trials and extending to include new items.

DISCUSSION

The guideline will be developed so it can be used independently of the guideline for individually randomized designs. The consensus guidelines will be published in an open-access journal, including key guidance as well as exploration and elaboration.

Review of the quality of reporting of statistical analysis plans for cluster randomized trials

BACKGROUND AND OBJECTIVES

The guideline for the content of Statistical Analysis Plans (SAPs) outlines recommendations for items to be included in SAPs. As yet there is no specific tailoring of this guideline for Cluster Randomized Trials (CRTs). There has also been no assessment of reporting quality of SAPs against this guideline. Our intention is to identify how well a sample of SAPs for CRTs are adhering to the reporting of key items in the current guidelines, as well as additional analysis aspects considered to be important in CRTs.

METHODS

We include (i) fully published standalone SAPs identified via Ovid-MEDLINE and (ii) SAPs published as supplementary material or appendices to the final published report identified by searching an existing database of nearly 800 CRTs.

RESULTS

The search identified 85 unique SAPs: 26 were published in standalone format and 59 were supplementary material to the full trial report. There was mixed clarity in reporting of items related to the current guideline (eg, most (61/85, 72%) reported what covariates will be included in any adjustment; but fewer (26/85, 31%) reported what method will be used to estimate the absolute measure of effect). Considering additional aspects important for CRTs, the majority (79/85, 93%) included a plan to allow for clustering in the analysis; but fewer (10/40, 25%) reported how a small number of clusters would be accommodated (this was only considered relevant for the subset of CRTs with fewer than 40 clusters). Few (5/85, 6%) reported how the intracluster correlation would be estimated. Few clearly reported statistical targets of inference: in only two SAPs (2/85, 2%) it was clear whether the objectives were related to the individual or cluster-level average; in trials where relevant, only three (3/70, 4%) clearly reported whether the objectives were related to the marginal or cluster-specific effect.

CONCLUSION

This review has identified specific areas of poor quality of reporting that might need additional consideration when developing the guidance for the reporting of SAPs for CRTs.

A review of current practice in the design and analysis of extremely small stepped-wedge cluster randomized trials

BACKGROUND/AIMS

Stepped-wedge cluster randomized trials tend to require fewer clusters than standard parallel-arm designs due to the switches between control and intervention conditions, but there are no recommendations for the minimum number of clusters. Trials randomizing an extremely small number of clusters are not uncommon, but the justification for small numbers of clusters is often unclear and appropriate analysis is often lacking. In addition, stepped-wedge cluster randomized trials are methodologically more complex due to their longitudinal correlation structure, and ignoring the distinct within- and between-period intracluster correlations can underestimate the sample size in small stepped-wedge cluster randomized trials. We conducted a review of published small stepped-wedge cluster randomized trials to understand how and why they are used, and to characterize approaches used in their design and analysis.

METHODS

Electronic searches were used to identify primary reports of full-scale stepped-wedge cluster randomized trials published during the period 2016-2022; the subset that randomized two to six clusters was identified. Two reviewers independently extracted information from each report and any available protocol. Disagreements were resolved through discussion.

RESULTS

We identified 61 stepped-wedge cluster randomized trials that randomized two to six clusters: median sample size (Q1-Q3) 1426 (420-7553) participants. Twelve (19.7%) gave some indication that the evaluation was considered a "preliminary" evaluation and 16 (26.2%) recognized the small number of clusters as a limitation. Sixteen (26.2%) provided an explanation for the limited number of clusters: the need to minimize contamination (e.g. by merging adjacent units), limited availability of clusters, and logistical considerations were common explanations. Majority (51, 83.6%) presented sample size or power calculations, but only one assumed distinct within- and between-period intracluster correlations. Few (10, 16.4%) utilized restricted randomization methods; more than half (34, 55.7%) identified baseline imbalances. The most common statistical method for analysis was the generalized linear mixed model (44, 72.1%). Only four trials (6.6%) reported statistical analyses considering small numbers of clusters: one used generalized estimating equations with small-sample correction, two used generalized linear mixed model with small-sample correction, and one used Bayesian analysis. Another eight (13.1%) used fixed-effects regression, the performance of which requires further evaluation under stepped-wedge cluster randomized trials with small numbers of clusters. None used permutation tests or cluster-period level analysis.

CONCLUSION

Methods appropriate for the design and analysis of small stepped-wedge cluster randomized trials have not been widely adopted in practice. Greater awareness is required that the use of standard sample size calculation methods can provide spuriously low numbers of required clusters. Methods such as generalized estimating equations or generalized linear mixed models with small-sample corrections, Bayesian approaches, and permutation tests may be more appropriate for the analysis of small stepped-wedge cluster randomized trials. Future research is needed to establish best practices for stepped-wedge cluster randomized trials with a small number of clusters.

Adherence to key recommendations for design and analysis of stepped-wedge cluster randomized trials: A review of trials published 2016-2022

BACKGROUND/AIMS

The stepped-wedge cluster randomized trial (SW-CRT), in which clusters are randomized to a time at which they will transition to the intervention condition - rather than a trial arm - is a relatively new design. SW-CRTs have additional design and analytical considerations compared to conventional parallel arm trials. To inform future methodological development, including guidance for trialists and the selection of parameters for statistical simulation studies, we conducted a review of recently published SW-CRTs. Specific objectives were to describe (1) the types of designs used in practice, (2) adherence to key requirements for statistical analysis, and (3) practices around covariate adjustment. We also examined changes in adherence over time and by journal impact factor.

METHODS

We used electronic searches to identify primary reports of SW-CRTs published 2016-2022. Two reviewers extracted information from each trial report and its protocol, if available, and resolved disagreements through discussion.

RESULTS

We identified 160 eligible trials, randomizing a median (Q1-Q3) of 11 (8-18) clusters to 5 (4-7) sequences. The majority (122, 76%) were cross-sectional (almost all with continuous recruitment), 23 (14%) were closed cohorts and 15 (9%) open cohorts. Many trials had complex design features such as multiple or multivariate primary outcomes (50, 31%) or time-dependent repeated measures (27, 22%). The most common type of primary outcome was binary (51%); continuous outcomes were less common (26%). The most frequently used method of analysis was a generalized linear mixed model (112, 70%); generalized estimating equations were used less frequently (12, 8%). Among 142 trials with fewer than 40 clusters, only 9 (6%) reported using methods appropriate for a small number of clusters. Statistical analyses clearly adjusted for time effects in 119 (74%), for within-cluster correlations in 132 (83%), and for distinct between-period correlations in 13 (8%). Covariates were included in the primary analysis of the primary outcome in 82 (51%) and were most often individual-level covariates; however, clear and complete pre-specification of covariates was uncommon. Adherence to some key methodological requirements (adjusting for time effects, accounting for within-period correlation) was higher among trials published in higher versus lower impact factor journals. Substantial improvements over time were not observed although a slight improvement was observed in the proportion accounting for a distinct between-period correlation.

CONCLUSIONS

Future methods development should prioritize methods for SW-CRTs with binary or time-to-event outcomes, small numbers of clusters, continuous recruitment designs, multivariate outcomes, or time-dependent repeated measures. Trialists, journal editors, and peer reviewers should be aware that SW-CRTs have additional methodological requirements over parallel arm designs including the need to account for period effects as well as complex intracluster correlations.

How should a cluster randomized trial be analyzed?

In cluster randomized trials, individuals from the same cluster tend to have more similar outcomes than individuals from different clusters. This correlation must be taken into account in the analysis of every cluster trial to avoid incorrect inferences. In this paper, we describe the principles guiding the analysis of cluster trials including how to correctly account for intra-cluster correlations as well as how to analyze more advanced designs such as stepped-wedge and cluster cross-over trials. We then describe how to handle specific issues such as small sample sizes and missing data.

Evaluating tests for cluster-randomized trials with few clusters under generalized linear mixed models with covariate adjustment: A simulation study

Generalized linear mixed models (GLMM) are commonly used to analyze clustered data, but when the number of clusters is small to moderate, standard statistical tests may produce elevated type I error rates. Small-sample corrections have been proposed for continuous or binary outcomes without covariate adjustment. However, appropriate tests to use for count outcomes or under covariate-adjusted models remains unknown. An important setting in which this issue arises is in cluster-randomized trials (CRTs). Because many CRTs have just a few clusters (eg, clinics or health systems), covariate adjustment is particularly critical to address potential chance imbalance and/or low power (eg, adjustment following stratified randomization or for the baseline value of the outcome). We conducted simulations to evaluate GLMM-based tests of the treatment effect that account for the small (10) or moderate (20) number of clusters under a parallel-group CRT setting across scenarios of covariate adjustment (including adjustment for one or more person-level or cluster-level covariates) for both binary and count outcomes. We find that when the intraclass correlation is non-negligible (≥ $$ \ge $$ 0.01) and the number of covariates is small (≤ $$ \le $$ 2), likelihood ratio tests with a between-within denominator degree of freedom have type I error rates close to the nominal level. When the number of covariates is moderate (≥ $$ \ge $$ 5), across our simulation scenarios, the relative performance of the tests varied considerably and no method performed uniformly well. Therefore, we recommend adjusting for no more than a few covariates and using likelihood ratio tests with a between-within denominator degree of freedom.

Preventing Respiratory Viral Illness Invisibly (PRiVII): protocol for a pragmatic cluster randomized trial evaluating far-UVC light devices in long-term care facilities to reduce infections

BACKGROUND

Respiratory viral illness (RVI)-e.g., influenza, COVID-19-is a serious threat in long-term care (LTC) facilities. Standard infection control measures are suboptimal in LTC facilities because of residents' cognitive impairments, care needs, and susceptibility to loneliness and mental illness. Further, LTC residents living with high degrees of frailty who contract RVIs often develop the so-called atypical symptoms (e.g., delirium, worse mobility) instead of typical cough and fever, delaying infection diagnosis and treatment. Although far-UVC (222 nm) light devices have shown potent antiviral activity in vitro, clinical efficacy remains unproven.

METHODS

Following a study to assay acceptability at each site, this multicenter, double-blinded, cluster-randomized, placebo-controlled trial aims to assess whether far-UVC light devices impact the incidence of RVIs in LTC facilities. Neighborhoods within LTC facilities are randomized to receive far-UVC light devices (222 nm) or identical placebo light devices that emit only visible spectrum light (400-700 nm) in common areas. All residents are monitored for RVIs using both a standard screening protocol and a novel screening protocol that target atypical symptoms. The 3-year incidence of RVIs will be compared using intention-to-treat analysis. A cost-consequence analysis will follow.

DISCUSSION

This trial aims to inform decisions about whether to implement far-UVC light in LTC facilities for RVI prevention. The trial design features align with this pragmatic intent. Appropriate additional ethical protections have been implemented to mitigate participant vulnerabilities that arise from conducting this study. Knowledge dissemination will be supported through media engagement, peer-reviewed presentations, and publications.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05084898. October 20, 2021.

Key considerations for designing, conducting and analysing a cluster randomized trial

Not only do cluster randomized trials require a larger sample size than individually randomized trials, they also face many additional complexities. The potential for contamination is the most commonly used justification for using cluster randomization, but the risk of contamination should be carefully weighed against the more serious problem of questionable scientific validity in settings with post-randomization identification or recruitment of participants unblinded to the treatment allocation. In this paper we provide some simple guidelines to help researchers conduct cluster trials in a way that minimizes potential biases and maximizes statistical efficiency. The overarching theme of this guidance is that methods that apply to individually randomized trials rarely apply to cluster randomized trials. We recommend that cluster randomization be only used when necessary-balancing the benefits of cluster randomization with its increased risks of bias and increased sample size. Researchers should also randomize at the lowest possible level-balancing the risks of contamination with ensuring an adequate number of randomization units-as well as exploring other options for statistically efficient designs. Clustering should always be allowed for in the sample size calculation; and the use of restricted randomization (and adjustment in the analysis for covariates used in the randomization) should be considered. Where possible, participants should be recruited before randomizing clusters and, when recruiting (or identifying) participants post-randomization, recruiters should be masked to the allocation. In the analysis, the target of inference should align with the research question, and adjustment for clustering and small sample corrections should be used when the trial includes less than about 40 clusters.

Restricted mean survival time to estimate an intervention effect in a cluster randomized trial

For time-to-event outcomes, the difference in restricted mean survival time is a measure of the intervention effect, an alternative to the hazard ratio, corresponding to the expected survival duration gain due to the intervention up to a predefined time t*. We extended two existing approaches of restricted mean survival time estimation for independent data to clustered data in the framework of cluster randomized trials: one based on the direct integration of Kaplan-Meier curves and the other based on pseudo-values regression. Then, we conducted a simulation study to assess and compare the statistical performance of the proposed methods, varying the number and size of clusters, the degree of clustering, and the magnitude of the intervention effect under proportional and non-proportional hazards assumption. We found that the extended methods well estimated the variance and controlled the type I error if there was a sufficient number of clusters (≥ 50) under both proportional and non-proportional hazards assumption. For cluster randomized trials with a limited number of clusters (< 50), a permutation test for pseudo-values regression was implemented and corrected the type I error. We also provided a procedure to estimate permutation-based confidence intervals which produced adequate coverage. All the extended methods performed similarly, but the pseudo-values regression offered the possibility to adjust for covariates. Finally, we illustrated each considered method with a cluster randomized trial evaluating the effectiveness of an asthma-control education program.

A randomized, parallel group, pragmatic comparative-effectiveness trial comparing medication-assisted treatment induction methods in primary care practices: The HOMER study protocol

Opioid use disorder (OUD) represents a public health crisis in the United States. Medication for opioid use disorder (MOUD) with buprenorphine in primary care is a proven OUD treatment strategy. MOUD induction is when patients begin withdrawal and receive the first doses of buprenorphine. Differences between induction methods might influence short-term stabilization, long-term maintenance, and quality of life. This paper describes the protocol for a study designed to: (1) compare short-term stabilization and long-term maintenance treatment engagement in MOUD in patients receiving office, home, or telehealth induction and (2) identify clinically-relevant practice and patient characteristics associated with successful long-term treatment. The study design is a randomized, parallel group, pragmatic comparative effectiveness trial of three care models of MOUD induction in 100 primary care practices in the United States. Eligible patients are at least 16 years old, have been identified by their clinician as having opioid dependence and would benefit from MOUD. Patients will be randomized to one of three induction comparators: office, home, or telehealth induction. Primary outcomes are buprenorphine medication-taking and illicit opioid use at 30, 90, and 270 days post-induction. Secondary outcomes include quality of life and potential mediators of treatment maintenance (intentions, planning, automaticity). Potential moderators include social determinants of health, substance use history and appeal, and executive function. An intent to treat analysis will assess effects of the interventions on long-term treatment, using general/generalized linear mixed models, adjusted for covariates, for the outcomes analysis. Analysis includes practice- and patient-level random effects for hierarchical/longitudinal data. No large-scale, randomized comparative effectiveness research has compared home induction to office or telehealth MOUD induction on long-term outcomes for patients with OUD seen in primary care settings. The results of this study will offer primary care providers evidence and guidance in selecting the most beneficial induction method(s) for specific patients.

A personalized intervention to prevent depression in primary care based on risk predictive algorithms and decision support systems: protocol of the e-predictD study

The predictD is an intervention implemented by general practitioners (GPs) to prevent depression, which reduced the incidence of depression-anxiety and was cost-effective. The e-predictD study aims to design, develop, and evaluate an evolved predictD intervention to prevent the onset of major depression in primary care based on Information and Communication Technologies, predictive risk algorithms, decision support systems (DSSs), and personalized prevention plans (PPPs). A multicenter cluster randomized trial with GPs randomly assigned to the e-predictD intervention + care-as-usual (CAU) group or the active-control + CAU group and 1-year follow-up is being conducted. The required sample size is 720 non-depressed patients (aged 18-55 years), with moderate-to-high depression risk, under the care of 72 GPs in six Spanish cities. The GPs assigned to the e-predictD-intervention group receive brief training, and those assigned to the control group do not. Recruited patients of the GPs allocated to the e-predictD group download the e-predictD app, which incorporates validated risk algorithms to predict depression, monitoring systems, and DSSs. Integrating all inputs, the DSS automatically proposes to the patients a PPP for depression based on eight intervention modules: physical exercise, social relationships, improving sleep, problem-solving, communication skills, decision-making, assertiveness, and working with thoughts. This PPP is discussed in a 15-min semi-structured GP-patient interview. Patients then choose one or more of the intervention modules proposed by the DSS to be self-implemented over the next 3 months. This process will be reformulated at 3, 6, and 9 months but without the GP-patient interview. Recruited patients of the GPs allocated to the control-group+CAU download another version of the e-predictD app, but the only intervention that they receive via the app is weekly brief psychoeducational messages (active-control group). The primary outcome is the cumulative incidence of major depression measured by the Composite International Diagnostic Interview at 6 and 12 months. Other outcomes include depressive symptoms (PHQ-9) and anxiety symptoms (GAD-7), depression risk (predictD risk algorithm), mental and physical quality of life (SF-12), and acceptability and satisfaction ('e-Health Impact' questionnaire) with the intervention. Patients are evaluated at baseline and 3, 6, 9, and 12 months. An economic evaluation will also be performed (cost-effectiveness and cost-utility analysis) from two perspectives, societal and health systems.

Trial registration

ClinicalTrials.gov, identifier: NCT03990792.

A scoping review described diversity in methods of randomization and reporting of baseline balance in stepped-wedge cluster randomized trials

OBJECTIVES

In stepped-wedge cluster randomized trials (SW-CRTs), clusters are randomized not to treatment and control arms but to sequences dictating the times of crossing from control to intervention conditions. Randomization is an essential feature of this design but application of standard methods to promote and report on balance at baseline is not straightforward. We aimed to describe current methods of randomization and reporting of balance at baseline in SW-CRTs.

STUDY DESIGN AND SETTING

We used electronic searches to identify primary reports of SW-CRTs published between 2016 and 2022.

RESULTS

Across 160 identified trials, the median number of clusters randomized was 11 (Q1-Q3: 8-18). Sixty-three (39%) used restricted randomization-most often stratification based on a single cluster-level covariate; 12 (19%) of these adjusted for the covariate(s) in the primary analysis. Overall, 50 (31%) and 134 (84%) reported on balance at baseline on cluster- and individual-level characteristics, respectively. Balance on individual-level characteristics was most often reported by condition in cross-sectional designs and by sequence in cohort designs. Authors reported baseline imbalances in 72 (45%) trials.

CONCLUSION

SW-CRTs often randomize a small number of clusters using unrestricted allocation. Investigators need guidance on appropriate methods of randomization and assessment and reporting of balance at baseline.

Human Papillomavirus vaccination clinical decision support for young adults in an upper midwestern healthcare system: a clinic cluster-randomized control trial

INTRODUCTION

Human papillomavirus (HPV) vaccination rates are low in young adults. Clinical decision support (CDS) in primary care may increase HPV vaccination. We tested the treatment effect of algorithm-driven, web-based, and electronic health record-linked CDS with or without shared decision-making tools (SDMT) on HPV vaccination rates compared to usual care (UC).

METHODS

In a clinic cluster-randomized control trial conducted in a healthcare system serving a largely rural population, we randomized 34 primary care clinic clusters (with three clinics sharing clinicians randomized together) to: CDS; CDS+SDMT; UC. The sample included young adults aged 18-26 due for HPV vaccination with a study index visit from 08/01/2018-03/15/2019 in a study clinic. Generalized linear mixed models tested differences in HPV vaccination status 12 months after index visits by study arm.

RESULTS

Among 10,253 patients, 6,876 (65.2%) were due for HPV vaccination, and 5,054 met study eligibility criteria. In adjusted analyses, the HPV vaccination series was completed by 12 months in 2.3% (95% CI: 1.6%-3.2%) of CDS, 1.6% (95% CI: 1.1%-2.3%) of CDS+SDMT, and 2.2% (95% CI: 1.6%-3.0%) of UC patients, and at least one HPV vaccine was received by 12 months in 13.1% (95% CI: 10.6%-16.1%) of CDS, 9.2% (95% CI: 7.3%-11.6%) of CDS+SDMT, and 11.2% (95% CI: 9.1%-13.7%) of UC patients. Differences were not significant between arms. Females, those with prior HPV vaccinations, and those seen at urban clinics had significantly higher odds of HPV vaccination in adjusted models.

DISCUSSION

CDS may require optimization for young adults to significantly impact HPV vaccination.

TRIAL REGISTRATION

clinicaltrials.gov NCT02986230, 12/6/2016.

Influential methods reports for group-randomized trials and related designs

BACKGROUND

This article identifies the most influential methods reports for group-randomized trials and related designs published through 2020. Many interventions are delivered to participants in real or virtual groups or in groups defined by a shared interventionist so that there is an expectation for positive correlation among observations taken on participants in the same group. These interventions are typically evaluated using a group- or cluster-randomized trial, an individually randomized group treatment trial, or a stepped wedge group- or cluster-randomized trial. These trials face methodological issues beyond those encountered in the more familiar individually randomized controlled trial.

METHODS

PubMed was searched to identify candidate methods reports; that search was supplemented by reports known to the author. Candidate reports were reviewed by the author to include only those focused on the designs of interest. Citation counts and the relative citation ratio, a new bibliometric tool developed at the National Institutes of Health, were used to identify influential reports. The relative citation ratio measures influence at the article level by comparing the citation rate of the reference article to the citation rates of the articles cited by other articles that also cite the reference article.

RESULTS

In total, 1043 reports were identified that were published through 2020. However, 55 were deemed to be the most influential based on their relative citation ratio or their citation count using criteria specific to each of the three designs, with 32 group-randomized trial reports, 7 individually randomized group treatment trial reports, and 16 stepped wedge group-randomized trial reports. Many of the influential reports were early publications that drew attention to the issues that distinguish these designs from the more familiar individually randomized controlled trial. Others were textbooks that covered a wide range of issues for these designs. Others were "first reports" on analytic methods appropriate for a specific type of data (e.g. binary data, ordinal data), for features commonly encountered in these studies (e.g. unequal cluster size, attrition), or for important variations in study design (e.g. repeated measures, cohort versus cross-section). Many presented methods for sample size calculations. Others described how these designs could be applied to a new area (e.g. dissemination and implementation research). Among the reports with the highest relative citation ratios were the CONSORT statements for each design.

CONCLUSIONS

Collectively, the influential reports address topics of great interest to investigators who might consider using one of these designs and need guidance on selecting the most appropriate design for their research question and on the best methods for design, analysis, and sample size.

Evaluating Health Education Module on Hand, Food, and Mouth Diseases Among Preschoolers in Malacca, Malaysia

This study aims to improve parents' perceptions of susceptibility, severity, benefits, and barriers to children's handwashing practice by utilizing the Health Belief Model. In Alor Gajah, Melaka, a parallel cluster-randomized controlled study was conducted over 26 months. Parents who agreed to participate completed pre-test (t0) questionnaires. Data analysis used IBM SPSS version 25. The descriptive analysis described the baseline data pre-intervention. Chi-square and T-test or Mann-Whitney U test for non-parametric analysis assessed baseline data comparability between intervention and control groups. Generalized Estimating Equation (GEE) analyzed between and within-group comparison of the outcomes, and multivariate analysis determined the effectiveness of the intervention with clustered data. The individual participation rate was 86%. Parents who followed up immediately had higher perceived susceptibility, perceived severity, and perceived barriers (p &lt; 0.001). Each unit increment in parents' practice score was 0.02-unit higher preschool children's hand hygiene practice score (p = 0.045). The intervention effectively improved parents' perceived susceptibility and benefits at immediate follow-up compared to baseline. However, there were no significant intervention effects on parents' perceived severity and barriers and preschool children's handwashing practices. The follow-up time significantly affected each outcome. There were significant covariates as the outcome predictors in this study, besides intervention groups and follow-up time. Parents' knowledge and age of the youngest child were significant predictors of parents' perceived susceptibility, besides parents' knowledge and perceived susceptibility being the predictors of parents' practice score. As a result, parents, teachers, and communities can implement this intervention in other schools with susceptible children.

How well do covariates perform when adjusting for sampling bias in online COVID-19 research? Insights from multiverse analyses

COVID-19 research has relied heavily on convenience-based samples, which-though often necessary-are susceptible to important sampling biases. We begin with a theoretical overview and introduction to the dynamics that underlie sampling bias. We then empirically examine sampling bias in online COVID-19 surveys and evaluate the degree to which common statistical adjustments for demographic covariates successfully attenuate such bias. This registered study analysed responses to identical questions from three convenience and three largely representative samples (total N = 13,731) collected online in Canada within the International COVID-19 Awareness and Responses Evaluation Study ( www.icarestudy.com ). We compared samples on 11 behavioural and psychological outcomes (e.g., adherence to COVID-19 prevention measures, vaccine intentions) across three time points and employed multiverse-style analyses to examine how 512 combinations of demographic covariates (e.g., sex, age, education, income, ethnicity) impacted sampling discrepancies on these outcomes. Significant discrepancies emerged between samples on 73% of outcomes. Participants in the convenience samples held more positive thoughts towards and engaged in more COVID-19 prevention behaviours. Covariates attenuated sampling differences in only 55% of cases and increased differences in 45%. No covariate performed reliably well. Our results suggest that online convenience samples may display more positive dispositions towards COVID-19 prevention behaviours being studied than would samples drawn using more representative means. Adjusting results for demographic covariates frequently increased rather than decreased bias, suggesting that researchers should be cautious when interpreting adjusted findings. Using multiverse-style analyses as extended sensitivity analyses is recommended.

Simple compared to covariate-constrained randomization methods in balancing baseline characteristics: a case study of randomly allocating 72 hemodialysis centers in a cluster trial

Background and aim

Some parallel-group cluster-randomized trials use covariate-constrained rather than simple randomization. This is done to increase the chance of balancing the groups on cluster- and patient-level baseline characteristics. This study assessed how well two covariate-constrained randomization methods balanced baseline characteristics compared with simple randomization.

Methods

We conducted a mock 3-year cluster-randomized trial, with no active intervention, that started April 1, 2014, and ended March 31, 2017. We included a total of 11,832 patients from 72 hemodialysis centers (clusters) in Ontario, Canada. We randomly allocated the 72 clusters into two groups in a 1:1 ratio on a single date using individual- and cluster-level data available until April 1, 2013. Initially, we generated 1000 allocation schemes using simple randomization. Then, as an alternative, we performed covariate-constrained randomization based on historical data from these centers. In one analysis, we restricted on a set of 11 individual-level prognostic variables; in the other, we restricted on principal components generated using 29 baseline historical variables.

We created 300,000 different allocations for the covariate-constrained randomizations, and we restricted our analysis to the 30,000 best allocations based on the smallest sum of the penalized standardized differences. We then randomly sampled 1000 schemes from the 30,000 best allocations. We summarized our results with each randomization approach as the median (25th and 75th percentile) number of balanced baseline characteristics. There were 156 baseline characteristics, and a variable was balanced when the between-group standardized difference was ≤ 10%.

Results

The three randomization techniques had at least 125 of 156 balanced baseline characteristics in 90% of sampled allocations. The median number of balanced baseline characteristics using simple randomization was 147 (142, 150). The corresponding value for covariate-constrained randomization using 11 prognostic characteristics was 149 (146, 151), while for principal components, the value was 150 (147, 151).

Conclusion

In this setting with 72 clusters, constraining the randomization using historical information achieved better balance on baseline characteristics compared with simple randomization; however, the magnitude of benefit was modest.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-021-05590-1.

Evaluation of a covariate-constrained randomization procedure in stepped wedge cluster randomized trials

In stepped wedge (SW) designs, differing cluster-level characteristics or individual-level covariate distributions that differ by cluster can lead to imbalance by treatment arm and potential confounding of the treatment effect. Adapting a method used in cluster-randomized trials, we propose a covariate-constrained randomization method to be used in SW designs. First, we define a balance metric to be calculated for all possible randomizations of cluster order for a given SW design. The resulting distribution of this balance metric across all possible randomizations is used to select a candidate set of randomizations with acceptable covariate balance. One cluster order is selected at random from this candidate set to be used as the cluster order for treatment implementation. In a simulation study, we implement the covariate-constrained randomization procedure and compare treatment effect estimation, type I error, and power under varying SW design and confounding settings, and using multiple analysis methods. We observed optimal statistical properties when the balance metric was used to exclude a small set of potential randomizations with the highest level of imbalance, and when analysis methods were adjusted for the potential confounders. The covariate-constrained randomization was most beneficial in settings with a small number of clusters and in the presence of cluster-level confounding.

Use of a personalised depression intervention in primary care to prevent anxiety: a secondary study of a cluster randomised trial

Background

In the predictD-intervention, GPs used a personalised biopsychosocial programme to prevent depression. This reduced the incidence of major depression by 21.0%, although the results were not statistically significant.

Aim

To determine whether the predictD-intervention is effective at preventing anxiety in primary care patients without depression or anxiety.

Design and setting

Secondary study of a cluster randomised trial with practices randomly assigned to either the predictD-intervention or usual care. This study was conducted in seven Spanish cities from October 2010 to July 2012.

Method

In each city, 10 practices and two GPs per practice, as well as four to six patients every recruiting day, were randomly selected until there were 26–27 eligible patients for each GP. The endpoint was cumulative incidence of anxiety as measured by the PRIME-MD screening tool over 18 months.

Results

A total of 3326 patients without depression and 140 GPs from 70 practices consented and were eligible to participate; 328 of these patients were removed because they had an anxiety syndrome at baseline. Of the 2998 valid patients, 2597 (86.6%) were evaluated at the end of the study. At 18 months, 10.4% (95% CI = 8.7% to 12.1%) of the patients in the predictD-intervention group developed anxiety compared with 13.1% (95% CI = 11.4% to 14.8%) in the usual-care group (absolute difference = −2.7% [95% CI = −5.1% to −0.3%]; P = 0.029).

Conclusion

A personalised intervention delivered by GPs for the prevention of depression provided a modest but statistically significant reduction in the incidence of anxiety.

Sample size requirements for detecting treatment effect heterogeneity in cluster randomized trials

Cluster randomized trials (CRTs) refer to experiments with randomization carried out at the cluster or the group level. While numerous statistical methods have been developed for the design and analysis of CRTs, most of the existing methods focused on testing the overall treatment effect across the population characteristics, with few discussions on the differential treatment effect among subpopulations. In addition, the sample size and power requirements for detecting differential treatment effect in CRTs remain unclear, but are helpful for studies planned with such an objective. In this article, we develop a new sample size formula for detecting treatment effect heterogeneity in two-level CRTs for continuous outcomes, continuous or binary covariates measured at cluster or individual level. We also investigate the roles of two intraclass correlation coefficients (ICCs): the adjusted ICC for the outcome of interest and the marginal ICC for the covariate of interest. We further derive a closed-form design effect formula to facilitate the application of the proposed method, and provide extensions to accommodate multiple covariates. Extensive simulations are carried out to validate the proposed formula in finite samples. We find that the empirical power agrees well with the prediction across a range of parameter constellations, when data are analyzed by a linear mixed effects model with a treatment-by-covariate interaction. Finally, we use data from the HF-ACTION study to illustrate the proposed sample size procedure for detecting heterogeneous treatment effects.

Preventive interventions for children in organized team sport tackling aggression: Results from the pilot evaluation of "Fairplayer.Sport"

Current reviews revealed that there is a lack of effective programs and valuable effectiveness studies related to prevention of aggressive behavior and fostering of social competence in early adolescents participating in organized team sports (e.g., ball sports, such as soccer). Using a randomized controlled design, the present pilot study presents first results regarding the effectiveness of the preventive intervention program "Fairplayer.Sport" that was implemented with preadolescent soccer players (N = 145 preadolescents; aged 9-14 years; mean = 12.2 years) in organized team sport (13 soccer teams). Results revealed a reduction of aggressive behavior in the intervention groups compared to waiting-control groups (small effect size). This effect remained stable 3 months after program implementation. Implications for planning and implementing preventive intervention programs are discussed.

Safety and Efficacy of Indocyanine Green Tracer-Guided Lymph Node Dissection During Laparoscopic Radical Gastrectomy in Patients With Gastric Cancer: A Randomized Clinical Trial

IMPORTANCE

The application of indocyanine green (ICG) imaging in laparoscopic radical gastrectomy is in the preliminary stages of clinical practice, and its safety and efficacy remain controversial.

OBJECTIVE

To investigate the safety and efficacy of ICG near-infrared tracer-guided imaging during laparoscopic D2 lymphadenectomy in patients with gastric cancer.

DESIGN, SETTING, AND PARTICIPANTS

Patients with potentially resectable gastric adenocarcinoma (clinical tumor stage cT1-cT4a, N0/+, M0) were enrolled in a prospective randomized clinical trial at a tertiary referral teaching hospital between November 2018 and July 2019. Patients were randomly assigned to the ICG group or the non-ICG group. The number of retrieved lymph nodes, rate of lymph node noncompliance, and postoperative recovery data were compared between the groups in a modified intention-to-treat analysis. Statistical analysis was performed from August to September 2019.

INTERVENTIONS

The ICG group underwent laparoscopic gastrectomy using near-infrared imaging after receiving an endoscopic peritumoral injection of ICG to the submucosa 1 day before surgery.

MAIN OUTCOMES AND MEASURES

Total number of retrieved lymph nodes.

RESULTS

Of 266 participants randomized, 133 underwent ICG tracer-guided laparoscopic gastrectomy, and 133 underwent conventional laparoscopic gastrectomy. After postsurgical exclusions, 258 patients were included in the modified intention-to-treat analysis, which comprised 129 patients (86 men and 43 women; mean [SD] age, 57.8 [10.7] years) in the ICG group and 129 patients (87 men and 42 women; mean [SD] age, 60.1 [9.1] years) in the non-ICG group. The mean number of lymph nodes retrieved in the ICG group was significantly more than the mean number retrieved in the non-ICG group (mean [SD], 50.5 [15.9] lymph nodes vs 42.0 [10.3] lymph nodes, respectively; P < .001). Significantly more perigastric and extraperigastric lymph nodes were retrieved in the ICG group than in the non-ICG group. In addition, the mean total number of lymph nodes retrieved in the ICG group within the scope of D2 lymphadenectomy was also significantly greater than the mean number retrieved in the non-ICG group (mean [SD], 49.6 [15.0] lymph nodes vs 41.7 [10.2] lymph nodes, respectively; P < .001). The lymph node noncompliance rate of the ICG group (41 of 129 patients [31.8%]) was lower than that of the non-ICG group (74 of 129 patients [57.4%]; P < .001). The postoperative recovery process was comparable, and no significant difference was found between the ICG and non-ICG groups in the incidence (20 of 129 patients [15.5%] vs 21 of 129 [16.3%], respectively; P = .86) or severity of complications within 30 days after surgery.

CONCLUSIONS AND RELEVANCE

Indocyanine green can noticeably improve the number of lymph node dissections and reduce lymph node noncompliance without increased complications in patients undergoing D2 lymphadenectomy. Indocyanine green fluorescence imaging can be performed for routine lymphatic mapping during laparoscopic gastrectomy, especially total gastrectomy.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT03050879.

Effectiveness and cost-effectiveness of The Daily Mile on childhood weight outcomes and wellbeing: a cluster randomised controlled trial

BACKGROUND

The Daily Mile is designed to increase physical activity levels with children running or walking around school grounds for 15-min daily. It has been adopted by schools worldwide and endorsed as a solution to tackle obesity, despite no robust evidence of its benefits. We conducted a cluster randomised controlled trial to determine its clinical and cost-effectiveness.

METHODS

Forty schools were randomly assigned (1:1) to either the Daily Mile intervention or control group in which only the usual school health and wellbeing activities were implemented. The primary outcome was BMI z-score (BMIz) at 12 months follow-up from baseline, with planned subgroup analysis to examine differential effects. Primary economic analysis outcome was incremental cost per Quality-Adjusted-Life-Year (QALY) gained.

RESULTS

Using a constrained randomisation approach, balanced on school size, baseline BMIz and proportion of pupils eligible for free school meals, 20 schools were allocated to intervention (n = 1,153 participants) and 20 to control (n = 1,127); 3 schools withdrew (2 intervention, 1 control). At 12 months, BMIz data were available for 18 intervention schools (n = 850) and 19 control schools (n = 820 participants). Using intention-to-treat analysis the adjusted mean difference (MD) in BMIz (intervention - control) was -0.036 (95% CI: -0.085 to 0.013, p = 0.146). Pre-specified subgroup analysis showed a significant interaction with sex (p = 0.001) suggesting a moderate size benefit of The Daily Mile in girls (MD -0.097, 95% CI -0.156 to -0.037). This was consistent with the exploratory economic results that showed The Daily Mile to be highly cost-effective in girls (£2,492 per QALY), but not in boys, and overall to have a 76% chance of cost-effectiveness for the whole sample, at the commonly applied UK threshold of £20,000 per QALY.

CONCLUSIONS

Overall the Daily Mile had a small but non-significant effect on BMIz, however, it had a greater effect in girls suggesting that it might be considered as a cost-effective component of a system-wide approach to childhood obesity prevention.

The Invested in Diabetes Study Protocol: a cluster randomized pragmatic trial comparing standardized and patient-driven diabetes shared medical appointments

Background

Shared medical appointments (SMAs) have been shown to be an efficient and effective strategy for providing diabetes self-management education and self-management support. SMA features vary and it is not known which features are most effective for different patients and practice settings. The Invested in Diabetes study tests the comparative effectiveness of SMAs with and without multidisciplinary care teams and patient topic choice for improving patient-centered and clinical outcomes related to diabetes.

Methods

This study compares the effectiveness of two SMA approaches using the Targeted Training for Illness Management (TTIM) curriculum. Standardized SMAs are led by a health educator with a set order of TTIM topics. Patient-driven SMAs are delivered collaboratively by a multidisciplinary care team (health educator, medical provider, behavioral health provider, and a peer mentor); patients select the order and emphasis on TTIM topics. Invested in Diabetes is a cluster randomized pragmatic trial involving approximately 1440 adult patients with type 2 diabetes. Twenty primary care practices will be randomly assigned to either standardized or patient-driven SMAs. A mixed-methods evaluation will include quantitative (practice- and patient-level data) and qualitative (practice and patient interviews, observation) components. The primary patient-centered outcome is diabetes distress. Secondary outcomes include autonomy support, self-management behaviors, clinical outcomes, patient reach, and practice-level value and sustainability.

Discussion

Practice and patient stakeholder input guided protocol development for this pragmatic trial comparing SMA approaches. Implementation strategies from the enhanced Replicating Effective Programs framework will help ensure practices maintain fidelity to intervention protocols while tailoring workflows to their settings. Invested in Diabetes will contribute to the literature on chronic illness management and implementation science using the RE-AIM model.

Trial registration

ClinicalTrials.gov, NCT03590041. Registered on 5 July 2018.

Impact of baseline covariate imbalance on bias in treatment effect estimation in cluster randomized trials: Race as an example

Individual-level baseline covariate imbalance could happen more frequently in cluster randomized trials, and may influence the observed treatment effect. Using computer and real-data simulations, this paper quantifies the extent and impact of covariate imbalance on the estimated treatment effect for both continuous and binary outcomes, and relates it to the degree of imbalance for different numbers of clusters, cluster sizes, and covariate intraclass correlation coefficients. We focused on the impact of race as a covariate, given the emphasis of regulatory and funding bodies on understanding the influence of demographic characteristics on treatment effectiveness. We found that bias in the treatment effect is proportional to both the degree of baseline covariate imbalance and the covariate effect size. Larger numbers of clusters result in lower covariate imbalance, and increasing cluster size is less effective in reducing imbalance compared to increasing the number of clusters. Models adjusted for important baseline confounders are superior to unadjusted models for minimizing bias in both model-based simulations and an innovative simulation based on real clinical trial data. Higher outcome intraclass correlation coefficients did not affect bias but resulted in greater variance in treatment estimates.

Essential Ingredients and Innovations in the Design and Analysis of Group-Randomized Trials

This article reviews the essential ingredients and innovations in the design and analysis of group-randomized trials. The methods literature for these trials has grown steadily since they were introduced to the biomedical research community in the late 1970s, and we summarize those developments. We review, in addition to the group-randomized trial, methods for two closely related designs, the individually randomized group treatment trial and the stepped-wedge group-randomized trial. After describing the essential ingredients for these designs, we review the most important developments in the evolution of their methods using a new bibliometric tool developed at the National Institutes of Health. We then discuss the questions to be considered when selecting from among these designs or selecting the traditional randomized controlled trial. We close with a review of current methods for the analysis of data from these designs, a case study to illustrate each design, and a brief summary.

Cluster randomised trials with different numbers of measurements at baseline and endline: Sample size and optimal allocation

BACKGROUND/AIMS

Published methods for sample size calculation for cluster randomised trials with baseline data are inflexible and primarily assume an equal amount of data collected at baseline and endline, that is, before and after the intervention has been implemented in some clusters. We extend these methods to any amount of baseline and endline data. We explain how to explore sample size for a trial if some baseline data from the trial clusters have already been collected as part of a separate study. Where such data aren't available, we show how to choose the proportion of data collection devoted to the baseline within the trial, when a particular cluster size or range of cluster sizes is proposed.

METHODS

We provide a design effect given the cluster size and correlation parameters, assuming different participants are assessed at baseline and endline in the same clusters. We show how to produce plots to identify the impact of varying the amount of baseline data accounting for the inevitable uncertainty in the cluster autocorrelation. We illustrate the methodology using an example trial.

RESULTS

Baseline data provide more power, or allow a greater reduction in trial size, with greater values of the cluster size, intracluster correlation and cluster autocorrelation.

CONCLUSION

Investigators should think carefully before collecting baseline data in a cluster randomised trial if this is at the expense of endline data. In some scenarios, this will increase the sample size required to achieve given power and precision.

Estimating cluster-level local average treatment effects in cluster randomised trials with non-adherence

Non-adherence to assigned treatment is a common issue in cluster randomised trials. In these settings, the efficacy estimand may also be of interest. Many methodological contributions in recent years have advocated using instrumental variables to identify and estimate the local average treatment effect. However, the clustered nature of randomisation in cluster randomised trials adds to the complexity of such analyses. In this paper, we show that the local average treatment effect can be estimated via two-stage least squares regression using cluster-level summaries of the outcome and treatment received under certain assumptions. We propose the use of baseline variables to adjust the cluster-level summaries before performing two-stage least squares in order to improve efficiency. Implementation needs to account for the reduced sample size, as well as the possible heteroscedasticity, to obtain valid inferences. Simulations are used to assess the performance of two-stage least squares of cluster-level summaries under cluster-level or individual-level non-adherence, with and without weighting and robust standard errors. The impact of adjusting for baseline covariates and of appropriate degrees of freedom correction for inference is also explored. The methods are then illustrated by re-analysing a cluster randomised trial carried out in a specific UK primary care setting. Two-stage least squares estimation using cluster-level summaries provides estimates with small to negligible bias and coverage close to nominal level, provided the appropriate small sample degrees of freedom correction and robust standard errors are used for inference.

A Cluster-Randomized Trial of a Mobile Produce Market Program in 12 Communities in North Carolina: Program Development, Methods, and Baseline Characteristics

BACKGROUND

Mobile markets are an increasingly popular method for providing access to fresh fruits and vegetables (F/V) in underserved communities; however, evaluation of these programs is limited, as are descriptions of their development, study designs, and needs of the populations they serve.

OBJECTIVE

Our aim was to describe the development and theoretical basis for Veggie Van (VV), a mobile produce market intervention, the study design for the VV evaluation, and baseline characteristics of the study population.

DESIGN

The protocol and sample for a cluster-randomized controlled trial with 12 sites are described.

PARTICIPANTS/SETTING

Community partner organizations in the Triangle region of North Carolina that primarily served lower-income families or were located in areas that had limited access to fresh produce were recruited. Eligible individuals at each site (older than 18 years of age, self-identified as the main shoppers for their household, and expressed interest in using a mobile market) were targeted for enrollment. A total of 201 participants at 12 sites participated in the VV program and evaluation, which was implemented from November 2013 to March 2016.

MAIN OUTCOME MEASURES

Change in F/V intake (cups/day), derived from self-reported responses to the National Cancer Institute F/V screener, was the main outcome measure.

STATISTICAL ANALYSES PERFORMED

We performed a descriptive analysis of baseline sample characteristics.

RESULTS

Mean reported F/V intake was 3.4 cups/day. Participants reported generally having some access to fresh F/V, and 57.7% agreed they could afford enough F/V to feed their family. The most frequently cited barriers were cost (55.7%) and time to prepare F/V (20.4%). Self-efficacy was lowest for buying more F/V than usual and trying new vegetables.

CONCLUSIONS

By addressing cost and convenience and building skills for purchasing and preparing F/V, the VV has the potential to improve F/V consumption in underserved communities.

Reporting non-adherence in cluster randomised trials: A systematic review

BACKGROUND

Treatment non-adherence in randomised trials refers to situations where some participants do not receive their allocated treatment as intended. For cluster randomised trials, where the unit of randomisation is a group of participants, non-adherence may occur at the cluster or individual level. When non-adherence occurs, randomisation no longer guarantees that the relationship between treatment receipt and outcome is unconfounded, and the power to detect the treatment effects in intention-to-treat analysis may be reduced. Thus, recording adherence and estimating the causal treatment effect adequately are of interest for clinical trials.

OBJECTIVES

To assess the extent of reporting of non-adherence issues in published cluster trials and to establish which methods are currently being used for addressing non-adherence, if any, and whether clustering is accounted for in these.

METHODS

We systematically reviewed 132 cluster trials published in English in 2011 previously identified through a search in PubMed.

RESULTS

One-hundred and twenty three cluster trials were included in this systematic review. Non-adherence was reported in 56 cluster trials. Among these, 19 reported a treatment efficacy estimate: per protocol in 15 and as treated in 4. No study discussed the assumptions made by these methods, their plausibility or the sensitivity of the results to deviations from these assumptions.

LIMITATIONS

The year of publication of the cluster trials included in this review (2011) could be considered a limitation of this study; however, no new guidelines regarding the reporting and the handling of non-adherence for cluster trials have been published since. In addition, a single reviewer undertook the data extraction. To mitigate this, a second reviewer conducted a validation of the extraction process on 15 randomly selected reports. Agreement was satisfactory (93%).

CONCLUSION

Despite the recommendations of the Consolidated Standards of Reporting Trials statement extension to cluster randomised trials, treatment adherence is under-reported. Among the trials providing adherence information, there was substantial variation in how adherence was defined, handled and reported. Researchers should discuss the assumptions required for the results to be interpreted causally and whether these are scientifically plausible in their studies. Sensitivity analyses to study the robustness of the results to departures from these assumptions should be performed.

An evaluation of constrained randomization for the design and analysis of group-randomized trials with binary outcomes

Group-randomized trials are randomized studies that allocate intact groups of individuals to different comparison arms. A frequent practical limitation to adopting such research designs is that only a limited number of groups may be available, and therefore, simple randomization is unable to adequately balance multiple group-level covariates between arms. Therefore, covariate-based constrained randomization was proposed as an allocation technique to achieve balance. Constrained randomization involves generating a large number of possible allocation schemes, calculating a balance score that assesses covariate imbalance, limiting the randomization space to a prespecified percentage of candidate allocations, and randomly selecting one scheme to implement. When the outcome is binary, a number of statistical issues arise regarding the potential advantages of such designs in making inference. In particular, properties found for continuous outcomes may not directly apply, and additional variations on statistical tests are available. Motivated by two recent trials, we conduct a series of Monte Carlo simulations to evaluate the statistical properties of model-based and randomization-based tests under both simple and constrained randomization designs, with varying degrees of analysis-based covariate adjustment. Our results indicate that constrained randomization improves the power of the linearization F-test, the KC-corrected GEE t-test (Kauermann and Carroll, 2001, Journal of the American Statistical Association 96, 1387-1396), and two permutation tests when the prognostic group-level variables are controlled for in the analysis and the size of randomization space is reasonably small. We also demonstrate that constrained randomization reduces power loss from redundant analysis-based adjustment for non-prognostic covariates. Design considerations such as the choice of the balance metric and the size of randomization space are discussed.

Review of Recent Methodological Developments in Group-Randomized Trials: Part 1-Design

In 2004, Murray et al. reviewed methodological developments in the design and analysis of group-randomized trials (GRTs). We have highlighted the developments of the past 13 years in design with a companion article to focus on developments in analysis. As a pair, these articles update the 2004 review. We have discussed developments in the topics of the earlier review (e.g., clustering, matching, and individually randomized group-treatment trials) and in new topics, including constrained randomization and a range of randomized designs that are alternatives to the standard parallel-arm GRT. These include the stepped-wedge GRT, the pseudocluster randomized trial, and the network-randomized GRT, which, like the parallel-arm GRT, require clustering to be accounted for in both their design and analysis.

Economic Evaluations Alongside Efficient Study Designs Using Large Observational Datasets: the PLEASANT Trial Case Study

BACKGROUND

Large observational datasets such as Clinical Practice Research Datalink (CPRD) provide opportunities to conduct clinical studies and economic evaluations with efficient designs.

OBJECTIVES

Our objectives were to report the economic evaluation methodology for a cluster randomised controlled trial (RCT) of a UK NHS-delivered public health intervention for children with asthma that was evaluated using CPRD and describe the impact of this methodology on results.

METHODS

CPRD identified eligible patients using predefined asthma diagnostic codes and captured 1-year pre- and post-intervention healthcare contacts (August 2012 to July 2014). Quality-adjusted life-years (QALYs) 4 months post-intervention were estimated by assigning utility values to exacerbation-related contacts; a systematic review identified these utility values because preference-based outcome measures were not collected. Bootstrapped costs were evaluated 12 months post-intervention, both with 1-year regression-based baseline adjustment (BA) and without BA (observed).

RESULTS

Of 12,179 patients recruited, 8190 (intervention 3641; control 4549) were evaluated in the primary analysis, which included patients who received the protocol-defined intervention and for whom CPRD data were available. The intervention's per-patient incremental QALY loss was 0.00017 (bias-corrected and accelerated 95% confidence intervals [BCa 95% CI] -0.00051 to 0.00018) and cost savings were £14.74 (observed; BCa 95% CI -75.86 to 45.19) or £36.07 (BA; BCa 95% CI -77.11 to 9.67), respectively. The probability of cost savings was much higher when accounting for BA versus observed costs due to baseline cost differences between trial arms (96.3 vs. 67.3%, respectively).

CONCLUSION

Economic evaluations using data from a large observational database without any primary data collection is feasible, informative and potentially efficient. Clinical Trials Registration Number: ISRCTN03000938.

Effectiveness of the EMPOWER-PAR Intervention in Improving Clinical Outcomes of Type 2 Diabetes Mellitus in Primary Care: A Pragmatic Cluster Randomised Controlled Trial

BACKGROUND

The chronic care model was proven effective in improving clinical outcomes of diabetes in developed countries. However, evidence in developing countries is scarce. The objective of this study was to evaluate the effectiveness of EMPOWER-PAR intervention (based on the chronic care model) in improving clinical outcomes for type 2 diabetes mellitus using readily available resources in the Malaysian public primary care setting.

METHODS

This was a pragmatic, cluster-randomised, parallel, matched pair, controlled trial using participatory action research approach, conducted in 10 public primary care clinics in Malaysia. Five clinics were randomly selected to provide the EMPOWER-PAR intervention for 1 year and another five clinics continued with usual care. Patients who fulfilled the criteria were recruited over a 2-week period by each clinic. The obligatory intervention components were designed based on four elements of the chronic care model i.e. healthcare organisation, delivery system design, self-management support and decision support. The primary outcome was the change in the proportion of patients achieving HbA1c < 6.5%. Secondary outcomes were the change in proportion of patients achieving targets for blood pressure, lipid profile, body mass index and waist circumference. Intention to treat analysis was performed for all outcome measures. A generalised estimating equation method was used to account for baseline differences and clustering effect.

RESULTS

A total of 888 type 2 diabetes mellitus patients were recruited at baseline (intervention: 471 vs.

CONTROL

417). At 1-year, 96.6 and 97.8% of patients in the intervention and control groups completed the study, respectively. The baseline demographic and clinical characteristics of both groups were comparable. The change in the proportion of patients achieving HbA1c target was significantly higher in the intervention compared to the control group (intervention: 3.0% vs.

CONTROL

-4.1%, P < 0.002). Patients who received the EMPOWER-PAR intervention were twice more likely to achieve HbA1c target compared to those in the control group (adjusted OR 2.16, 95% CI 1.34-3.50, P < 0.002). However, there was no significant improvement found in the secondary outcomes.

CONCLUSIONS

This study demonstrates that the EMPOWER-PAR intervention was effective in improving the primary outcome for type 2 diabetes in the Malaysian public primary care setting.

TRIAL REGISTRATION

Registered with: ClinicalTrials.gov.: NCT01545401 . Date of registration: 1st March 2012.

The effect of a patient centred care bundle intervention on pressure ulcer incidence (INTACT): A cluster randomised trial

BACKGROUND

Hospital-acquired pressure ulcers are a serious patient safety concern, associated with poor patient outcomes and high healthcare costs. They are also viewed as an indicator of nursing care quality.

OBJECTIVE

To evaluate the effectiveness of a pressure ulcer prevention care bundle in preventing hospital-acquired pressure ulcers among at risk patients.

DESIGN

Pragmatic cluster randomised trial.

SETTING

Eight tertiary referral hospitals with >200 beds each in three Australian states.

PARTICIPANTS

1600 patients (200/hospital) were recruited. Patients were eligible if they were: ≥18 years old; at risk of pressure ulcer because of limited mobility; expected to stay in hospital ≥48h and able to read English.

METHODS

Hospitals (clusters) were stratified in two groups by recent pressure ulcer rates and randomised within strata to either a pressure ulcer prevention care bundle or standard care. The care bundle was theoretically and empirically based on patient participation and clinical practice guidelines. It was multi-component, with three messages for patients' participation in pressure ulcer prevention care: keep moving; look after your skin; and eat a healthy diet. Training aids for patients included a DVD, brochure and poster. Nurses in intervention hospitals were trained in partnering with patients in their pressure ulcer prevention care. The statistician, recruiters, and outcome assessors were blinded to group allocation and interventionists blinded to the study hypotheses, tested at both the cluster and patient level. The primary outcome, incidence of hospital-acquired pressure ulcers, which applied to both the cluster and individual participant level, was measured by daily skin inspection.

RESULTS

Four clusters were randomised to each group and 799 patients per group analysed. The intraclass correlation coefficient was 0.035. After adjusting for clustering and pre-specified covariates (age, pressure ulcer present at baseline, body mass index, reason for admission, residence and number of comorbidities on admission), the hazard ratio for new pressure ulcers developed (pressure ulcer prevention care bundle relative to standard care) was 0.58 (95% CI: 0.25, 1.33; p=0.198). No adverse events or harms were reported.

CONCLUSIONS

Although the pressure ulcer prevention care bundle was associated with a large reduction in the hazard of ulceration, there was a high degree of uncertainty around this estimate and the difference was not statistically significant. Possible explanations for this non-significant finding include that the pressure ulcer prevention care bundle was effective but the sample size too small to detect this.

Propensity score to detect baseline imbalance in cluster randomized trials: the role of the c-statistic

Background

Despite randomization, baseline imbalance and confounding bias may occur in cluster randomized trials (CRTs). Covariate imbalance may jeopardize the validity of statistical inferences if they occur on prognostic factors. Thus, the diagnosis of a such imbalance is essential to adjust statistical analysis if required.

Methods

We developed a tool based on the c-statistic of the propensity score (PS) model to detect global baseline covariate imbalance in CRTs and assess the risk of confounding bias. We performed a simulation study to assess the performance of the proposed tool and applied this method to analyze the data from 2 published CRTs.

Results

The proposed method had good performance for large sample sizes (n =500 per arm) and when the number of unbalanced covariates was not too small as compared with the total number of baseline covariates (≥40 % of unbalanced covariates). We also provide a strategy for pre selection of the covariates needed to be included in the PS model to enhance imbalance detection.

Conclusion

The proposed tool could be useful in deciding whether covariate adjustment is required before performing statistical analyses of CRTs.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-015-0100-4) contains supplementary material, which is available to authorized users.

The Method of Randomization for Cluster-Randomized Trials: Challenges of Including Patients with Multiple Chronic Conditions

Cluster-randomized clinical trials (CRT) are trials in which the unit of randomization is not a participant but a group (e.g. healthcare systems or community centers). They are suitable when the intervention applies naturally to the cluster (e.g. healthcare policy); when lack of independence among participants may occur (e.g. nursing home hygiene); or when it is most ethical to apply an intervention to all within a group (e.g. school-level immunization). Because participants in the same cluster receive the same intervention, CRT may approximate clinical practice, and may produce generalizable findings. However, when not properly designed or interpreted, CRT may induce biased results. CRT designs have features that add complexity to statistical estimation and inference. Chief among these is the cluster-level correlation in response measurements induced by the randomization. A critical consideration is the experimental unit of inference; often it is desirable to consider intervention effects at the level of the individual rather than the cluster. Finally, given that the number of clusters available may be limited, simple forms of randomization may not achieve balance between intervention and control arms at either the cluster- or participant-level. In non-clustered clinical trials, balance of key factors may be easier to achieve because the sample can be homogenous by exclusion of participants with multiple chronic conditions (MCC). CRTs, which are often pragmatic, may eschew such restrictions. Failure to account for imbalance may induce bias and reducing validity. This article focuses on the complexities of randomization in the design of CRTs, such as the inclusion of patients with MCC, and imbalances in covariate factors across clusters.