Adherence to key recommendations for design and analysis of stepped-wedge cluster randomized trials: A review of trials published 2016-2022

BACKGROUND/AIMS

The stepped-wedge cluster randomized trial (SW-CRT), in which clusters are randomized to a time at which they will transition to the intervention condition - rather than a trial arm - is a relatively new design. SW-CRTs have additional design and analytical considerations compared to conventional parallel arm trials. To inform future methodological development, including guidance for trialists and the selection of parameters for statistical simulation studies, we conducted a review of recently published SW-CRTs. Specific objectives were to describe (1) the types of designs used in practice, (2) adherence to key requirements for statistical analysis, and (3) practices around covariate adjustment. We also examined changes in adherence over time and by journal impact factor.

METHODS

We used electronic searches to identify primary reports of SW-CRTs published 2016-2022. Two reviewers extracted information from each trial report and its protocol, if available, and resolved disagreements through discussion.

RESULTS

We identified 160 eligible trials, randomizing a median (Q1-Q3) of 11 (8-18) clusters to 5 (4-7) sequences. The majority (122, 76%) were cross-sectional (almost all with continuous recruitment), 23 (14%) were closed cohorts and 15 (9%) open cohorts. Many trials had complex design features such as multiple or multivariate primary outcomes (50, 31%) or time-dependent repeated measures (27, 22%). The most common type of primary outcome was binary (51%); continuous outcomes were less common (26%). The most frequently used method of analysis was a generalized linear mixed model (112, 70%); generalized estimating equations were used less frequently (12, 8%). Among 142 trials with fewer than 40 clusters, only 9 (6%) reported using methods appropriate for a small number of clusters. Statistical analyses clearly adjusted for time effects in 119 (74%), for within-cluster correlations in 132 (83%), and for distinct between-period correlations in 13 (8%). Covariates were included in the primary analysis of the primary outcome in 82 (51%) and were most often individual-level covariates; however, clear and complete pre-specification of covariates was uncommon. Adherence to some key methodological requirements (adjusting for time effects, accounting for within-period correlation) was higher among trials published in higher versus lower impact factor journals. Substantial improvements over time were not observed although a slight improvement was observed in the proportion accounting for a distinct between-period correlation.

CONCLUSIONS

Future methods development should prioritize methods for SW-CRTs with binary or time-to-event outcomes, small numbers of clusters, continuous recruitment designs, multivariate outcomes, or time-dependent repeated measures. Trialists, journal editors, and peer reviewers should be aware that SW-CRTs have additional methodological requirements over parallel arm designs including the need to account for period effects as well as complex intracluster correlations.

The staircase cluster randomised trial design: A pragmatic alternative to the stepped wedge

This article introduces the 'staircase' design, derived from the zigzag pattern of steps along the diagonal of a stepped wedge design schematic where clusters switch from control to intervention conditions. Unlike a complete stepped wedge design where all participating clusters must collect and provide data for the entire trial duration, clusters in a staircase design are only required to be involved and collect data for a limited number of pre- and post-switch periods. This could alleviate some of the burden on participating clusters, encouraging involvement in the trial and reducing the likelihood of attrition. Staircase designs are already being implemented, although in the absence of a dedicated methodology, approaches to sample size and power calculations have been inconsistent. We provide expressions for the variance of the treatment effect estimator when a linear mixed model for an outcome is assumed for the analysis of staircase designs in order to enable appropriate sample size and power calculations. These include explicit variance expressions for basic staircase designs with one pre- and one post-switch measurement period. We show how the variance of the treatment effect estimator is related to key design parameters and demonstrate power calculations for examples based on a real trial.

Does it decay? Obtaining decaying correlation parameter values from previously analysed cluster randomised trials

A frequently applied assumption in the analysis of data from cluster randomised trials is that the outcomes from all participants within a cluster are equally correlated. That is, the intracluster correlation, which describes the degree of dependence between outcomes from participants in the same cluster, is the same for each pair of participants in a cluster. However, recent work has discussed the importance of allowing for this correlation to decay as the time between the measurement of participants in a cluster increases. Incorrect omission of such a decay can lead to under-powered studies, and confidence intervals for estimated treatment effects can be too narrow or too wide, depending on the characteristics of the design. When planning studies, researchers often rely on previously reported analyses of trials to inform their choice of intracluster correlation. However, most reported analyses of clustered data do not incorporate a correlation decay. Thus, often all that is available are estimates of intracluster correlations obtained under the potentially incorrect assumption of no decay. In this article, we show that it is possible to use intracluster correlation values obtained from models that incorrectly omit a decay to inform plausible choices of decaying correlations. Our focus is on intracluster correlation estimates for continuous outcomes obtained by fitting linear mixed models with exchangeable or block-exchangeable correlation structures. We describe how plausible values for decaying correlations may be obtained given these estimated intracluster correlations. An online app is presented that allows users to obtain plausible values of the decay, which can be used at the trial planning stage to assess the sensitivity of sample size and power calculations to decaying correlation structures.

Intraclass Correlations Values in International Development: Evidence Across Commonly Studied Domains in sub-Saharan Africa

The sharp increase in the number of experimental studies evaluating development programs raises the need for accurate intraclass correlations (ICC) to conduct power calculations so that researchers can design studies to detect meaningful effects with sufficient statistical power. The intraclass correlation is an important parameter for determining the statistical power of cluster-randomized trials. The parameter is rarely available to researchers planning a study until after the design is set and data are already collected. This paper takes an important step towards helping researchers working in sub-Saharan Africa to accurately estimate appropriate sample sizes for their clustered RCTs. The study draws from rich data sets in Kenya, Malawi, Zambia, and Zimbabwe. We present ICCs for a wide range of domains common for development research. Our results suggest that ICCs for commonly studied indicators in sub-Saharan Africa are lower than is often assumed in power calculations. ICC values are especially low for indicators associated with child nutrition and food security, suggesting that cluster-RCTs might be a viable design even when faced with limited budgets because sample size requirements are not much different from an individual random assignment design.

The batched stepped wedge design: A design robust to delays in cluster recruitment

Stepped wedge designs are an increasingly popular variant of longitudinal cluster randomized trial designs, and roll out interventions across clusters in a randomized, but step-wise fashion. In the standard stepped wedge design, assumptions regarding the effect of time on outcomes may require that all clusters start and end trial participation at the same time. This would require ethics approvals and data collection procedures to be in place in all clusters before a stepped wedge trial can start in any cluster. Hence, although stepped wedge designs are useful for testing the impacts of many cluster-based interventions on outcomes, there can be lengthy delays before a trial can commence. In this article, we introduce "batched" stepped wedge designs. Batched stepped wedge designs allow clusters to commence the study in batches, instead of all at once, allowing for staggered cluster recruitment. Like the stepped wedge, the batched stepped wedge rolls out the intervention to all clusters in a randomized and step-wise fashion: a series of self-contained stepped wedge designs. Provided that separate period effects are included for each batch, software for standard stepped wedge sample size calculations can be used. With this time parameterization, in many situations including when linear models are assumed, sample size calculations reduce to the setting of a single stepped wedge design with multiple clusters per sequence. In these situations, sample size calculations will not depend on the delays between the commencement of batches. Hence, the power of batched stepped wedge designs is robust to unexpected delays between batches.

Methodological challenges in pragmatic trials in Alzheimer's disease and related dementias: Opportunities for improvement

BACKGROUND AND AIMS

We need more pragmatic trials of interventions to improve care and outcomes for people living with Alzheimer's disease and related dementias. However, these trials present unique methodological challenges in their design, analysis, and reporting-often, due to the presence of one or more sources of clustering. Failure to account for clustering in the design and analysis can lead to increased risks of Type I and Type II errors. We conducted a review to describe key methodological characteristics and obtain a "baseline assessment" of methodological quality of pragmatic trials in dementia research, with a view to developing new methods and practical guidance to support investigators and methodologists conducting pragmatic trials in this field.

METHODS

We used a published search filter in MEDLINE to identify trials more likely to be pragmatic and identified a subset that focused on people living with Alzheimer's disease or other dementias or included them as a defined subgroup. Pairs of reviewers extracted descriptive information and key methodological quality indicators from each trial.

RESULTS

We identified N = 62 eligible primary trial reports published across 36 different journals. There were 15 (24%) individually randomized, 38 (61%) cluster randomized, and 9 (15%) individually randomized group treatment designs; 54 (87%) trials used repeated measures on the same individual and/or cluster over time and 17 (27%) had a multivariate primary outcome (e.g. due to measuring an outcome on both the patient and their caregiver). Of the 38 cluster randomized trials, 16 (42%) did not report sample size calculations accounting for the intracluster correlation and 13 (34%) did not account for intracluster correlation in the analysis. Of the 9 individually randomized group treatment trials, 6 (67%) did not report sample size calculations accounting for intracluster correlation and 8 (89%) did not account for it in the analysis. Of the 54 trials with repeated measurements, 45 (83%) did not report sample size calculations accounting for repeated measurements and 19 (35%) did not utilize at least some of the repeated measures in the analysis. No trials accounted for the multivariate nature of their primary outcomes in sample size calculation; only one did so in the analysis.

CONCLUSION

There is a need and opportunity to improve the design, analysis, and reporting of pragmatic trials in dementia research. Investigators should pay attention to the potential presence of one or more sources of clustering. While methods for longitudinal and cluster randomized trials are well developed, accessible resources and new methods for dealing with multiple sources of clustering are required. Involvement of a statistician with expertise in longitudinal and clustered designs is recommended.

Information content of stepped wedge designs with unequal cluster-period sizes in linear mixed models: Informing incomplete designs

In practice, stepped wedge trials frequently include clusters of differing sizes. However, investigations into the theoretical aspects of stepped wedge designs have, until recently, typically assumed equal numbers of subjects in each cluster and in each period. The information content of the cluster-period cells, clusters, and periods of stepped wedge designs has previously been investigated assuming equal cluster-period sizes, and has shown that incomplete stepped wedge designs may be efficient alternatives to the full stepped wedge. How this changes when cluster-period sizes are not equal is unknown, and we investigate this here. Working within the linear mixed model framework, we show that the information contributed by design components (clusters, sequences, and periods) does depend on the sizes of each cluster-period. Using a particular trial that assessed the impact of an individual education intervention on log-length of stay in rehabilitation units, we demonstrate how strongly the efficiency of incomplete designs depends on which cells are excluded: smaller incomplete designs may be more powerful than alternative incomplete designs that include a greater total number of participants. This also serves to demonstrate how the pattern of information content can be used to inform a set of incomplete designs to be considered as alternatives to the complete stepped wedge design. Our theoretical results for the information content can be extended to a broad class of longitudinal (ie, multiple period) cluster randomized trial designs.

Information content of stepped-wedge designs when treatment effect heterogeneity and/or implementation periods are present

Stepped-wedge cluster randomized trials, which randomize clusters of subjects to treatment sequences in which clusters switch from control to intervention conditions, are being conducted with increasing frequency. Due to the real-world nature of this design, methodological and implementation challenges are ubiquitous. To account for such challenges, more complex statistical models to plan studies and analyze data are required. In this paper, we consider stepped-wedge trials that accommodate treatment effect heterogeneity across clusters, implementation periods during which no data are collected, or both treatment effect heterogeneity and implementation periods. Previous work has shown that the sequence-period cells of a stepped-wedge design contribute unequal amounts of information to the estimation of the treatment effect. In this paper, we extend that work by considering the amount of information available for the estimation of the treatment effect in each sequence-period cell, sequence, and period of stepped-wedge trials with more complex designs and outcome models. When either treatment effect heterogeneity and/or implementation periods are present, the pattern of information content of sequence-period cells tends to be clustered around the times of the switch from control to intervention condition, similarly to when these complexities are absent. However, the presence and degree of treatment effect heterogeneity and the number of implementation periods can influence the information content of periods and sequences markedly.

Inference for the treatment effect in multiple-period cluster randomised trials when random effect correlation structure is misspecified

Multiple-period cluster randomised trials, such as stepped wedge or cluster cross-over trials, are being conducted with increasing frequency. In the design and analysis of these trials, it is necessary to specify the form of the within-cluster correlation structure, and a common assumption is that the correlation between the outcomes of any pair of subjects within a cluster is identical. More complex models that allow for correlations within a cluster to decay over time have recently been suggested. However, most software packages cannot fit these models. As a result, practitioners may choose a simpler model. We analytically examine the impact of incorrectly omitting a decay in correlation on the variance of the treatment effect estimator and show that misspecification of the within-cluster correlation structure can lead to incorrect conclusions regarding estimated treatment effects for stepped wedge and cluster crossover trials.

Accounting for twin births in sample size calculations for randomised trials

BACKGROUND

Including twins in randomised trials leads to non-independence or clustering in the data. Clustering has important implications for sample size calculations, yet few trials take this into account. Estimates of the intracluster correlation coefficient (ICC), or the correlation between outcomes of twins, are needed to assist with sample size planning. Our aims were to provide ICC estimates for infant outcomes, describe the information that must be specified in order to account for clustering due to twins in sample size calculations, and develop a simple tool for performing sample size calculations for trials including twins.

METHODS

ICCs were estimated for infant outcomes collected in four randomised trials that included twins. The information required to account for clustering due to twins in sample size calculations is described. A tool that calculates the sample size based on this information was developed in Microsoft Excel and in R as a Shiny web app.

RESULTS

ICC estimates ranged between -0.12, indicating a weak negative relationship, and 0.98, indicating a strong positive relationship between outcomes of twins. Example calculations illustrate how the ICC estimates and sample size calculator can be used to determine the target sample size for trials including twins.

CONCLUSIONS

Clustering among outcomes measured on twins should be taken into account in sample size calculations to obtain the desired power. Our ICC estimates and sample size calculator will be useful for designing future trials that include twins. Publication of additional ICCs is needed to further assist with sample size planning for future trials.

Using threshold regression to analyze survival data from complex surveys: With application to mortality linked NHANES III Phase II genetic data

The Cox proportional hazards (PH) model is a common statistical technique used for analyzing time-to-event data. The assumption of PH, however, is not always appropriate in real applications. In cases where the assumption is not tenable, threshold regression (TR) and other survival methods, which do not require the PH assumption, are available and widely used. These alternative methods generally assume that the study data constitute simple random samples. In particular, TR has not been studied in the setting of complex surveys that involve (1) differential selection probabilities of study subjects and (2) intracluster correlations induced by multistage cluster sampling. In this paper, we extend TR procedures to account for complex sampling designs. The pseudo-maximum likelihood estimation technique is applied to estimate the TR model parameters. Computationally efficient Taylor linearization variance estimators that consider both the intracluster correlation and the differential selection probabilities are developed. The proposed methods are evaluated by using simulation experiments with various complex designs and illustrated empirically by using mortality-linked Third National Health and Nutrition Examination Survey Phase II genetic data.

Simulation-based evaluation of the linear-mixed model in the presence of an increasing proportion of singletons

Data in medical sciences often have a hierarchical structure with lower level units (e.g. children) nested in higher level units (e.g. departments). Several specific but frequently studied settings, mainly in longitudinal and family research, involve a large number of units that tend to be quite small, with units containing only one element referred to as singletons. Regardless of sparseness, hierarchical data should be analyzed with appropriate methodology such as, for example linear-mixed models. Using a simulation study, based on the structure of a data example on Ceftriaxone consumption in hospitalized children, we assess the impact of an increasing proportion of singletons (0-95%), in data with a low, medium, or high intracluster correlation, on the stability of linear-mixed models parameter estimates, confidence interval coverage and F test performance. Some techniques that are frequently used in the presence of singletons include ignoring clustering, dropping the singletons from the analysis and grouping the singletons into an artificial unit. We show that both the fixed and random effects estimates and their standard errors are stable in the presence of an increasing proportion of singletons. We demonstrate that ignoring clustering and dropping singletons should be avoided as they come with biased standard error estimates. Grouping the singletons into an artificial unit might be considered, although the linear-mixed model performs better even when the proportion of singletons is high. We conclude that the linear-mixed model is stable in the presence of singletons when both lower- and higher level sample sizes are fixed. In this setting, the use of remedial measures, such as ignoring clustering and grouping or removing singletons, should be dissuaded.

A composite likelihood approach in testing for Hardy Weinberg Equilibrium using family-based genetic survey data

In population-based household surveys, for example, the National Health and Nutrition Examination Survey, households are often sampled by stratified multistage cluster sampling, and multiple individuals related by blood are often sampled within households. Therefore, genetic data collected from these population-based household surveys, called National Genetic Household Surveys, can be correlated because of two levels of correlation. One level of correlation is caused by the multistage geographical cluster sampling and the other is caused by biological inheritance among participants within the same sampled family. In this paper, we develop an efficient Hardy Weinberg Equilibrium (HWE) test utilizing pairwise composite likelihood methods that incorporate the sample weighting effect induced by the differential selection probabilities in complex sample designs, as well as the two-level clustering (correlation) effects described above. Monte Carlo simulation studies show that the proposed HWE test maintains the nominal levels, and is more powerful than existing methods (Li et al. 2011) under various (non)informative sample designs that depend on genotypes (explicitly or implicitly), family relationships or both, especially when within-household sampling depends on the genotypes. The developed tests are further evaluated using simulated genetic data based on the Hispanic Health and Nutrition Survey. Copyright © 2016 John Wiley & Sons, Ltd.

Nonparametric Sample Size Estimation for Sensitivity and Specificity with Multiple Observations per Subject

We propose a sample size calculation approach for the estimation of sensitivity and specificity of diagnostic tests with multiple observations per subjects. Many diagnostic tests such as diagnostic imaging or periodontal tests are characterized by the presence of multiple observations for each subject. The number of observations frequently varies among subjects in diagnostic imaging experiments or periodontal studies. Nonparametric statistical methods for the analysis of clustered binary data have been recently developed by various authors. In this paper, we derive a sample size formula for sensitivity and specificity of diagnostic tests using the sign test while accounting for multiple observations per subjects. Application of the sample size formula for the design of a diagnostic test is discussed. Since the sample size formula is based on large sample theory, simulation studies are conducted to evaluate the finite sample performance of the proposed method. We compare the performance of the proposed sample size formula with that of the parametric sample size formula that assigns equal weight to each observation. Simulation studies show that the proposed sample size formula generally yields empirical powers closer to the nominal level than the parametric method. Simulation studies also show that the number of subjects required increases as the variability in the number of observations per subject increases and the intracluster correlation increases.