Stepped wedge cluster randomised trials: a review of the statistical methodology used and available

Inference for the treatment effect in staircase designs with continuous outcomes: a simulation study

BACKGROUND

Staircase designs are incomplete stepped wedge designs that, unlike standard stepped wedge designs, require clusters to contribute data for only a limited number of trial periods. Previous work has provided formulae based on asymptotic results for the calculation of the power of staircase designs to detect treatment effects of interest.

METHODS

We conduct a simulation study to assess the finite sample performance of these formulae, and the impact of misspecifying the correlation structure when analysing data from staircase designs on inference for the treatment effect, under a range of realistic trial settings. This study focuses on basic staircase designs with one control period followed by one intervention period in each sequence. We simulate staircase trial datasets with continuous outcomes and a repeated cross-sectional measurement scheme under exchangeable and block-exchangeable intracluster correlation structures, and then fit linear mixed models with linear and categorical time period effects. For settings with a small number of clusters, Kenward-Roger and Satterthwaite small-sample corrections are applied. Comparisons are made between nominal and observed Type I error rates, and theoretically-derived study power and empirical power. The impact on inference for the treatment effect when misspecifying the intracluster correlation structure is assessed through considering performance metrics including bias and 95% confidence interval coverage.

RESULTS

Data analysis assuming an exchangeable correlation structure and application of the Satterthwaite correction controls Type I error well when the correlation structure is correctly specified, and there are a sufficient number of clusters. For the true block-exchangeable model, when fitting the correct model with the Satterthwaite correction, the observed Type I error (empirical power) can be higher (lower) than the nominal (i.e., theoretical) value when there is only 1 cluster per sequence, but otherwise, it aligns well with the nominal (theoretical) value. Misspecification of the correlation structure (fitting an exchangeable model when the true structure is block-exchangeable) can lead to inflated Type I error and poor confidence interval coverage.

CONCLUSIONS

Staircase designs with one cluster per sequence should be used with caution. Additionally, using a correlation structure that allows for decay is preferable for making valid inferences for the estimation of the treatment effect.

Protocol to evaluate the effectiveness of the implementation of transdiagnostic cognitive behavioural therapy for emotional disorders in primary care and its mechanisms of change: a randomized step-wedge clinical trial (PsicAP-CV)

INTRODUCTION

Emotional disorders (ED) are highly prevalent worldwide. The PsicAP trial, conducted in Spain, demonstrated the benefits of adding transdiagnostic cognitive behavioural therapy (TD-CBT) to treatment as usual (TAU) for the attention of these disorders in primary care (PC). Here we describe the design of a stepped wedge randomized controlled trial (RCT), inspired by the PsicAP project. This RCT has two main aims: 1) to test the implementation of the PsicAP protocol in a real clinical setting, further evaluating possible mechanisms of change underlying the efficacy of TD-CBT (emotional regulation, alliance, and therapist experience and training), and 2) to assess the impact of psychotropic medication use on neuropsychological function and treatment outcomes.

METHODS

A single-blind multicentre RCT with a stepped wedge design will be conducted. Participants (N=320) will be randomly assigned to an experimental group (EG1) or to a waiting list group (WG). The EG1 will receive immediate treatment and the WG will remain on the waiting list for 3 months. After this time, the WG will become a second experimental group (EG2) that will receive the same treatment as EG1 (PsicAP protocol). Patients will be assessed at post-treatment, at 3 and 9 months. Before starting treatment, a random subsample of patients (n=90) will undergo a neuropsychological assessment. These patients will be assigned to three groups based on their use of psychotropic medication at the time of randomization: no psychotropic medication, short-term use (< 3 months) and long-term use (≥ 3 months). All 90 participants will undergo the same neuropsychological assessment at one year. The RCT is expected to run from 01/05/23 to 01/10/25.

DISCUSSION

The results of this trial are expected to provide further support for the efficacy of the PsicAP TD-CBT protocol, as well as insight into the mechanisms of change that lead to the positive therapeutic outcomes of this protocol. In addition, this study will help determine the effects of short- and long-term psychotropic use on neuropsychological function and therapeutic outcomes. In short, it is hoped that this RCT will help to better understand how to implement evidence-based psychological treatment in the PC setting.

TRIAL REGISTRATION

EURADICT 2013-001955-11/ ISRCTN58437086.

Implementation of an electronic medication management support system in hospitalised polypharmacy patients: study protocol of a stepped-wedge cluster-randomised controlled trial (TOP study)

INTRODUCTION

Polypharmacy is associated with an increased risk of adverse patient outcomes across various settings, including inpatient care. To enhance the appropriateness of medication therapy management for patients during hospital stays, computerised interventions have shown promise with regard to patient safety. This study assesses whether the implementation of a clinical decision support system will optimise the process of inpatient medication therapy to prevent inappropriate medication use and thus promote patient safety.

METHODS AND ANALYSIS

The intervention will be evaluated in a prospective, cluster-randomised controlled trial using a stepped-wedge design. The study will be conducted in 12 hospitals across Germany over a total period of 33 months. Patients will be treated according to the group status of the hospital and receive either standard care or the Transsektorale Optimierung der Patientensicherheit or trans-sectoral optimisation of patient safety intervention. The primary outcome is the combined endpoint of all-cause mortality and all-cause hospitalisation. Secondary endpoints are, for example, inappropriate prescriptions, utilisation of different health services, cost-effectiveness, as well as patient-reported outcome measures. Parameters describing the attitudes of patients and healthcare professionals towards the intervention and organisational change processes will be collected as part of the process evaluation. The primary endpoint will be evaluated using hospital and outpatient claims data from participating statutory health insurances at the population level. There are multiple secondary endpoints with data linkage of primary and secondary data at study participant level. Statistical analysis will make use of (generalised) linear mixed models or generalised estimating equations, taking account of independent covariables. All data analyses of the process evaluation will be descriptive and explorative.

ETHICS AND DISSEMINATION

Data collection, storage and evaluation meet all applicable data protection regulations. The trial has been approved by the Ethics Committees of the University of Wuppertal and the Medical Association of Saarland, Germany. Results will be disseminated through workshops, peer-reviewed publications and local and international conferences.

TRIAL REGISTRATION NUMBER

DRKS00025485.

Active support as good support in group homes? A longitudinal interview study with service users

BACKGROUND

Support from staff plays an important role in quality of life for people with intellectual disability. This study focuses on service users' views of Active Support as good support and whether Active Support increases the quality of everyday support in group homes.

METHOD

Nine service users were interviewed at baseline and at follow-up one year after staff received Active Support training. Thematic analyses were used to develop themes.

RESULTS

Three main themes were created: (1) Home is more than just a place, it is a feeling; (2) Good care is caring with accessible communication; and (3) Time is precious. Participants were more satisfied with most of the support at follow-up, particularly choice-making, control in everyday life, relationships, and emotional support.

CONCLUSIONS

Active Support corresponds well with service users' perceptions of good support. In future studies of Active Support service users' voices should be a part of the follow-up.

Bayesian Hierarchical Penalized Spline Models for Immediate and Time-Varying Intervention Effects in Stepped Wedge Cluster Randomized Trials

Stepped wedge cluster randomized trials (SWCRTs) often face challenges related to potential confounding by time. Traditional frequentist methods may not provide adequate coverage of an intervention's true effect using confidence intervals, whereas Bayesian approaches show potential for better coverage of intervention effects. However, Bayesian methods remain underexplored in the context of SWCRTs. To bridge this gap, we propose two innovative Bayesian hierarchical penalized spline models. Our first model accommodates large numbers of clusters and time periods, focusing on immediate intervention effects. To evaluate this approach, we compared this model to traditional frequentist methods. We then extend our approach to account for time-varying intervention effects, conducting a comprehensive comparison with an existing Bayesian monotone effect curve model and alternative frequentist methods. The proposed models were applied in the Primary Palliative Care for Emergency Medicine stepped wedge trial to evaluate the effectiveness of the intervention. Through extensive simulations and real-world application, we demonstrate the robustness of our proposed Bayesian models. Notably, the Bayesian immediate effect model consistently achieves the nominal coverage probability, providing more reliable interval estimations while maintaining high estimation accuracy. Furthermore, our proposed Bayesian time-varying effect model represents a significant advancement over the existing Bayesian monotone effect curve model, offering improved accuracy and reliability in estimation while also achieving higher coverage probability than alternative frequentist methods. To the best of our knowledge, this marks the first development of Bayesian hierarchical spline modeling for SWCRTs. Our proposed models offer promising tools for researchers and practitioners, enabling more precise evaluation of intervention impacts.

A review of current practice in the design and analysis of extremely small stepped-wedge cluster randomized trials

BACKGROUND/AIMS

Stepped-wedge cluster randomized trials tend to require fewer clusters than standard parallel-arm designs due to the switches between control and intervention conditions, but there are no recommendations for the minimum number of clusters. Trials randomizing an extremely small number of clusters are not uncommon, but the justification for small numbers of clusters is often unclear and appropriate analysis is often lacking. In addition, stepped-wedge cluster randomized trials are methodologically more complex due to their longitudinal correlation structure, and ignoring the distinct within- and between-period intracluster correlations can underestimate the sample size in small stepped-wedge cluster randomized trials. We conducted a review of published small stepped-wedge cluster randomized trials to understand how and why they are used, and to characterize approaches used in their design and analysis.

METHODS

Electronic searches were used to identify primary reports of full-scale stepped-wedge cluster randomized trials published during the period 2016-2022; the subset that randomized two to six clusters was identified. Two reviewers independently extracted information from each report and any available protocol. Disagreements were resolved through discussion.

RESULTS

We identified 61 stepped-wedge cluster randomized trials that randomized two to six clusters: median sample size (Q1-Q3) 1426 (420-7553) participants. Twelve (19.7%) gave some indication that the evaluation was considered a "preliminary" evaluation and 16 (26.2%) recognized the small number of clusters as a limitation. Sixteen (26.2%) provided an explanation for the limited number of clusters: the need to minimize contamination (e.g. by merging adjacent units), limited availability of clusters, and logistical considerations were common explanations. Majority (51, 83.6%) presented sample size or power calculations, but only one assumed distinct within- and between-period intracluster correlations. Few (10, 16.4%) utilized restricted randomization methods; more than half (34, 55.7%) identified baseline imbalances. The most common statistical method for analysis was the generalized linear mixed model (44, 72.1%). Only four trials (6.6%) reported statistical analyses considering small numbers of clusters: one used generalized estimating equations with small-sample correction, two used generalized linear mixed model with small-sample correction, and one used Bayesian analysis. Another eight (13.1%) used fixed-effects regression, the performance of which requires further evaluation under stepped-wedge cluster randomized trials with small numbers of clusters. None used permutation tests or cluster-period level analysis.

CONCLUSION

Methods appropriate for the design and analysis of small stepped-wedge cluster randomized trials have not been widely adopted in practice. Greater awareness is required that the use of standard sample size calculation methods can provide spuriously low numbers of required clusters. Methods such as generalized estimating equations or generalized linear mixed models with small-sample corrections, Bayesian approaches, and permutation tests may be more appropriate for the analysis of small stepped-wedge cluster randomized trials. Future research is needed to establish best practices for stepped-wedge cluster randomized trials with a small number of clusters.

A systematic review of stepped wedge cluster randomized trials in high impact journals: assessing the design, rationale, and analysis

OBJECTIVES

Stepped wedge cluster randomized trials (SW-CRTs) are an appealing study design because they enable sequential roll out of an intervention across clusters, bringing logistical advantages. This review aimed to evaluate the design rationale, design features, stepped wedge diagram, and analytical approaches of SW-CRTs published in high-impact medical journals from 2020 to 2023, focusing particularly on adherence to key guidelines from the Consolidated Standards of Reporting Trials extension to SW-CRTs.

STUDY DESIGN AND SETTING

We conducted searches across PubMed and Cochrane Central Registry of Controlled Trials databases for SW-CRTs published between January 2020 and December 2023 in eight high-impact journals. Eligibility criteria included peer-reviewed publications of randomized SW-CRTs involving human participants, published in English.

RESULTS

Of the 23 SW-CRTs included in the review, 70% had "stepped wedge" explicitly mentioned in their titles. Most studies (96%) included a stepped wedge diagram, but only 65% of these diagrams clearly communicated the duration of each time period. There was considerable variability in design features, including number of sequences (median of 7, range 3-20) and clusters (median of 15, range 9-19). The majority of trials (78%) provided robust justifications for selecting a SW-CRT design, for example, citing practical or logistical constraints. However, 22% of the studies offered less convincing rationales. Generalized linear mixed models were the most frequent analysis method employed.

CONCLUSION

Our review has highlighted areas for improvement in the presentation of SW-CRTs, particularly in clearly indicating the duration of time periods within diagrams and providing robust justifications for selecting a SW-CRT design.

PLAIN LANGUAGE SUMMARY

The stepped wedge cluster randomized trial (SW-CRT) is a type of study design that introduces interventions to different groups at different times. This review examined reports of SW-CRTs published in top medical journals from 2020 to 2023 to see if they followed certain guidelines such as including the word "stepped wedge" in their title. A total of 23 SW-CRTs were included in the review, with 70% mentioning "stepped wedge" in the title. Most (96%) included diagrams, but only 65% showed the duration of each time period clearly. There was variability in design, such as variations in the number of sequences and groups. 78% gave valid reasons for using SW-CRTs, citing practical benefits, whereas 22% did not give convincing reasons. This review suggests that improvements can be made in the presentation of stepped wedge diagrams and in the reporting of SW-CRTs. Researchers should clearly report the length of time periods and provide strong justifications for their design choice.

Impact of learning health systems on cross-system collaboration between youth legal and community mental health systems: a type II hybrid effectiveness-implementation trial

BACKGROUND

Youth involved in the legal system have disproportionately higher rates of problematic substance use than non-involved youth. Identifying and connecting legal-involved youth to substance use intervention is critical and relies on the connection between legal and behavioral health agencies, which may be facilitated by learning health systems (LHS). We analyzed the impact of an LHS intervention on youth legal and behavioral health personnel ratings of their cross-system collaboration. We also examined organizational climate toward evidence-based practice (EBP) over and above the LHS intervention.

METHODS

Data were derived from a type II hybrid effectiveness trial implementing an LHS intervention with youth legal and community mental health centers (CMHCs) in eight Indiana counties. Using a stepped wedge design, counties were randomly assigned to one of three cohorts and stepped in at nine-month intervals. Counties were in the treatment phase for 18 months, after which they were in the maintenance phase. Youth legal system and CMHC personnel completed five waves of data collection (n=307 total respondents, ranging from 108-178 per wave). Cross-system collaboration was measured via the Cultural Exchange Inventory, organizational EBP climate via the Implementation Climate Scale and Implementation Citizenship Behavior Scale, and intervention via a dummy-coded indicator variable. We conducted linear mixed models to examine: 1) the treatment indicator, and 2) the treatment indicator and organizational EBP climate variables on cross-system collaboration.

RESULTS

The treatment indicator was not significantly associated with cross-system collaboration. When including the organizational EBP climate variables, the treatment indicator significantly predicted cross-system collaboration. Compared to the control phase, treatment (B=0.41, standard error [SE]=0.20) and maintenance (B=0.60, SE=0.29) phases were associated with greater cross-system collaboration output.

CONCLUSIONS

The analysis may have been underpowered to detect an effect; third variables may have explained variance in cross-system collaboration, and, thus, the inclusion of important covariates may have reduced residual errors and increased the estimation precision. The LHS intervention may have affected cross-system collaboration perception and offers a promising avenue of research to determine how systems work together to improve legal-involved-youth substance use outcomes. Future research is needed to replicate results among a larger sample and examine youth-level outcomes.

TRIAL REGISTRATION

Clinicaltrials.gov identifier: NCT04499079. Registered 30 July 2020. https://clinicaltrials.gov/study/NCT04499079 .

An Occupational Health Literacy Intervention in Nursing Homes Improved Organizational Health Literacy-A Quasi-Experimental Stepped Wedge Cluster Trial

OBJECTIVE

This study examined the effectiveness of a workplace health literacy intervention on individual, interpersonal, and organizational health literacy.

METHOD

Using a quasi-experimental stepped wedge cluster design, we evaluated an intervention for 509 nursing home employees with two elements: 1) courses for employees and management on pain prevention, management, and communication and 2) structured dialogues between employees and supervisors, emphasizing pain prevention.

RESULTS

One organizational health literacy item improved, with supervisors helping with pain prevention increasing by 0.42 points (95% CI 0.11;0.73). Positive trends were observed in supervisor actions when informed about pain (0.39 points, 95% CI -0.09;0.86), ease of finding workplace pain solutions (0.12 points, 95% CI -0.03;0.79), and employees having pain management information (0.44 points, 95% CI -0.03;0.92).

CONCLUSION

The intervention improved one organizational health literacy item, with positive trends in three other items.

Inclusion of unexposed clusters improves the precision of fixed effects analysis of stepped-wedge cluster randomized trials with binary and count outcomes

BACKGROUND

The fixed effects model is a useful alternative to the mixed effects model for analyzing stepped-wedge cluster randomized trials (SW-CRTs). It controls for all time-invariant cluster-level confounders and has proper control of type I error when the number of clusters is small. While all clusters in a SW-CRT are typically designed to crossover from the control to receive the intervention, some trials can end with unexposed clusters (clusters that never receive the intervention), such as when a trial is terminated early due to safety concerns. It was previously unclear whether unexposed clusters would contribute to the estimation of the intervention effect in a fixed effects analysis. However, recent work has demonstrated that including an unexposed cluster can improve the precision of the intervention effect estimator in a fixed effects analysis of SW-CRTs with continuous outcomes. Still, SW-CRTs are commonly designed with binary outcomes and it is unknown if those previous results extend to SW-CRTs with non-continuous outcomes.

METHODS

In this article, we mathematically prove that the inclusion of unexposed clusters improves the precision of the fixed effects intervention effect estimator for SW-CRTs with binary and count outcomes. We then explore the benefits of including an unexposed cluster in simulated datasets with binary or count outcomes and a real palliative care data example with binary outcomes.

RESULTS

The simulations show that including unexposed clusters leads to tangible improvements in the precision, power, and root mean square error of the intervention effect estimator. The inclusion of the unexposed cluster in the SW-CRT of a novel palliative care intervention with binary outcomes yielded smaller standard errors and narrower 95% Wald Confidence Intervals.

CONCLUSIONS

In this article, we demonstrate that the inclusion of unexposed clusters in the fixed effects analysis can lead to the improvements in precision, power, and RMSE of the fixed effects intervention effect estimator for SW-CRTs with binary or count outcomes.

The fixed-effects model for robust analysis of stepped-wedge cluster trials with a small number of clusters and continuous outcomes: a simulation study

BACKGROUND

Stepped-wedge cluster trials (SW-CTs) describe a cluster trial design where treatment rollout is staggered over the course of the trial. Clusters are commonly randomized to receive treatment beginning at different time points in this study design (commonly referred to as a Stepped-wedge cluster randomized trial; SW-CRT), but they can also be non-randomized. Trials with this design regularly have a low number of clusters and can be vulnerable to covariate imbalance. To address such covariate imbalance, previous work has examined covariate-constrained randomization and analysis adjustment for imbalanced covariates in mixed-effects models. These methods require the imbalanced covariate to always be known and measured. In contrast, the fixed-effects model automatically adjusts for all imbalanced time-invariant covariates, both measured and unmeasured, and has been implicated to have proper type I error control in SW-CTs with a small number of clusters and binary outcomes.

METHODS

We present a simulation study comparing the performance of the fixed-effects model against the mixed-effects model in randomized and non-randomized SW-CTs with small numbers of clusters and continuous outcomes. Additionally, we compare these models in scenarios with cluster-level covariate imbalances or confounding.

RESULTS

We found that the mixed-effects model can have low coverage probabilities and inflated type I error rates in SW-CTs with continuous outcomes, especially with a small number of clusters or when the ICC is low. Furthermore, mixed-effects models with a Satterthwaite or Kenward-Roger small sample correction can still result in inflated or overly conservative type I error rates, respectively. In contrast, the fixed-effects model consistently produced the target level of coverage probability and type I error rates without dramatically compromising power. Furthermore, the fixed-effects model was able to automatically account for all time-invariant cluster-level covariate imbalances and confounding to robustly yield unbiased estimates.

CONCLUSIONS

We recommend the fixed-effects model for robust analysis of SW-CTs with a small number of clusters and continuous outcomes, due to its proper type I error control and ability to automatically adjust for all potential imbalanced time-invariant cluster-level covariates and confounders.

Maintaining the validity of inference from linear mixed models in stepped-wedge cluster randomized trials under misspecified random-effects structures

Linear mixed models are commonly used in analyzing stepped-wedge cluster randomized trials. A key consideration for analyzing a stepped-wedge cluster randomized trial is accounting for the potentially complex correlation structure, which can be achieved by specifying random-effects. The simplest random effects structure is random intercept but more complex structures such as random cluster-by-period, discrete-time decay, and more recently, the random intervention structure, have been proposed. Specifying appropriate random effects in practice can be challenging: assuming more complex correlation structures may be reasonable but they are vulnerable to computational challenges. To circumvent these challenges, robust variance estimators may be applied to linear mixed models to provide consistent estimators of standard errors of fixed effect parameters in the presence of random-effects misspecification. However, there has been no empirical investigation of robust variance estimators for stepped-wedge cluster randomized trials. In this article, we review six robust variance estimators (both standard and small-sample bias-corrected robust variance estimators) that are available for linear mixed models in R, and then describe a comprehensive simulation study to examine the performance of these robust variance estimators for stepped-wedge cluster randomized trials with a continuous outcome under different data generators. For each data generator, we investigate whether the use of a robust variance estimator with either the random intercept model or the random cluster-by-period model is sufficient to provide valid statistical inference for fixed effect parameters, when these working models are subject to random-effect misspecification. Our results indicate that the random intercept and random cluster-by-period models with robust variance estimators performed adequately. The CR3 robust variance estimator (approximate jackknife) estimator, coupled with the number of clusters minus two degrees of freedom correction, consistently gave the best coverage results, but could be slightly conservative when the number of clusters was below 16. We summarize the implications of our results for the linear mixed model analysis of stepped-wedge cluster randomized trials and offer some practical recommendations on the choice of the analytic model.

Robust analysis of stepped wedge trials using composite likelihood models

Stepped wedge trials (SWTs) are a type of cluster randomized trial that involve repeated measures on clusters and design-induced confounding between time and treatment. Although mixed models are commonly used to analyze SWTs, they are susceptible to misspecification particularly for cluster-longitudinal designs such as SWTs. Mixed model estimation leverages both "horizontal" or within-cluster information and "vertical" or between-cluster information. To use horizontal information in a mixed model, both the mean model and correlation structure must be correctly specified or accounted for, since time is confounded with treatment and measurements are likely correlated within clusters. Alternative non-parametric methods have been proposed that use only vertical information; these are more robust because between-cluster comparisons in a SWT preserve randomization, but these non-parametric methods are not very efficient. We propose a composite likelihood method that focuses on vertical information, but has the flexibility to recover efficiency by using additional horizontal information. We compare the properties and performance of various methods, using simulations based on COVID-19 data and a demonstration of application to the LIRE trial. We found that a vertical composite likelihood model that leverages baseline data is more robust than traditional methods, and more efficient than methods that use only vertical information. We hope that these results demonstrate the potential value of model-based vertical methods for SWTs with a large number of clusters, and that these new tools are useful to researchers who are concerned about misspecification of traditional models.

Stir-up Regimen After General Anesthesia in the Postanesthesia Care Unit: A Nurse Led Stepped Wedge Cluster Randomized Control Trial

PURPOSE

To implement a standardized Stir-up Regimen (deep breathing, coughing, repositioning, mobilization [moving arms/legs], assessing and managing pain and nausea) within the first 30 minutes of arrival in the postanesthesia care unit (PACU), with a goal of decreasing recovery time in the immediate postanesthesia period (Phase I).

DESIGN

A pragmatic stepped wedge cluster randomized control trial. Initially, data were collected on time in Phase I in three PACUs (control). Subsequently, the same three units were randomized to sequentially transition to the Stir-up Regimen (intervention).

METHODS

A stepped wedge cluster randomized control trial design was used to implement a standardized Stir-up Regimen in three PACUs in an academic hospital for adult patients who received at least 30 minutes of general anesthesia. The measured outcome was the PACU time in minutes from patient arrival to when the patient met Phase I discharge criteria. Differences between intervention and control groups were evaluated using a generalized mixed-effects model. Nurses were educated about the Stir-up Regimen in team huddles, in-services, video demonstrations, email notifications and reminders, and immediate feedback at the bedside. Implementation science principles were used to assess the adoption of the Stir-up Regimen through a presurvey, postsurvey and spot-check observations in all three PACUs.

FINDINGS

A total of 5,809 PACU adult patient admissions were included: control group (n = 2,860); intervention group (n = 2,949); males (n = 2,602), and females (n = 3,206). The intervention was associated with a reduction in overall mean Phase I recovery time of 4.9 minutes (95% CI: -8.4 to -1.4, P = .007). One PACU decreased time by 9.6 minutes (95% CI: -15.3 to -4.0, P < .001). The other units also reduced Phase I recovery time, but this did not reach statistical significance. The spot-check observations confirmed the intervention was adopted by the nurses, as most interventions were nurse-initiated versus patient-initiated during the first 30 minutes in PACU.

CONCLUSIONS

Standardization of a Stir-up Regimen within 30 minutes of patient PACU arrival resulted in decreased Phase I recovery time.

Adherence to key recommendations for design and analysis of stepped-wedge cluster randomized trials: A review of trials published 2016-2022

BACKGROUND/AIMS

The stepped-wedge cluster randomized trial (SW-CRT), in which clusters are randomized to a time at which they will transition to the intervention condition - rather than a trial arm - is a relatively new design. SW-CRTs have additional design and analytical considerations compared to conventional parallel arm trials. To inform future methodological development, including guidance for trialists and the selection of parameters for statistical simulation studies, we conducted a review of recently published SW-CRTs. Specific objectives were to describe (1) the types of designs used in practice, (2) adherence to key requirements for statistical analysis, and (3) practices around covariate adjustment. We also examined changes in adherence over time and by journal impact factor.

METHODS

We used electronic searches to identify primary reports of SW-CRTs published 2016-2022. Two reviewers extracted information from each trial report and its protocol, if available, and resolved disagreements through discussion.

RESULTS

We identified 160 eligible trials, randomizing a median (Q1-Q3) of 11 (8-18) clusters to 5 (4-7) sequences. The majority (122, 76%) were cross-sectional (almost all with continuous recruitment), 23 (14%) were closed cohorts and 15 (9%) open cohorts. Many trials had complex design features such as multiple or multivariate primary outcomes (50, 31%) or time-dependent repeated measures (27, 22%). The most common type of primary outcome was binary (51%); continuous outcomes were less common (26%). The most frequently used method of analysis was a generalized linear mixed model (112, 70%); generalized estimating equations were used less frequently (12, 8%). Among 142 trials with fewer than 40 clusters, only 9 (6%) reported using methods appropriate for a small number of clusters. Statistical analyses clearly adjusted for time effects in 119 (74%), for within-cluster correlations in 132 (83%), and for distinct between-period correlations in 13 (8%). Covariates were included in the primary analysis of the primary outcome in 82 (51%) and were most often individual-level covariates; however, clear and complete pre-specification of covariates was uncommon. Adherence to some key methodological requirements (adjusting for time effects, accounting for within-period correlation) was higher among trials published in higher versus lower impact factor journals. Substantial improvements over time were not observed although a slight improvement was observed in the proportion accounting for a distinct between-period correlation.

CONCLUSIONS

Future methods development should prioritize methods for SW-CRTs with binary or time-to-event outcomes, small numbers of clusters, continuous recruitment designs, multivariate outcomes, or time-dependent repeated measures. Trialists, journal editors, and peer reviewers should be aware that SW-CRTs have additional methodological requirements over parallel arm designs including the need to account for period effects as well as complex intracluster correlations.

Power calculation for detecting interaction effect in cross-sectional stepped-wedge cluster randomized trials: an important tool for disparity research

BACKGROUND

The stepped-wedge cluster randomized trial (SW-CRT) design has become popular in healthcare research. It is an appealing alternative to traditional cluster randomized trials (CRTs) since the burden of logistical issues and ethical problems can be reduced. Several approaches for sample size determination for the overall treatment effect in the SW-CRT have been proposed. However, in certain situations we are interested in examining the heterogeneity in treatment effect (HTE) between groups instead. This is equivalent to testing the interaction effect. An important example includes the aim to reduce racial disparities through healthcare delivery interventions, where the focus is the interaction between the intervention and race. Sample size determination and power calculation for detecting an interaction effect between the intervention status variable and a key covariate in the SW-CRT study has not been proposed yet for binary outcomes.

METHODS

We utilize the generalized estimating equation (GEE) method for detecting the heterogeneity in treatment effect (HTE). The variance of the estimated interaction effect is approximated based on the GEE method for the marginal models. The power is calculated based on the two-sided Wald test. The Kauermann and Carroll (KC) and the Mancl and DeRouen (MD) methods along with GEE (GEE-KC and GEE-MD) are considered as bias-correction methods.

RESULTS

Among three approaches, GEE has the largest simulated power and GEE-MD has the smallest simulated power. Given cluster size of 120, GEE has over 80% statistical power. When we have a balanced binary covariate (50%), simulated power increases compared to an unbalanced binary covariate (30%). With intermediate effect size of HTE, only cluster sizes of 100 and 120 have more than 80% power using GEE for both correlation structures. With large effect size of HTE, when cluster size is at least 60, all three approaches have more than 80% power. When we compare an increase in cluster size and increase in the number of clusters based on simulated power, the latter has a slight gain in power. When the cluster size changes from 20 to 40 with 20 clusters, power increases from 53.1% to 82.1% for GEE; 50.6% to 79.7% for GEE-KC; and 48.1% to 77.1% for GEE-MD. When the number of clusters changes from 20 to 40 with cluster size of 20, power increases from 53.1% to 82.1% for GEE; 50.6% to 81% for GEE-KC; and 48.1% to 79.8% for GEE-MD.

CONCLUSIONS

We propose three approaches for cluster size determination given the number of clusters for detecting the interaction effect in SW-CRT. GEE and GEE-KC have reasonable operating characteristics for both intermediate and large effect size of HTE.

Solar-powered O2 delivery for the treatment of children with hypoxaemia in Uganda: a stepped-wedge, cluster randomised controlled trial

BACKGROUND

Supplemental O2 is not always available at health facilities in low-income and middle-income countries (LMICs). Solar-powered O2 delivery can overcome gaps in O2 access, generating O2 independent of grid electricity. We hypothesized that installation of solar-powered O2 systems on the paediatrics ward of rural Ugandan hospitals would lead to a reduction in mortality among hypoxaemic children.

METHODS

In this pragmatic, country-wide, stepped-wedge, cluster randomised controlled trial, solar-powered O2 systems (ie, photovoltaic cells, battery bank, and O2 concentrator) were sequentially installed at 20 rural health facilities in Uganda. Sites were selected for inclusion based on the following criteria: District Hospital or Health Centre IV with paediatric inpatient services; supplemental O2 on the paediatric ward was not available or was unreliable; and adequate space to install solar panels, a battery bank, and electrical wiring. Allocation concealment was achieved for sites up to 2 weeks before installation, but the study was not masked overall. Children younger than 5 years admitted to hospital with hypoxaemia and respiratory signs were included. The primary outcome was mortality within 48 h of detection of hypoxaemia. The statistical analysis used a linear mixed effects logistic regression model accounting for cluster as random effect and calendar time as fixed effect. The trial is registered at ClinicalTrials.gov, NCT03851783.

FINDINGS

Between June 28, 2019, and Nov 30, 2021, 2409 children were enrolled across 20 hospitals and, after exclusions, 2405 children were analysed. 964 children were enrolled before site randomisation and 1441 children were enrolled after site randomisation (intention to treat). There were 104 deaths, 91 of which occurred within 48 h of detection of hypoxaemia. The 48 h mortality was 49 (5·1%) of 964 children before randomisation and 42 (2·9%) of 1440 (one individual did not have vital status documented at 48 h) after randomisation (adjusted odds ratio 0·50, 95% CI 0·27-0·91, p=0·023). Results were sensitive to alternative parameterisations of the secular trend. There was a relative risk reduction of 48·7% (95% CI 8·5-71·5), and a number needed to treat with solar-powered O2 of 45 (95% CI 28-230) to save one life. Use of O2 increased from 484 (50·2%) of 964 children before randomisation to 1424 (98·8%) of 1441 children after randomisation (p<0·0001). Adverse events were similar before and after randomisation and were not considered to be related to the intervention. The estimated cost-effectiveness was US$25 (6-505) per disability-adjusted life-year saved.

INTERPRETATION

This stepped-wedge, cluster randomised controlled trial shows the mortality benefit of improving O2 access with solar-powered O2. This study could serve as a model for scale-up of solar-powered O2 as one solution to O2 insecurity in LMICs.

FUNDING

Grand Challenges Canada and The Women and Children's Health Research Institute.

The effect of computerised decision support alerts tailored to intensive care on the administration of high-risk drug combinations, and their monitoring: a cluster randomised stepped-wedge trial

BACKGROUND

Drug-drug interactions (DDIs) can harm patients admitted to the intensive care unit (ICU). Yet, clinical decision support systems (CDSSs) aimed at helping physicians prevent DDIs are plagued by low-yield alerts, causing alert fatigue and compromising patient safety. The aim of this multicentre study was to evaluate the effect of tailoring potential DDI alerts to the ICU setting on the frequency of administered high-risk drug combinations.

METHODS

We implemented a cluster randomised stepped-wedge trial in nine ICUs in the Netherlands. Five ICUs already used potential DDI alerts. Patients aged 18 years or older admitted to the ICU with at least two drugs administered were included. Our intervention was an adapted CDSS, only providing alerts for potential DDIs considered as high risk. The intervention was delivered at the ICU level and targeted physicians. We hypothesised that showing only relevant alerts would improve CDSS effectiveness and lead to a decreased number of administered high-risk drug combinations. The order in which the intervention was implemented in the ICUs was randomised by an independent researcher. The primary outcome was the number of administered high-risk drug combinations per 1000 drug administrations per patient and was assessed in all included patients. This trial was registered in the Netherlands Trial Register (identifier NL6762) on Nov 26, 2018, and is now closed.

FINDINGS

In total, 10 423 patients admitted to the ICU between Sept 1, 2018, and Sept 1, 2019, were assessed and 9887 patients were included. The mean number of administered high-risk drug combinations per 1000 drug administrations per patient was 26·2 (SD 53·4) in the intervention group (n=5534), compared with 35·6 (65·0) in the control group (n=4353). Tailoring potential DDI alerts to the ICU led to a 12% decrease (95% CI 5-18%; p=0·0008) in the number of administered high-risk drug combinations per 1000 drug administrations per patient, after adjusting for clustering and prognostic factors.

INTERPRETATION

This cluster randomised stepped-wedge trial showed that tailoring potential DDI alerts to the ICU setting significantly reduced the number of administered high-risk drug combinations. Our list of high-risk drug combinations can be used in other ICUs, and our strategy of tailoring alerts based on clinical relevance could be applied to other clinical settings.

FUNDING

ZonMw.

Analysing cluster randomised controlled trials using GLMM, GEE1, GEE2, and QIF: results from four case studies

BACKGROUND

Using four case studies, we aim to provide practical guidance and recommendations for the analysis of cluster randomised controlled trials.

METHODS

Four modelling approaches (Generalized Linear Mixed Models with parameters estimated by maximum likelihood/restricted maximum likelihood; Generalized Linear Models with parameters estimated by Generalized Estimating Equations (1st order or second order) and Quadratic Inference Function, for analysing correlated individual participant level outcomes in cluster randomised controlled trials were identified after we reviewed the literature. We systematically searched the online bibliography databases of MEDLINE, EMBASE, PsycINFO (via OVID), CINAHL (via EBSCO), and SCOPUS. We identified the above-mentioned four statistical analytical approaches and applied them to four case studies of cluster randomised controlled trials with the number of clusters ranging from 10 to 100, and individual participants ranging from 748 to 9,207. Results were obtained for both continuous and binary outcomes using R and SAS statistical packages.

RESULTS

The intracluster correlation coefficient (ICC) estimates for the case studies were less than 0.05 and are consistent with the observed ICC values commonly reported in primary care and community-based cluster randomised controlled trials. In most cases, the four methods produced similar results. However, in a few analyses, quadratic inference function produced different results compared to the generalized linear mixed model, first-order generalized estimating equations, and second-order generalized estimating equations, especially in trials with small to moderate numbers of clusters.

CONCLUSION

This paper demonstrates the analysis of cluster randomised controlled trials with four modelling approaches. The results obtained were similar in most cases, however, for trials with few clusters we do recommend that the quadratic inference function should be used with caution, and where possible a small sample correction should be used. The generalisability of our results is limited to studies with similar features to our case studies, for example, studies with a similar-sized ICC. It is important to conduct simulation studies to comprehensively evaluate the performance of the four modelling approaches.

Estimating intra-cluster correlation coefficients for planning longitudinal cluster randomized trials: a tutorial

It is well-known that designing a cluster randomized trial (CRT) requires an advance estimate of the intra-cluster correlation coefficient (ICC). In the case of longitudinal CRTs, where outcomes are assessed repeatedly in each cluster over time, estimates for more complex correlation structures are required. Three common types of correlation structures for longitudinal CRTs are exchangeable, nested/block exchangeable and exponential decay correlations-the latter two allow the strength of the correlation to weaken over time. Determining sample sizes under these latter two structures requires advance specification of the within-period ICC and cluster autocorrelation coefficient as well as the intra-individual autocorrelation coefficient in the case of a cohort design. How to estimate these coefficients is a common challenge for investigators. When appropriate estimates from previously published longitudinal CRTs are not available, one possibility is to re-analyse data from an available trial dataset or to access observational data to estimate these parameters in advance of a trial. In this tutorial, we demonstrate how to estimate correlation parameters under these correlation structures for continuous and binary outcomes. We first introduce the correlation structures and their underlying model assumptions under a mixed-effects regression framework. With practical advice for implementation, we then demonstrate how the correlation parameters can be estimated using examples and we provide programming code in R, SAS, and Stata. An Rshiny app is available that allows investigators to upload an existing dataset and obtain the estimated correlation parameters. We conclude by identifying some gaps in the literature.

Generalizing the information content for stepped wedge designs: A marginal modeling approach

Stepped wedge trials are increasingly adopted because practical constraints necessitate staggered roll-out. While a complete design requires clusters to collect data in all periods, resource and patient-centered considerations may call for an incomplete stepped wedge design to minimize data collection burden. To study incomplete designs, we expand the metric of information content to discrete outcomes. We operate under a marginal model with general link and variance functions, and derive information content expressions when data elements (cells, sequences, periods) are omitted. We show that the centrosymmetric patterns of information content can hold for discrete outcomes with the variance-stabilizing link function. We perform numerical studies under the canonical link function, and find that while the patterns of information content for cells are approximately centrosymmetric for all examined underlying secular trends, the patterns of information content for sequences or periods are more sensitive to the secular trend, and may be far from centrosymmetric.

A Multi-component Intervention (NEXpro) Reduces Neck Pain-Related Work Productivity Loss: A Randomized Controlled Trial Among Swiss Office Workers

Purpose Neck pain is common among office workers and leads to work productivity loss. This study aimed to investigate the effect of a multi-component intervention on neck pain-related work productivity loss among Swiss office workers. Methods Office workers, aged 18-65 years, and without serious neck-related health problems were recruited from two organisations for our stepped-wedge cluster randomized controlled trial. The 12-week multi-component intervention included neck exercises, health-promotion information, and workplace ergonomics. The primary outcome of neck pain-related work productivity loss was measured using the Work Productivity and Activity Impairment Questionnaire and expressed as percentages of working time. In addition, we reported the weekly monetary value of neck pain-related work productivity loss. Data was analysed on an intention-to-treat basis using a generalized linear mixed-effects model. Results Data from 120 participants were analysed with 517 observations. At baseline, the mean age was 43.7 years (SD 9.8 years), 71.7% of participants were female (N = 86), about 80% (N = 95) reported mild to moderate neck pain, and neck pain-related work productivity loss was 12% of working time (absenteeism: 1.2%, presenteeism: 10.8%). We found an effect of our multi-component intervention on neck pain-related work productivity loss, with a marginal predicted mean reduction of 2.8 percentage points (b = -0.27; 95% CI: -0.54 to -0.001, p = 0.049). Weekly saved costs were Swiss Francs 27.40 per participant. Conclusions: Our study provides evidence for the effectiveness of a multi-component intervention to reduce neck pain-related work productivity loss with implications for employers, employees, and policy makers.Trial Registration ClinicalTrials.gov, NCT04169646. Registered 15 November 2019-Retrospectively registered, https://clinicaltrials.gov/ct2/show/NCT04169646 .

Evidence of secular trends during the COVID-19 pandemic in a stepped wedge cluster randomized trial examining sexual and reproductive health outcomes among Indigenous youth

BACKGROUND

Nen ŨnkUmbi/EdaHiYedo ("We Are Here Now," or NE) is an intervention to prevent STIs, HIV, HCV, and teen pregnancy among Assiniboine and Sioux youth of the Fort Peck Reservation in the state of Montana in the USA. A cluster-randomized stepped-wedge design (SWD) trial is used to evaluate NE, where clusters are schools. The purpose of this study is to evaluate whether there is evidence of a secular trend associated with the COVID-19 pandemic.

METHODS

The original study design is a cluster-randomized stepped-wedge design (SWD), in which five schools that youth from Fort Peck attend are the clusters to be randomized into the intervention one at a time, with all schools eventually being randomized to the intervention across three steps. N/E is a 5-year study involving 456 15- to 18-year-old youth. For this study, we use a mixed quantitative and qualitative methods approach to understand how the COVID-19 pandemic may have been associated with the study's primary outcome variables. Data were drawn from the first cluster exposed to the intervention and one control cluster that did not yet receive the intervention during the period in which COVID-19 mitigation efforts were being implemented. A pre-post COVID questionnaire was added to core measures administered, and semistructured qualitative interviews were conducted with youths regarding their perceptions of how the pandemic altered their sexual behaviors.

RESULTS

One hundred eighteen youth responded to the questionnaire and 31 youth participated in semistructured qualitative interviews. Youth reporting having sex with less people due to COVID-19 reported more sex acts (incident rate ratio (IRR)=3.6, 95% CI 1.6-8.1) in comparison to those who did not report having sex with less people, and youth who reported having sex with the same amount of people due to COVID-19 reported less sex acts (IRR=0.31, 95% CI 0.14-0.7) in comparison to those who did not report having sex with the same amount of people. Youth reporting having sex less times due to COVID-19 experienced a greater number of sex acts in comparison to those who did not report having sex less times (IRR=2.7, 1.2-6.4). Results suggest that more sexually active individuals reported perceiving having sex with less people and less frequent engagement in sex during the pandemic. It is possible that the COVID-19 pandemic period was associated with a truncation in the distribution of sexual activity that would bias an estimate of the intervention's effect.

CONCLUSION

Findings suggest evidence of a secular trend. This trend must be accounted for at trial end, and sensitivity analyses are recommended. Documenting and reporting on these findings encourages transparent reporting during the implementation of a SWD trial during a global pandemic, and informs endline analyses.

TRIAL REGISTRATION

This trial is registered with the Clinical trials registry of the US National Library of Medicine at the National Institutes of Health (NIH). It was registered on October 1, 2018. The study presented in this manuscript is funded by NIH National Institute on Minority Health and Health Disparities (NIMHD), Award # R01MD012761-01, Elizabeth Rink (Principal Investigator). The study's ClinicalTrials.gov number is NCT03694418.

GEEMAEE: A SAS macro for the analysis of correlated outcomes based on GEE and finite-sample adjustments with application to cluster randomized trials

BACKGROUND AND OBJECTIVES

Generalized estimating equations (GEE) are used to analyze correlated outcomes in marginal regression models with population-averaged interpretations of exposure effects. Limitations of popular software for GEE include: (i) user choice is restricted to a small set of within-cluster pairwise correlation (intra-class correlation; ICC) structures; and (ii) inference on ICC parameters is usually not possible because the precision of their estimates is not quantified. This is important because ICC values inform the design of cluster randomized trials. Beyond the standard GEE implementation, use of paired estimating equations (Prentice 1988) provides: (i) flexible specification of models for pairwise correlations and (ii) standard errors for ICC estimates. However, most GEEs give biased estimates of standard errors and correlations when the number of clusters is small (roughly, ≤40). Consequently, there is a need for software to provide GEE analysis with finite-sample bias-corrections.

METHODS

The SAS macro GEEMAEE implements paired estimating equations to simultaneously estimate parameters in marginal mean and ICC models. It provides bias-corrected standard errors and uses matrix-adjusted estimating equations (MAEE) for bias-corrected estimation of correlations. Several built-in correlation matrix options, rarely found in software, are offered for multi-period, cluster randomized trials and similarly structured longitudinal observational data structures. Additional options include user-specified correlation structures and deletion diagnostics, namely Cooks' Distance and DBETA statistics that estimate the influence of observations, cluster-periods (when applicable) and clusters.

RESULTS

GEEMAEE is illustrated for a binary and a count outcome in two stepped wedge cluster randomized trials and a binary outcome in a longitudinal study of disease surveillance. Use of MAEE resulted in larger values of correlation estimates compared to uncorrected estimating equations. Use of bias-corrected variance estimators resulted in (appropriately) larger values of standard errors compared to the usual sandwich estimators. Deletion diagnostics identified the clusters and cluster-periods having the most influence.

CONCLUSIONS

The SAS macro GEEMAEE provides regression analysis for clustered or longitudinal responses, and is particularly useful when the number of clusters is small. Flexible specification and bias-corrected estimation of pairwise correlation parameters and standard errors are key features of the software to provide valid inference in real-world settings.

A general method for calculating power for GEE analysis of complete and incomplete stepped wedge cluster randomized trials

Stepped wedge designs have uni-directional crossovers at randomly assigned time points (steps) where clusters switch from control to intervention condition. Incomplete stepped wedge designs are increasingly used in cluster randomized trials of health care interventions and have periods without data collection due to logistical, resource and patient-centered considerations. The development of sample size formulae for stepped wedge trials has primarily focused on complete designs and continuous responses. Addressing this gap, a general, fast, non-simulation based power procedure is proposed for generalized estimating equations analysis of complete and incomplete stepped wedge designs and its predicted power is compared to simulated power for binary and continuous responses. An extensive set of simulations for six and twelve clusters is based upon the Connect-Home trial with an incomplete stepped wedge design. Results show that empirical test size is well controlled using a t-test with bias-corrected sandwich variance estimator for as few as six clusters. Analytical power agrees well with a simulated power in scenarios with twelve clusters. For six clusters, analytical power is similar to simulated power with estimation using the correctly specified model-based variance estimator. To explore the impact of study design choice on power, the proposed fast GEE power method is applied to the Connect-Home trial design, four alternative incomplete stepped wedge designs and one complete design.

power swgee: GEE-based power calculations in stepped wedge cluster randomized trials

Stepped wedge cluster randomized trials are increasingly being used to evaluate interventions in medical, public health, educational, and social science contexts. With the longitudinal and crossover nature of a SW-CRT, complex analysis techniques are often needed which makes appropriately powering SW-CRTs challenging. In this paper, we introduce a newly-developed SW-CRT power calculator, embedded within the power command in Stata. The power calculator assumes a marginal model (i.e., generalized estimating equations [GEE]) for the primary analysis of SW-CRTs, for which other currently available SW-CRT power calculators may not be suitable. The program accommodates complete cross-sectional and closed-cohort designs, and includes multilevel correlation structures appropriate for such designs. We discuss the methods and formulae underlying our SW-CRT calculator, and provide illustrative examples of the use of power swgee. We provide suggestions about the choice of parameters in power swgee, and conclude by discussing areas of future research which may improve the program.

Implementation of Coach McLungsSM into primary care using a cluster randomized stepped wedge trial design

Background

Asthma is a prevalent chronic disease that is difficult to manage and associated with marked disparities in outcomes. One promising approach to addressing disparities is shared decision making (SDM), a method by which the patient and provider cooperatively make a decision about asthma care. SDM is associated with improved outcomes for patients; however, time constraints and staff availability are noted implementation barriers. Use of health information technology (IT) solutions may facilitate the utilization of SDM. Coach McLungs^SM is a collaborative web-based application that involves pediatric patients, their caregivers, and providers in a personalized experience while gathering patient-reported data. Background logic provides decision support so both audiences can develop a well-informed treatment plan together. The goal of this study is to evaluate the implementation of the Coach McLungs^SM intervention into primary care.

Methods

Implementation will be evaluated using a stepped wedge randomized control study design at 21 pediatric and family medicine practices within a large, integrated, nonprofit healthcare system. We will measure changes in emergency department visits, hospitalizations, and oral steroid use, which serve as surrogate measures for patient-centered asthma outcomes. We will use a generalized linear mixed models with logit link to test the hypothesis for the reduction in exacerbation rates specifying the fixed effects of intervention and time and random effects for practice and practice*time. This design achieves 84% power to detect the hypothesized effect size difference of 10% in overall exacerbation between control (40%) and intervention (30%) periods (two-sided, p = 0.05). Implementation will be guided using the Expert Recommendations for Implementing Change (ERIC), a compilation of implementation strategies, and evaluated using the CFIR (Consolidated Framework for Implementation Research) and RE-AIM (Reach Effectiveness, Adoption, Implementation, Maintenance).

Discussion

We anticipate that a tailored implementation of Coach McLungs^SM across diverse primary care practices will lead to a decrease in emergency department visits, hospitalizations, and oral steroid use for patients in the intervention group as compared to the control condition.

Trial Registration: Clincaltrials.gov, NCT05059210. Registered 28 September 2021, https://www.clinicaltrials.gov/ct2/show/NCT05059210

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-02030-1.

Cost-effectiveness of a community-based integrated care model compared with usual care for older adults with complex needs: a stepped-wedge cluster-randomised trial

Objective

To assess the cost of implementation, delivery and cost-effectiveness (CE) of a flagship community-based integrated care model (OPEN ARCH) against the usual primary care.

Design

A 9-month stepped-wedge cluster-randomised trial.

Setting and participants

Community-dwelling older adults with chronic conditions and complex care needs were recruited from primary care (14 general practices) in Far North Queensland, Australia.

Methods

Costs and outcomes were measured at 3-month windows from the healthcare system and patient’s out-of-pocket perspectives for the analysis. Outcomes included functional status (Functional Independence Measure (FIM)) and health-related quality of life (EQ-5D-3L and AQoL-8D). Bayesian CE analysis with 10 000 Monte Carlo simulations was performed using the BCEA package in R (V.3.6.1).

Results

The OPEN ARCH model of care had an average cost of $A1354 per participant. The average age of participants was 81, and 55% of the cohort were men. Within-trial multilevel regression models adjusted for time, general practitioner cluster and baseline confounders showed no significant differences in costs, resource use or effect measures regardless of the analytical perspective. Probabilistic sensitivity analysis with 10 000 simulations showed that OPEN ARCH could be recommended over usual care for improving functional independence at a willing to pay above $A600 (US$440) per improvement of one point on the FIM Scale and for avoiding or reducing inpatient stay for any willingness-to-pay threshold up to $A50 000 (US$36 500).

Conclusions and implications

OPEN ARCH was associated with a favourable Bayesian CE profile in improving functional status and dependency levels, avoiding or reducing inpatient stay compared with usual primary care in the Australian context.

Trial registration number

ACTRN12617000198325.

Experience of Inpatient Mental Health Care Assessed With Service User-Developed and Conventional Patient-Reported Outcome Measures

OBJECTIVE

The goal of this study was to examine and compare the psychometric properties of a patient-reported outcome measure (PROM) generated with patients’ input (Views on Inpatient Care [VOICE]) and a PROM conventionally generated without patients’ input (Service Satisfaction Scale: Residential Services Evaluation [SSS-Res]) for assessing a patient’s perception of psychiatric ward care.

METHODS

In a stepped-wedge cluster-randomized trial conducted in the United Kingdom, 1,058 participants admitted to 16 wards reported on their perceptions of care via VOICE and SSS-Res before or up to 2 years after the staff training. Exploratory and confirmatory factor analyses were used to investigate the structure of the PROMs and to assess reliability and convergent validity as well as sensitivity to change; the analyses also considered whether study participants had been admitted voluntarily to the ward.

RESULTS

Two factors emerged from VOICE, labeled “trust” and “involvement,” and from SSS-Res, labeled “environment” and “care,” at baseline. All subscales had high internal consistency and good convergent validity. An ability to detect change in care due to the staff training was observed on the trust subscale of VOICE (N=1,058, mean difference=−0.25, 95% CI=−0.48 to −0.02), but no change was detected on any of the SSS-Res subscales. Patients admitted involuntarily benefited the most from the staff training.

CONCLUSIONS

VOICE captured patients’ perceptions of ward care better than SSS-Res and was sensitive to changes in aspects of trust, suggesting that participatory approaches for developing PROMs improve patients’ self-reports on the care they received.

Influence of cluster-period cells in stepped wedge cluster randomized trials

Stepped wedge cluster randomized trials (SWCRT) are increasingly used for the evaluation of complex interventions in health services research. They randomly allocate treatments to clusters that switch to intervention under investigation at variable time points without returning to control condition. The resulting unbalanced allocation over time periods and the uncertainty about the underlying correlation structures at cluster-level renders designing and analyzing SWCRTs a challenge. Adjusting for time trends is recommended, appropriate parameterizations depend on the particular context. For sample size calculation, the covariance structure and covariance parameters are usually assumed to be known. These assumptions greatly affect the influence single cluster-period cells have on the effect estimate. Thus, it is important to understand how cluster-period cells contribute to the treatment effect estimate. We therefore discuss two measures of cell influence. These are functions of the design characteristics and covariance structure only and can thus be calculated at the planning stage: the coefficient matrix as discussed by Matthews and Forbes and information content (IC) as introduced by Kasza and Forbes. The main result is a new formula for IC that is more general and computationally more efficient. The formula applies to any generalized least squares estimator, especially for any type of time trend adjustment or nonblock diagonal matrices. We further show a functional relationship between IC and the coefficient matrix. We give two examples that tie in with current literature. All discussed tools and methods are implemented in the R package SteppedPower.

Influential methods reports for group-randomized trials and related designs

BACKGROUND

This article identifies the most influential methods reports for group-randomized trials and related designs published through 2020. Many interventions are delivered to participants in real or virtual groups or in groups defined by a shared interventionist so that there is an expectation for positive correlation among observations taken on participants in the same group. These interventions are typically evaluated using a group- or cluster-randomized trial, an individually randomized group treatment trial, or a stepped wedge group- or cluster-randomized trial. These trials face methodological issues beyond those encountered in the more familiar individually randomized controlled trial.

METHODS

PubMed was searched to identify candidate methods reports; that search was supplemented by reports known to the author. Candidate reports were reviewed by the author to include only those focused on the designs of interest. Citation counts and the relative citation ratio, a new bibliometric tool developed at the National Institutes of Health, were used to identify influential reports. The relative citation ratio measures influence at the article level by comparing the citation rate of the reference article to the citation rates of the articles cited by other articles that also cite the reference article.

RESULTS

In total, 1043 reports were identified that were published through 2020. However, 55 were deemed to be the most influential based on their relative citation ratio or their citation count using criteria specific to each of the three designs, with 32 group-randomized trial reports, 7 individually randomized group treatment trial reports, and 16 stepped wedge group-randomized trial reports. Many of the influential reports were early publications that drew attention to the issues that distinguish these designs from the more familiar individually randomized controlled trial. Others were textbooks that covered a wide range of issues for these designs. Others were "first reports" on analytic methods appropriate for a specific type of data (e.g. binary data, ordinal data), for features commonly encountered in these studies (e.g. unequal cluster size, attrition), or for important variations in study design (e.g. repeated measures, cohort versus cross-section). Many presented methods for sample size calculations. Others described how these designs could be applied to a new area (e.g. dissemination and implementation research). Among the reports with the highest relative citation ratios were the CONSORT statements for each design.

CONCLUSIONS

Collectively, the influential reports address topics of great interest to investigators who might consider using one of these designs and need guidance on selecting the most appropriate design for their research question and on the best methods for design, analysis, and sample size.

Alternative Designs for Testing Speech, Language, and Hearing Interventions: Cluster-Randomized Trials and Stepped-Wedge Designs

PURPOSE

Individual-randomized trials are the gold standard for testing the efficacy and effectiveness of drugs, devices, and behavioral interventions. Health care delivery, educational, and programmatic interventions are often complex, involving multiple levels of change and measurement precluding individual randomization for testing. Cluster-randomized trials and cluster-randomized stepped-wedge trials are alternatives where the intervention is allocated at the group level, such as a clinic or a school, and the outcomes are measured at the person level. These designs are introduced along with the statistical implications of similarities among individuals within the same cluster. We also illustrate the parameters that have the most impact on the likelihood of detecting intervention effects, which must be considered when planning these trials.

CONCLUSION

Cluster-randomized and stepped-wedge designs should be considered by researchers as experimental alternatives to individual-randomized trials when testing speech, language, and hearing care interventions in real-world settings.

Rationale, Design, and Methods for Nen Unkumbi/Edahiyedo ("We Are Here Now"): A Multi-Level Randomized Controlled Trial to Improve Sexual and Reproductive Health Outcomes in a Northern Plains American Indian Reservation Community

American Indian (AI) youth in the United States experience disproportionate sexual and reproductive health (SRH) disparities relative to their non-Indigenous, white counterparts, including increased rates of sexually transmitted infections (STIs), earlier sexual debut, increased rates of teen birth, and reduced access to SRH services. Past research shows that to improve SRH outcomes for AI youth in reservation communities, interventions must address complex factors and multiple levels of community that influence sexual risk behaviors. Here, we describe development of a multi-level, multi-component randomized controlled trial (RCT) to intervene upon SRH outcomes in a Northern Plains American Indian reservation community. Our intervention is rooted in a community based participatory research framework and is evaluated with a stepped wedge design that integrates 5 reservation high schools into a 5-year, cluster-randomized RCT. Ecological Systems Theory was used to design the intervention that includes (1) an individual level component of culturally specific SRH curriculum in school, (2) a parental component of education to improve parent-child communication about SRH and healthy relationships, (3) a community component of cultural mentorship, and (4) a systems-level component to improve delivery of SRH services from reservation healthcare agencies. In this article we present the rationale and details of our research design, instrumentation, data collection protocol, analytical methods, and community participation in the intervention. Our intervention builds upon existing community strengths and integrates traditional Indigenous knowledge and values with current public health knowledge to reduce SRH disparities.

Encouraging older adults with pre-frailty and frailty to "MoveStrong": an analysis of secondary outcomes for a pilot randomized controlled trial

BACKGROUND

This 8-week pilot stepped-wedge randomized controlled trial evaluated the MoveStrong program for teaching adults who have frailty/pre-frailty about balance and functional strength training and sufficient protein intake to prevent falls and improve mobility.

METHODS

We recruited individuals aged 60 years and over, with a FRAIL scale score of 1 or higher and at least one chronic condition, who were not currently strength training. The program included 16 exercise physiologist-led hour-long group sessions and two dietitian-led hour-long nutrition sessions. We analyzed secondary outcomes-weight, gait speed, grip strength, physical capacity (fatigue levels), sit-to-stand functioning, dynamic balance, health-related quality of life (HRQoL), physical activity levels and protein intake-using a paired t test and a generalized estimating equation (GEE).

RESULTS

Of 44 participants (mean [SD] age 79 [9.82] years), 35 were pre-frail and 9 were frail. At follow-up, participants had significantly improved grip strength (1.63 kg, 95% CI: 0.62 to 2.63); sit-to-stand functioning (2 sit-to-stands, 95% CI: 1 to 3); and dynamic balance (1.68 s, 95% CI: 0.47 to 2.89). There were no significant improvements in gait speed, HRQoL index scores, self-rated health, physical activity levels (aerobic activity and strength training) or protein intake. GEE analysis revealed an interaction between exposure to MoveStrong and gait speed, sit-to-stand functioning, dynamic balance and HRQoL index scores. The total cost to administer the program and purchase equipment was CAD 14 700, equivalent to CAD 377 per participant.

CONCLUSION

Exploratory analyses suggest MoveStrong exercises may improve gait speed, sit-to-stand functioning, dynamic balance and HRQoL index scores in older individuals who are frail and pre-frail.

An intervention package for supporting the mental well-being of community health workers in low, and middle-income countries during the COVID-19 pandemic

BACKGROUND

As the COVID-19 pandemic continues, there is an increasing reliance on community health workers (CHWs) to achieve its control especially in low, and middle-income countries (LMICs). An increase in the demand for their services and the challenges they already face make them prone to mental health illness. Therefore, there is a need to further support the mental health and well-being of CHWs during the COVID-19 pandemic.

METHODS

We organised a workshop on Zoom to deliberate on relevant components of an intervention package for supporting the mental health of CHWs in LMICs during the COVID-19 pandemic. We used a thematic analysis approach to summarise deliberations from this workshop.

OUTCOMES

Participants identified the need for a hub for coordinating CHW activities, a care coordination team to manage their health, training programs aimed at improving their work performance and taking control of their health, a communication system that keeps them in touch with colleagues, family, and the communities they serve. They cautioned against confidentiality breaches while handling personal health information and favoured tailoring interventions to the unique needs of CHWs. Participants also advised on the need to ensure job security for CHWs and draw on available resources in the community. To measure the impact of such an intervention package, participants encouraged the use of mixed methods and a co-designed approach.

INTERPRETATION

As CHWs contribute to the pandemic response in LMICs, their mental health and well-being need to be protected. Such protection can be provided by using an intervention package that harnesses inputs from members of the broader health system, their families, and communities.

Stepped Wedge Cluster Randomized Trials: A Methodological Overview

BACKGROUND

Stepped wedge cluster randomized trials enable rigorous evaluations of health intervention programs in pragmatic settings. In the present study, we aimed to update neurosurgeon scientists on the design of stepped wedge randomized trials.

METHODS

We have presented an overview of recent methodological developments for stepped wedge designs and included an update on the newer associated methodological tools to aid with future study designs.

RESULTS

We defined the stepped wedge trial design and reviewed the indications for the design in depth. In addition, key considerations, including mainstream methods of analysis and sample size determination, were discussed.

CONCLUSIONS

Stepped wedge designs can be attractive for study intervention programs aiming to improve the delivery of patient care, especially when examining a small number of heterogeneous clusters.

Evaluating the performance of Bayesian and restricted maximum likelihood estimation for stepped wedge cluster randomized trials with a small number of clusters

Background

Stepped wedge trials are an appealing and potentially powerful cluster randomized trial design. However, they are frequently implemented with a small number of clusters. Standard analysis methods for these trials such as a linear mixed model with estimation via maximum likelihood or restricted maximum likelihood (REML) rely on asymptotic properties and have been shown to yield inflated type I error when applied to studies with a small number of clusters. Small-sample methods such as the Kenward-Roger approximation in combination with REML can potentially improve estimation of the fixed effects such as the treatment effect. A Bayesian approach may also be promising for such multilevel models but has not yet seen much application in cluster randomized trials.

Methods

We conducted a simulation study comparing the performance of REML with and without a Kenward-Roger approximation to a Bayesian approach using weakly informative prior distributions on the intracluster correlation parameters. We considered a continuous outcome and a range of stepped wedge trial configurations with between 4 and 40 clusters. To assess method performance we calculated bias and mean squared error for the treatment effect and correlation parameters and the coverage of 95% confidence/credible intervals and relative percent error in model-based standard error for the treatment effect.

Results

Both REML with a Kenward-Roger standard error and degrees of freedom correction and the Bayesian method performed similarly well for the estimation of the treatment effect, while intracluster correlation parameter estimates obtained via the Bayesian method were less variable than REML estimates with different relative levels of bias.

Conclusions

The use of REML with a Kenward-Roger approximation may be sufficient for the analysis of stepped wedge cluster randomized trials with a small number of clusters. However, a Bayesian approach with weakly informative prior distributions on the intracluster correlation parameters offers a viable alternative, particularly when there is interest in the probability-based inferences permitted within this paradigm.

Supplementary Information

The online version contains supplementary material available at (10.1186/s12874-022-01550-8).

Ethical and epistemic issues in the design and conduct of pragmatic stepped-wedge cluster randomized clinical trials

Stepped-wedge cluster randomized trial (SW-CRT) designs are increasingly employed in pragmatic research; they differ from traditional parallel cluster randomized trials in which an intervention is delivered to a subset of clusters, but not to all. In a SW-CRT, all clusters receive the intervention under investigation by the end of the study. This approach is thought to avoid ethical concerns about the denial of a desired intervention to participants in control groups. Such concerns have been cited in the literature as a primary motivation for choosing SW-CRT design, however SW-CRTs raise additional ethical concerns related to the delayed implementation of an intervention and consent. Yet, PCT investigators may choose SW-CRT designs simply because they are concerned that other study designs are infeasible. In this paper, we examine justifications for the use of SW-CRT study design, over other designs, by drawing on the experience of the National Institutes of Health's Health Care Systems Research Collaboratory (NIH Collaboratory) with five pragmatic SW-CRTs. We found that decisions to use SW-CRT design were justified by practical and epistemic reasons rather than ethical ones. These include concerns about feasibility, the heterogeneity of cluster characteristics, and the desire for simultaneous clinical evaluation and implementation. In this paper we compare the potential benefits of SW-CRTs against the ethical and epistemic challenges brought forth by the design and suggest that the choice of SW-CRT design must balance epistemic, feasibility and ethical justifications. Moreover, given their complexity, such studies need rigorous and informed ethical oversight.

A Quality Improvement Emergency Department Surge Management Platform (SurgeCon): Protocol for a Stepped Wedge Cluster Randomized Trial

BACKGROUND

Despite many efforts, long wait times and overcrowding in emergency departments (EDs) have remained a significant health service issue in Canada. For several years, Canada has had one of the longest wait times among the Organisation for Economic Co-operation and Development countries. From a patient's perspective, this challenge has been described as "patients wait in pain or discomfort for hours before being seen at EDs." To overcome the challenge of increased wait times, we developed an innovative ED management platform called SurgeCon that was designed based on continuous quality improvement principles to maintain patient flow and mitigate the impact of patient surge on ED efficiency. The SurgeCon quality improvement intervention includes a protocol-driven software platform, restructures ED organization and workflow, and aims to establish a more patient-centric environment. We piloted SurgeCon at an ED in Carbonear, Newfoundland and Labrador, and found that there was a 32% reduction in ED wait times.

OBJECTIVE

The primary objective of this trial is to determine the effects of SurgeCon on ED performance by assessing its impact on length of stay, the time to a physician's initial assessment, and the number of patients leaving the ED without being seen by a physician. The secondary objectives of this study are to evaluate SurgeCon's effects on patient satisfaction and patient-reported experiences with ED wait times and its ability to create better-value care by reducing the per-patient cost of delivering ED services.

METHODS

The implementation of the intervention will be assessed using a comparative effectiveness-implementation hybrid design. This type of hybrid design is known to shorten the amount of time associated with transitioning interventions from being the focus of research to being used for practice and health care services. All EDs with 24/7 on-site physician support (category A hospitals) will be enrolled in a 31-month, pragmatic, stepped wedge cluster randomized trial. All clusters (hospitals) will start with a baseline period of usual care and will be randomized to determine the order and timing of transitioning to intervention care until all hospitals are using the intervention to manage and operationalize their EDs.

RESULTS

Data collection for this study is continuing. As of February 2022, a total of 570 randomly selected patients have participated in telephone interviews concerning patient-reported experiences and patient satisfaction with ED wait times. The first of the 4 EDs was randomly selected, and it is currently using SurgeCon's eHealth platform and applying efficiency principles that have been learned through training since September 2021. The second randomly selected site will begin intervention implementation in winter 2022.

CONCLUSIONS

By assessing the impact of SurgeCon on ED services, we hope to be able to improve wait times and create better-value ED care in this health care context.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04789902; https://clinicaltrials.gov/ct2/show/NCT04789902.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/30454.

Random effect misspecification in stepped wedge designs

Stepped wedge cluster randomized trials are often analysed using linear mixed effects models that may include random effects for cluster, time and/or treatment. We investigate the impact of misspecification of the random effects structure of the model. Specifically, we considered two cases of misspecification of the random effects in a cross-sectional stepped wedge cluster randomized trials model - fit a linear mixed effects model with random time effects but the true model includes random treatment effects (case 1) or fit a linear mixed effects model with random treatment effect but the true model includes random time effects (case 2) - and derived the variance of the estimated treatment effect under misspecification. We defined two measures of the effect of misspecification: validity and efficiency. Validity is the ratio of the model-based variance of the treatment effect from the mis-specified model divided by the true variance of the treatment effect from the mis-specified model (based on a sandwich estimate of the variance). Efficiency is the ratio of the model-based variance of the treatment effect from the correctly specified model divided by the true variance of the treatment effect from the mis-specified model. We found that validity is less than 1.0 (anti-conservative) in almost all situations investigated with the exception of case 1 with two sequences, when validity could be greater than 1.0. Efficiency is less than 1 in all cases and depends on the intracluster correlation coefficient, the relative magnitude of the variance of the misclassified variance component, and the number of sequences. In general, there is no universal recommendation as to the most robust approach except for the case of a classic stepped wedge cluster randomized trial with only 2 sequences, where fitting a random time model is less likely to lead to anti-conservative inference compared with fitting a random intervention model.

Effect of extended trading hours on alcohol sales in Norway: study protocol for a stepped wedge cluster-randomized trial

BACKGROUND AND AIMS

Norwegian alcohol policy measures include national restrictions on sales hours and a state monopoly on retail sales. A 1-hour extension of sales hours on Saturdays in the monopoly outlets took effect from September 2020. We aim to evaluate whether increase in sales hours results in (1) an increase in alcohol sales in the monopoly outlets and (2) an increase in total alcohol sales, including substitution effects from beer sales in grocery stores.

DESIGN

The extension of Saturday sales hours is implemented within a stepped wedge cluster-randomized trial design. Block randomization of 62 of the 66 Norwegian trade districts allocated monopoly outlets to one of three sequences regarding date of implementation.

SETTING AND PARTICIPANTS

A total of 228 of 335 in total Norwegian state monopoly outlets are eligible and included.

INTERVENTION

The extension of sales hours is from 3 p.m. to 4 p.m. starting on the first Saturday in (i) September 2020, (ii) December 2020 or (iii) March 2021.

MEASUREMENTS

Growth rates in monthly alcohol sales, measured in litres of pure alcohol, in eligible monopoly outlets (primary outcome) are obtained together with beverage-specific sales and alcohol sales in grocery stores (secondary outcomes). The observation period is set to 72 months prior to and 24 months after implementation.

FINDINGS

Power analyses indicate that this stepped wedge cluster-randomized controlled trial has a power above 90%, even at a high significance level (α = 0.01) and with other conservative model specifications. The planned trial offers a rare opportunity to study possible causal effects of a relatively small change in a widely used alcohol policy measure.

Model misspecification in stepped wedge trials: Random effects for time or treatment

Mixed models are commonly used to analyze stepped wedge trials (SWTs) to account for clustering and repeated measures on clusters. One critical issue researchers face is whether to include a random time effect or a random treatment effect. When the wrong model is chosen, inference on the treatment effect may be invalid. We explore asymptotic and finite-sample convergence of variance component estimates when the model is misspecified and how misspecification affects the estimated variance of the treatment effect. For asymptotic results, we rely on analytical solutions rather than simulation studies, which allow us to succinctly describe the convergence of misspecified estimates, even though there are multiple roots for each misspecified model. We found that both direction and magnitude of the bias associated with model-based standard errors depends on the study design and magnitude of the true variance components. We identify some scenarios in which choosing the wrong random effect has a large impact on model-based inference. However, many trends depend on trial design and assumptions about the true correlation structure, so we provide tools for researchers to investigate specific scenarios of interest. We use data from an SWT on disinvesting from weekend services in hospital wards to demonstrate how these results can be applied as a sensitivity analysis, which quantifies the impact of misspecification under a variety of settings and directly compares the potential consequences of different modeling choices. Our results will provide guidance for prespecified model choices and supplement sensitivity analyses to inform confidence in the validity of results.

American Physical Therapy Association Clinical Practice Guideline Implementation for Neck and Low Back Pain in Outpatient Physical Therapy: A Nonrandomized, Cross-sectional Stepped-Wedge Pilot Study

OBJECTIVE

To determine whether a multifaceted implementation strategy for American Physical Therapy Association neck and low back pain clinical practice guidelines (CPGs) was associated with changes in clinician and patient outcomes.

DESIGN

Cross-sectional stepped-wedge pilot study.

METHODS

Physical therapy clinics (n = 9) were allocated to 1 of 4 clusters that varied by CPG implementation timing. Clinics crossed over from usual care (control) to CPG implementation (intervention) every 8 weeks and ended with a 24-week follow-up period. Implementation outcomes were measured at the clinician (perspectives and behaviors) and patient (pain and disability outcomes) levels. Descriptive statistics were used to summarize clinician perspectives and behaviors. Generalized linear mixed models were used to analyze patient-level outcomes data (pain and disability) and total number of physical therapy visits.

RESULTS

Improvements in several clinician perspectives about CPGs were observed 8 weeks after training and sustained at 16 weeks (P<.05), although it is unclear whether these changes were meaningful. Training on CPGs was relevant to physical therapists and more acceptable at 16 weeks (P<.05). In a random sample (n = 764/1994, 38.3%), the overall rate of CPG classification was 65.0% (n = 497/764), and CPG intervention concordance was 71.2% (n = 354/497). Implementation of a CPG was not associated with final pain and disability scores (P>.05) but was associated with an approximate increase of 8% in total visits.

CONCLUSION

Our multifaceted implementation strategy was associated with statistical changes in clinician perspectives and behaviors, but not in patient outcomes. J Orthop Sports Phys Ther 2022;52(2):113-123. doi:10.2519/jospt.2022.10545.

Adaptations Due to the COVID-19 Pandemic in a Community-Based Participatory Research Randomized Control Trial Examining Sexual and Reproductive Health Outcomes among American Indian Youth

In this manuscript, we present changes in study design and analytical strategy due to the COVID-19 pandemic for Nen ŨnkUmbi/EdaHiYedo ("We Are Here Now," or NE). NE is a community-based participatory research multi-level randomized control trial using a stepped wedge design to address sexual and reproductive health disparities among American Indian youth. Adaptations in NE's research design, data collection, and analysis due to the COVID-19 pandemic were made based on meetings with tribally based research team members and outside non-Indigenous researchers involved in NE, as well as the study's Community Advisory Board and the Data Safety Monitoring Board. Based on these iterative discussions, decisions were made to: 1) reorganize the sequence of NE's stepped wedge design clusters, and 2) include additional quantitative and qualitative data collection and analysis in the research design that specifically addressed the impact of COVID-19 on the research participants. These adaptations have the potential to foster greater scientific knowledge in understanding how to address unanticipated 3-way interaction effects in randomized control trials with tribal communities. Findings can also contribute to understanding how public health disasters impact sexual and reproductive health among American Indian youth.

A comparative study of R functions for clustered data analysis

Background

Clustered or correlated outcome data is common in medical research studies, such as the analysis of national or international disease registries, or cluster-randomized trials, where groups of trial participants, instead of each trial participant, are randomized to interventions. Within-group correlation in studies with clustered data requires the use of specific statistical methods, such as generalized estimating equations and mixed-effects models, to account for this correlation and support unbiased statistical inference.

Methods

We compare different approaches to estimating generalized estimating equations and mixed effects models for a continuous outcome in R through a simulation study and a data example. The methods are implemented through four popular functions of the statistical software R, “geese”, “gls”, “lme”, and “lmer”. In the simulation study, we compare the mean squared error of estimating all the model parameters and compare the coverage proportion of the 95% confidence intervals. In the data analysis, we compare estimation of the intervention effect and the intra-class correlation.

Results

In the simulation study, the function “lme” takes the least computation time. There is no difference in the mean squared error of the four functions. The “lmer” function provides better coverage of the fixed effects when the number of clusters is small as 10. The function “gls” produces close to nominal scale confidence intervals of the intra-class correlation. In the data analysis and the “gls” function yields a positive estimate of the intra-class correlation while the “geese” function gives a negative estimate. Neither of the confidence intervals contains the value zero.

Conclusions

The “gls” function efficiently produces an estimate of the intra-class correlation with a confidence interval. When the within-group correlation is as high as 0.5, the confidence interval is not always obtainable.

Supplementary Information

The online version contains supplementary material available at (10.1186/s13063-021-05900-7).

Power calculation for analyses of cross-sectional stepped-wedge cluster randomized trials with binary outcomes via generalized estimating equations

Power calculation for stepped-wedge cluster randomized trials (SW-CRTs) presents unique challenges, beyond those of standard parallel cluster randomized trials, due to the need to consider temporal within cluster correlations and background period effects. To date, power calculation methods specific to SW-CRTs have primarily been developed under a linear model. When the outcome is binary, the use of a linear model corresponds to assessing a prevalence difference; yet trial analysis often employs a nonlinear link function. We propose power calculation methods for cross-sectional SW-CRTs under a logistic model fitted by generalized estimating equations. Firstly, under an exchangeable correlation structure, we show the power based on a logistic model is lower than that from assuming a linear model in the absence of period effects. We then evaluate the impact of background prevalence changes over time on power. To allow the correlation among outcomes in the same cluster to change over time and with treatment status, we generalize the methods to more complex correlation structures. Our simulation studies demonstrate that the proposed power calculation methods perform well with the model-based variance under the true correlation structure and reveal that a working independence structure can result in substantial efficiency loss, while a working exchangeable structure performs well even when the underlying correlation structure deviates from exchangeable. An extension to our methods accounts for variable cluster sizes and reveals that unequal cluster sizes have a modest impact on power. We illustrate the approaches by application to a quality of care improvement trial for acute coronary syndrome.

Organisational determinants and consequences of diagnostic discrepancy in two large patient groups in the emergency departments: a national study of consecutive episodes between 2008 and 2016

BACKGROUND

Diagnostic discrepancy (DD) is a common phenomenon in healthcare, but little is known about its organisational determinants and consequences. Thus, the aim of the study was to evaluate this among selected emergency department (ED) patients.

METHOD

We conducted an observational study including all consecutive ED patients (hip fracture or erysipelas) in the Danish healthcare sector admitted between 2008 and 2016. DD was defined as a discrepancy between discharge and admission diagnoses. Episode and department statistics were retrieved from Danish registers. We conducted a survey among all 21 Danish EDs to gather information about organisational determinants. To estimate the results while adjusting for episode- and department-level heterogeneity, we used mixed effect models of ED organisational determinants and 30-day readmission, 30-day mortality and episode costs (2018-DKK) of DDs.

RESULTS

DD was observed in 2308 (3.3%) of 69,928 hip fracture episodes and 3206 (8.5%) of 37,558 erysipelas episodes. The main organisational determinant of DD was senior physicians (nonspecific medical specialty) being employed at the ED (hip fracture: odds ratio (OR) 2.74, 95% confidence interval (CI) 2.15-3.51; erysipelas: OR 3.29, 95% CI 2.65-4.07). However, 24-h presence of senior physicians (nonspecific medical specialty) (hip fracture) and availability of external senior physicians (specific medical specialty) (both groups) were negatively associated with DD. DD was associated with increased 30-day readmission (hip fracture, mean 9.45% vs 13.76%, OR 1.46, 95% CI 1.28-1.66, p < 0.001) and episode costs (hip fracture, 61,681 DKK vs 109,860 DKK, log cost 0.58, 95% CI 0.53-0.63, p < 0.001; erysipelas, mean 20,818 DKK vs 56,329 DKK, log cost 0.97, 95% CI 0.92-1.02, p < 0.001) compared with episodes without DD.

CONCLUSION

DD was found to have a negative impact on two out of three study outcomes, and particular organisational characteristics seem to be associated with DD. Yet, the complexity of organisations and settings warrant further studies into these associations.

Impact of unequal cluster sizes for GEE analyses of stepped wedge cluster randomized trials with binary outcomes

The stepped wedge (SW) design is a type of unidirectional crossover design where cluster units switch from control to intervention condition at different prespecified time points. While a convention in study planning is to assume the cluster‐period sizes are identical, SW cluster randomized trials (SW‐CRTs) involving repeated cross‐sectional designs frequently have unequal cluster‐period sizes, which can impact the efficiency of the treatment effect estimator. In this paper, we provide a comprehensive investigation of the efficiency impact of unequal cluster sizes for generalized estimating equation analyses of SW‐CRTs, with a focus on binary outcomes as in the Washington State Expedited Partner Therapy trial. Several major distinctions between our work and existing work include the following: (i) we consider multilevel correlation structures in marginal models with binary outcomes; (ii) we study the implications of both the between‐cluster and within‐cluster imbalances in sizes; and (iii) we provide a comparison between the independence working correlation versus the true working correlation and detail the consequences of ignoring correlation estimation in SW‐CRTs with unequal cluster sizes. We conclude that the working independence assumption can lead to substantial efficiency loss and a large sample size regardless of cluster‐period size variability in SW‐CRTs, and recommend accounting for correlations in the analysis. To improve study planning, we additionally provide a computationally efficient search algorithm to estimate the sample size in SW‐CRTs accounting for unequal cluster‐period sizes, and conclude by illustrating the proposed approach in the context of the Washington State study.