The merits of breaking the matches: a cautionary tale

Geographic pair matching in large-scale cluster randomized trials

Cluster randomized trials are often used to study large-scale public health interventions. In large trials, even small improvements in statistical efficiency can have profound impacts on the required sample size and cost. Location integrates many socio-demographic and environmental characteristics into a single, readily available feature. Here we show that pair matching by geographic location leads to substantial gains in statistical efficiency for 14 child health outcomes that span growth, development, and infectious disease through a re-analysis of two large-scale trials of nutritional and environmental interventions in Bangladesh and Kenya. Relative efficiencies from pair matching are ≥1.1 for all outcomes and regularly exceed 2.0, meaning an unmatched trial would need to enroll at least twice as many clusters to achieve the same level of precision as the geographically pair matched design. We also show that geographically pair matched designs enable estimation of fine-scale, spatially varying effect heterogeneity under minimal assumptions. Our results demonstrate broad, substantial benefits of geographic pair matching in large-scale, cluster randomized trials.

Geographic pair-matching in large-scale cluster randomized trials

Custer randomized trials are often used to study large-scale public health interventions. In large trials, even small improvements in statistical efficiency can have profound impacts on the required sample size and cost. Pair matched randomization is one strategy with potential to increase trial efficiency, but to our knowledge there have been no empirical evaluations of pair-matching in large-scale, epidemiologic field trials. Location integrates many socio-demographic and environmental characteristics into a single feature. Here, we show that geographic pair-matching leads to substantial gains in statistical efficiency for 14 child health outcomes that span growth, development, and infectious disease through a re-analysis of two large-scale trials of nutritional and environmental interventions in Bangladesh and Kenya. We estimate relative efficiencies ≥1.1 for all outcomes assessed and relative efficiencies regularly exceed 2.0, meaning an unmatched trial would have needed to enroll at least twice as many clusters to achieve the same level of precision as the geographically pair-matched design. We also show that geographically pair-matched designs enable estimation of fine-scale, spatially varying effect heterogeneity under minimal assumptions. Our results demonstrate broad, substantial benefits of geographic pair-matching in large-scale, cluster randomized trials.

Matching in cluster randomized trials using the Goldilocks Approach

In group or cluster-randomized trials (GRTs), matching is a technique that can be used to improve covariate balance. When baseline data are available, we suggest a strategy that can be used to achieve the desired balance between treatment and control groups across numerous potential confounding variables. This strategy minimizes the overall within-pair Mahalanobis distance; and involves iteratively: 1) making pairs that minimize the distance between pairs of clusters with respect to potentially confounding variables; 2) visually assessing the potential effects of these pairs and resulting possible randomizations; and 3) reweighting variables of selecting weights to make pairs of clusters. In step 2, we plot the between-arm differences with a parallel-coordinates plot. Investigators can compare plots of different weighting schemes to determine the one that best suits their needs prior to the actual, final, randomization. We demonstrate application of the approach with the Mupirocin-Iodophor Swap Out trial. A webapp is provided.

Efficacy and safety of intermittent preventive treatment and intermittent screening and treatment versus single screening and treatment with dihydroartemisinin-piperaquine for the control of malaria in pregnancy in Indonesia: a cluster-randomised, open-label, superiority trial

BACKGROUND

Plasmodium falciparum and Plasmodium vivax infections are important causes of adverse pregnancy outcomes in the Asia-Pacific region. We hypothesised that monthly intermittent preventive treatment (IPT) or intermittent screening and treatment (IST) with dihydroartemisinin-piperaquine is more effective in reducing malaria in pregnancy than the existing single screening and treatment (SST) strategy, which is used to screen women for malaria infections at the first antenatal visit followed by passive case detection, with management of febrile cases.

METHODS

We did an open-label, three-arm, cluster-randomised, superiority trial in Sumba (low malaria transmission site) and Papua (moderate malaria transmission site), Indonesia. Eligible participants were 16-30 weeks pregnant. Clusters (antenatal clinics with at least ten new pregnancies per year matched by location, size, and malaria risk) were randomly assigned (1:1:1) via computer-generated lists to IPT, IST, or SST clusters. In IPT clusters, participants received the fixed-dose combination of dihydroartemisinin-piperaquine (4 and 18 mg/kg per day). In IST clusters, participants were screened with malaria rapid diagnostic tests once a month, whereas, in SST clusters, they were screened at enrolment only. In all groups, participants with fever were tested for malaria. Any participant who tested positive received dihydroartemisinin-piperaquine regardless of symptoms. The primary outcome was malaria infection in the mother at delivery. Laboratory staff were unaware of group allocation. Analyses included all randomly assigned participants contributing outcome data and were adjusted for clustering at the clinic level. This trial is complete and is registered with ISRCTN, number 34010937.

FINDINGS

Between May 16, 2013, and April 21, 2016, 78 clusters (57 in Sumba and 21 in Papua) were randomly assigned to SST, IPT, or IST clusters (26 clusters each). Of 3553 women screened for eligibility, 2279 were enrolled (744 in SST clusters, 681 in IPT clusters, and 854 in IST clusters). At enrolment, malaria prevalence was lower in IST (5·7%) than in SST (12·6%) and IPT (10·6%) clusters. At delivery, malaria prevalence was 20·2% (128 of 633) in SST clusters, compared with 11·6% (61 of 528) in IPT clusters (relative risk [RR] 0·59, 95% CI 0·42-0·83, p=0·0022) and 11·8% (84 of 713) in IST clusters (0·56, 0·40-0·77, p=0·0005). Conditions related to the pregnancy, the puerperium, and the perinatal period were the most common serious adverse events for the mothers, and infections and infestations for the infants. There were no differences between groups in serious adverse events in the mothers or in their infants.

INTERPRETATION

IST was associated with a lower prevalence of malaria than SST at delivery, but the prevalence of malaria in this group was also lower at enrolment, making interpretation of the effect of IST challenging. Further studies with highly sensitive malaria rapid diagnostic tests should be considered. Monthly IPT with dihydroartemisinin-piperaquine is a promising alternative to SST in areas in the Asia-Pacific region with moderate or high transmission of malaria.

FUNDING

Joint Global Health Trials Scheme of the Medical Research Council, Department for International-Development, and the Wellcome Trust.

A parent-oriented alcohol prevention program "Effekt" had no impact on adolescents' alcohol use: Findings from a cluster-randomized controlled trial in Estonia

AIM

To evaluate the effectiveness of a universal parent-oriented alcohol prevention program ("Effekt") in Estonia. The main objective of the program was to delay and reduce adolescents' alcohol consumption by maintaining parental restrictive attitudes towards adolescents' alcohol use over time.

METHODS

A matched-pair cluster randomized controlled trial with a three-year assessment period (baseline (T1), 18-months (T2) and 30-months (T3) follow-ups) was undertaken in 2012-2015 among 985 fifth grade adolescents and 790 parents in sixty-six schools (34 intervention, 32 control). The primary outcome measure was adolescents' alcohol use initiation. Secondary outcome measures were lifetime drunkenness and alcohol use in the past year. Intermediate outcomes were restrictive parental attitudes towards adolescents' alcohol use reported by parents and perceived restrictive parental attitudes and parental alcohol supply reported by adolescents.

RESULTS

There were no significant differences in adolescents' alcohol use initiation, lifetime drunkenness, alcohol use in the past year, parental alcohol supply, and adolescent's perception of parental restrictive attitudes between intervention and control school participants at T2 and T3. There were significant differences in parental attitudes - the odds of having restrictive attitudes were 2.05 (95% confidence interval (CI) = 1.32-3.17) times higher at T2 and 1.92 (95% CI = 1.31-2.83) times higher at T3 in the intervention group than in the control group.

CONCLUSIONS

The Estonian version of the "Effekt" program had a positive effect on parental attitudes, but it did not succeed in delaying or reducing adolescents' alcohol consumption.

A cluster randomized trial of community mobilization to reduce methamphetamine use and HIV risk among youth in Thailand: Design, implementation and results

BACKGROUND

Use of methamphetamines (MA) and other stimulants have consistently been associated with HIV/STI risk globally. We evaluated a community mobilization intervention (Connect to Protect, C2P^®) to prevent MA use among youth and reduce HIV risk behaviors through community structural changes.

DESIGN

A community-randomized trial in northern Thailand with matched districts randomized to C2P intervention or a standard voluntary counseling and testing (VCT) control. Intervention districts formed stakeholders' coalitions to plan tailored local prevention programs. Two independent random household samples of youth aged 14-24 were surveyed in 2009 and 2012. Lifetime and recent MA use was modeled with multilevel logistic regression.

RESULTS

Intervention initiatives included family-strengthening programs, school-based programs and opening a community drug treatment center. Control communities applied the government-led "war on drugs" approach in addition to youth and family programs. Baseline (N = 1077) and follow-up (N = 1225) samples included 47.5% females and 21.7% aged ≤16. Lifetime MA use in intervention districts reduced from 13.4% to 11.7% compared to reduction from 16.2% to 10.4% in the control districts (non-significant). In models, lifetime MA use was associated with: time (aOR = 0.6, 95%CI: 0.38-0.83), females (aOR = 0.2, 95%CI: 0.15-0.29), increasing age (aOR = 2.4, 95%CI: 1.40-4.20, ages 16-19; aOR = 3.5, 95%CI: 2.00-6.12, ages ≥20), and not being full-time students (aOR = 5.3, 95%CI: 3.77-7.37). Recent MA use showed similar results. Additionally, lifetime MA use was significantly associated with alcohol use, risky sexual behaviors and elevated depressive symptoms.

CONCLUSIONS

Delay in developing and implementing specific prevention programs in the intervention districts slowed diffusion of the effect into the communities. Secular trends with contentious civil unrest and active drug-enforcement efforts in the control communities also contributed to the null intervention effect. Longer time and intensified efforts stressing a public health approach are needed to demonstrate the effectiveness of community mobilization in reducing substance use and HIV risk in this rural Thai community.

Effectiveness of a Household Water, Sanitation and Hygiene Package on an Outpatient Program for Severe Acute Malnutrition: A Pragmatic Cluster-Randomized Controlled Trial in Chad

Water, sanitation and hygiene (WASH) interventions have a small but measurable benefit on stunting, but not on wasting. Our objective was to assess the effectiveness of a household WASH package on the performance of an Outpatient Therapeutic feeding Program (OTP) for severe acute malnutrition (SAM). We conducted a cluster-randomized controlled trial embedded in a routine OTP. The study population included 20 health centers (clusters) from Mao and Mondo districts in Chad. Both arms received the OTP. The intervention arm received an additional household WASH package (chlorine, soap, water storage container, and promotion on its use). The primary objective measures were the relapse rates to SAM at 2 and 6 months post-recovery. The secondary objectives included the recovery rate from SAM, the time-to-recovery, the weight gain, and the diarrhea longitudinal prevalence in OTP. The study lasted from April 2015 to May 2016. Among the 1,603 recruited children, 845 were in the intervention arm and 758 in the control arm. No differences in the relapse rates were noticed at 2 (-0.4%; P = 0.911) and 6 (-1.0%; P = 0.532) months. The intervention decreased the time-to-recovery (-4.4 days; P = 0.038), improved the recovery rate (10.5%; P = 0.034), and the absolute weight gain (3.0 g/d; P = 0.014). No statistical differences were noticed for the diarrhea longitudinal prevalence (-1.7%; P = 0.223) and the weight gain velocity (0.4 g/kg/d; P = 0.086). Our results showed that adding a household WASH package did not decrease post-recovery relapse rates but increased the recovery rate among children admitted in OTP. We recommend further robust trials in other settings to confirm our results.

Effectiveness of a universal classroom-based preventive intervention (PAX GBG): A research protocol for a matched-pair cluster-randomized controlled trial

INTRODUCTION

The PAX Good Behavior Game (PAX GBG) is a universal classroom-based program that promotes children's mental health. In Estonia, the intervention is delivered to first grade students (aged seven to eight) within the regular school curriculum. The current work describes a protocol for a cluster-randomized controlled trial (RCT) of the PAX GBG conducted in Estonia.

DESIGN AND METHODS

This is an ongoing, pragmatic, two-year, matched-pair, cluster-RCT conducted in Estonian elementary schools. Schools were matched to pairs based on their geographical location and number of students per classroom. One school in each pair was randomly selected to receive the intervention and the other placed on a wait-list as a control. 42 schools provided baseline data during the autumn of 2016. Data is collected at two more points in time - seven months and 19 months post-baseline. Outcomes of children's mental health and behavior are measured by the teacher- and parent-rated Strengths and Difficulties Questionnaire, parent-rated Swanson, Nolan, and Pelham - IV Questionnaire and the Go/No-Go task completed by children. Teachers also rate their self-efficacy and overall classroom behavior.

DISCUSSION

This study aims to test the effectiveness of the intervention in Estonian classrooms with a newer version of the rigorously tested GBG program. To our knowledge, this study is the first to measure the effects of the intervention on children's inhibitory control, which has been associated with externalizing problems in the literature. The results from this trial will provide further understanding into how the program influences children's well-being and self-control.

TRIAL REGISTRATION

ClinicalTrials.gov registry (NCT02865603).

Review of Recent Methodological Developments in Group-Randomized Trials: Part 2-Analysis

In 2004, Murray et al. reviewed methodological developments in the design and analysis of group-randomized trials (GRTs). We have updated that review with developments in analysis of the past 13 years, with a companion article to focus on developments in design. We discuss developments in the topics of the earlier review (e.g., methods for parallel-arm GRTs, individually randomized group-treatment trials, and missing data) and in new topics, including methods to account for multiple-level clustering and alternative estimation methods (e.g., augmented generalized estimating equations, targeted maximum likelihood, and quadratic inference functions). In addition, we describe developments in analysis of alternative group designs (including stepped-wedge GRTs, network-randomized trials, and pseudocluster randomized trials), which require clustering to be accounted for in their design and analysis.

Review of Recent Methodological Developments in Group-Randomized Trials: Part 1-Design

In 2004, Murray et al. reviewed methodological developments in the design and analysis of group-randomized trials (GRTs). We have highlighted the developments of the past 13 years in design with a companion article to focus on developments in analysis. As a pair, these articles update the 2004 review. We have discussed developments in the topics of the earlier review (e.g., clustering, matching, and individually randomized group-treatment trials) and in new topics, including constrained randomization and a range of randomized designs that are alternatives to the standard parallel-arm GRT. These include the stepped-wedge GRT, the pseudocluster randomized trial, and the network-randomized GRT, which, like the parallel-arm GRT, require clustering to be accounted for in both their design and analysis.

Patient, Provider, and Combined Interventions for Managing Osteoarthritis in Primary Care: A Cluster Randomized Trial

BACKGROUND

A single-site study showed that a combined patient and provider intervention improved outcomes for patients with knee osteoarthritis, but it did not assess separate effects of the interventions.

OBJECTIVE

To examine whether patient-based, provider-based, and patient-provider interventions improve osteoarthritis outcomes.

DESIGN

Cluster randomized trial with assignment to patient, provider, and patient-provider interventions or usual care. (ClinicalTrials.gov: NCT01435109).

SETTING

10 Duke University Health System community-based primary care clinics.

PARTICIPANTS

537 outpatients with symptomatic hip or knee osteoarthritis.

INTERVENTION

The telephone-based patient intervention focused on weight management, physical activity, and cognitive behavioral pain management. The provider intervention involved electronic delivery of patient-specific osteoarthritis treatment recommendations to providers.

MEASUREMENTS

The primary outcome was the Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) total score at 12 months. Secondary outcomes were objective physical function (Short Physical Performance Battery) and depressive symptoms (Patient Health Questionnaire). Linear mixed models assessed the difference in improvement among groups.

RESULTS

No difference was observed in WOMAC score changes from baseline to 12 months in the patient (-1.5 [95% CI, -5.1 to 2.0]; P = 0.40), provider (2.5 [CI, -0.9 to 5.9]; P = 0.152), or patient-provider (-0.7 [CI, -4.2 to 2.8]; P = 0.69) intervention groups compared with usual care. All groups had improvements in WOMAC scores at 12 months (range, -3.7 to -7.7). In addition, no differences were seen in objective physical function or depressive symptoms at 12 months in any of the intervention groups compared with usual care.

LIMITATIONS

The study involved 1 health care network. Data on provider referrals were not collected.

CONCLUSION

Contrary to a previous study of a combined patient and provider intervention for osteoarthritis in a Department of Veterans Affairs medical center, this study found no statistically significant improvements in the osteoarthritis intervention groups compared with usual care.

PRIMARY FUNDING SOURCE

National Institute of Arthritis and Musculoskeletal and Skin Diseases.

Making Healthy Eating Policy Practice: A Group Randomized Controlled Trial on Changes in Snack Quality, Costs, and Consumption in After-School Programs

PURPOSE

The aim of this study was to evaluate an intervention designed to assist after-school programs (ASPs) in meeting snack nutrition policies that specify that a fruit or vegetable be served daily and sugar-sweetened beverages/foods and artificially flavored foods eliminated.

DESIGN

The study used a 1-year group-randomized controlled trial.

SETTING

The study took place in ASPs operating in South Carolina, United States.

SUBJECTS

Twenty ASPs serving over 1700 children were recruited, match-paired postbaseline on enrollment size and days fruits/vegetables were served per week, and randomized to either intervention (n = 10) or control (n = 10) groups.

INTERVENTION

The study used Strategies To Enhance Practice for Healthy Eating (STEPs-HE), a multistep adaptive intervention framework that assists ASP leaders and staff to serve snacks that meet nutrition policies while maintaining cost.

MEASURES

Direct observation of snacks served and consumed and monthly snack expenditures as determined by receipts were used.

ANALYSIS

The study used nonparametric and mixed-model repeated measures.

RESULTS

By postassessment, intervention ASPs increased serving of fruits/vegetables to 3.9 ± 2.1 vs. 0.7 ± 1.7 d/wk and decreased serving sugar-sweetened beverages to 0.1 ± 0.7 vs. 1.8 ± 2.4 d/wk and sugar-sweetened foods to 0.3 ± 1.1 vs. 2.7 ± 2.5 d/wk compared to controls, respectively. Cost of snacks increased by $0.02/snack in the intervention ASPs ($0.36 to $0.38) compared to a $0.01 per snack decrease in the control group ($0.39 to $0.38). Across both assessments and groups, 80% to 100% of children consumed FVs.

CONCLUSIONS

The STEPs-HE intervention can assist ASPs in meeting nationally endorsed nutrition policies with marginal increases in cost.

Statistical lessons learned for designing cluster randomized pragmatic clinical trials from the NIH Health Care Systems Collaboratory Biostatistics and Design Core

BACKGROUND/AIMS

Pragmatic clinical trials embedded within health care systems provide an important opportunity to evaluate new interventions and treatments. Networks have recently been developed to support practical and efficient studies. Pragmatic trials will lead to improvements in how we deliver health care and promise to more rapidly translate research findings into practice.

METHODS

The National Institutes of Health (NIH) Health Care Systems Collaboratory was formed to conduct pragmatic clinical trials and to cultivate collaboration across research areas and disciplines to develop best practices for future studies. Through a two-stage grant process including a pilot phase (UH2) and a main trial phase (UH3), investigators across the Collaboratory had the opportunity to work together to improve all aspects of these trials before they were launched and to address new issues that arose during implementation. Seven Cores were created to address the various considerations, including Electronic Health Records; Phenotypes, Data Standards, and Data Quality; Biostatistics and Design Core; Patient-Reported Outcomes; Health Care Systems Interactions; Regulatory/Ethics; and Stakeholder Engagement. The goal of this article is to summarize the Biostatistics and Design Core's lessons learned during the initial pilot phase with seven pragmatic clinical trials conducted between 2012 and 2014.

RESULTS

Methodological issues arose from the five cluster-randomized trials, also called group-randomized trials, including consideration of crossover and stepped wedge designs. We outlined general themes and challenges and proposed solutions from the pilot phase including topics such as study design, unit of randomization, sample size, and statistical analysis. Our findings are applicable to other pragmatic clinical trials conducted within health care systems.

CONCLUSION

Pragmatic clinical trials using the UH2/UH3 funding mechanism provide an opportunity to ensure that all relevant design issues have been fully considered in order to reliably and efficiently evaluate new interventions and treatments. The integrity and generalizability of trial results can only be ensured if rigorous designs and appropriate analysis choices are an essential part of their research protocols.

Making policy practice in afterschool programs: a randomized controlled trial on physical activity changes

INTRODUCTION

In the U.S., afterschool programs are asked to promote moderate to vigorous physical activity. One policy that has considerable public health importance is California's afterschool physical activity guidelines that indicate all children attending an afterschool program accumulate 30 minutes each day the program is operating. Few effective strategies exist for afterschool programs to meet this policy goal. The purpose of this study was to evaluate a multistep adaptive intervention designed to assist afterschool programs in meeting the 30-minute/day moderate to vigorous physical activity policy goal.

DESIGN

A 1-year group randomized controlled trial with baseline (spring 2013) and post-assessment (spring 2014). Data were analyzed 2014.

SETTING/PARTICIPANTS

Twenty afterschool programs, serving >1,700 children (aged 6-12 years), randomized to either an intervention (n=10) or control (n=10) group.

INTERVENTION

The employed framework, Strategies To Enhance Practice, focused on intentional programming of physical activity opportunities in each afterschool program's daily schedule and included professional development training to establish core physical activity competencies of staff and afterschool program leaders with ongoing technical assistance.

MAIN OUTCOME MEASURES

The primary outcome was accelerometry-derived proportion of children meeting the 30-minute/day moderate to vigorous physical activity policy.

RESULTS

Children attending intervention afterschool programs had an OR of 2.37 (95% CI=1.58, 3.54) to achieve the physical activity policy at post-assessment compared to control afterschool programs. Sex-specific models indicated that the percentage of intervention girls and boys achieving the physical activity policy increased from 16.7% to 21.4% (OR=2.85, 95% CI=1.43, 5.68) and 34.2% to 41.6% (OR=2.26, 95% CI=1.35, 3.80), respectively. At post-assessment, six intervention afterschool programs increased the proportion of boys achieving the physical activity policy to ≥45% compared to one control afterschool program, whereas three intervention afterschool programs increased the proportion of girls achieving physical activity policy to ≥30% compared to no control afterschool programs.

CONCLUSIONS

The Strategies To Enhance Practice intervention can make meaningful changes in the proportion of children meeting the moderate to vigorous physical activity policy within one school year. Additional efforts are required to enhance the impact of the intervention.

Methods for sample size determination in cluster randomized trials

BACKGROUND

The use of cluster randomized trials (CRTs) is increasing, along with the variety in their design and analysis. The simplest approach for their sample size calculation is to calculate the sample size assuming individual randomization and inflate this by a design effect to account for randomization by cluster. The assumptions of a simple design effect may not always be met; alternative or more complicated approaches are required.

METHODS

We summarise a wide range of sample size methods available for cluster randomized trials. For those familiar with sample size calculations for individually randomized trials but with less experience in the clustered case, this manuscript provides formulae for a wide range of scenarios with associated explanation and recommendations. For those with more experience, comprehensive summaries are provided that allow quick identification of methods for a given design, outcome and analysis method.

RESULTS

We present first those methods applicable to the simplest two-arm, parallel group, completely randomized design followed by methods that incorporate deviations from this design such as: variability in cluster sizes; attrition; non-compliance; or the inclusion of baseline covariates or repeated measures. The paper concludes with methods for alternative designs.

CONCLUSIONS

There is a large amount of methodology available for sample size calculations in CRTs. This paper gives the most comprehensive description of published methodology for sample size calculation and provides an important resource for those designing these trials.

Confidence intervals for a difference between lognormal means in cluster randomization trials

Cluster randomization trials, in which intact social units are randomized to different interventions, have become popular in the last 25 years. Outcomes from these trials in many cases are positively skewed, following approximately lognormal distributions. When inference is focused on the difference between treatment arm arithmetic means, existent confidence interval procedures either make restricting assumptions or are complex to implement. We approach this problem by assuming log-transformed outcomes from each treatment arm follow a one-way random effects model. The treatment arm means are functions of multiple parameters for which separate confidence intervals are readily available, suggesting that the method of variance estimates recovery may be applied to obtain closed-form confidence intervals. A simulation study showed that this simple approach performs well in small sample sizes in terms of empirical coverage, relatively balanced tail errors, and interval widths as compared to existing methods. The methods are illustrated using data arising from a cluster randomization trial investigating a critical pathway for the treatment of community acquired pneumonia.

Estimation of treatment effects in matched-pair cluster randomized trials by calibrating covariate imbalance between clusters

We address estimation of intervention effects in experimental designs in which (a) interventions are assigned at the cluster level; (b) clusters are selected to form pairs, matched on observed characteristics; and (c) intervention is assigned to one cluster at random within each pair. One goal of policy interest is to estimate the average outcome if all clusters in all pairs are assigned control versus if all clusters in all pairs are assigned to intervention. In such designs, inference that ignores individual level covariates can be imprecise because cluster-level assignment can leave substantial imbalance in the covariate distribution between experimental arms within each pair. However, most existing methods that adjust for covariates have estimands that are not of policy interest. We propose a methodology that explicitly balances the observed covariates among clusters in a pair, and retains the original estimand of interest. We demonstrate our approach through the evaluation of the Guided Care program.

The effectiveness of community action in reducing risky alcohol consumption and harm: a cluster randomised controlled trial

BACKGROUND

The World Health Organization, governments, and communities agree that community action is likely to reduce risky alcohol consumption and harm. Despite this agreement, there is little rigorous evidence that community action is effective: of the six randomised trials of community action published to date, all were US-based and focused on young people (rather than the whole community), and their outcomes were limited to self-report or alcohol purchase attempts. The objective of this study was to conduct the first non-US randomised controlled trial (RCT) of community action to quantify the effectiveness of this approach in reducing risky alcohol consumption and harms measured using both self-report and routinely collected data.

METHODS AND FINDINGS

We conducted a cluster RCT comprising 20 communities in Australia that had populations of 5,000-20,000, were at least 100 km from an urban centre (population ≥ 100,000), and were not involved in another community alcohol project. Communities were pair-matched, and one member of each pair was randomly allocated to the experimental group. Thirteen interventions were implemented in the experimental communities from 2005 to 2009: community engagement; general practitioner training in alcohol screening and brief intervention (SBI); feedback to key stakeholders; media campaign; workplace policies/practices training; school-based intervention; general practitioner feedback on their prescribing of alcohol medications; community pharmacy-based SBI; web-based SBI; Aboriginal Community Controlled Health Services support for SBI; Good Sports program for sports clubs; identifying and targeting high-risk weekends; and hospital emergency department-based SBI. Primary outcomes based on routinely collected data were alcohol-related crime, traffic crashes, and hospital inpatient admissions. Routinely collected data for the entire study period (2001-2009) were obtained in 2010. Secondary outcomes based on pre- and post-intervention surveys (n = 2,977 and 2,255, respectively) were the following: long-term risky drinking, short-term high-risk drinking, short-term risky drinking, weekly consumption, hazardous/harmful alcohol use, and experience of alcohol harm. At the 5% level of statistical significance, there was insufficient evidence to conclude that the interventions were effective in the experimental, relative to control, communities for alcohol-related crime, traffic crashes, and hospital inpatient admissions, and for rates of risky alcohol consumption and hazardous/harmful alcohol use. Although respondents in the experimental communities reported statistically significantly lower average weekly consumption (1.90 fewer standard drinks per week, 95% CI = -3.37 to -0.43, p = 0.01) and less alcohol-related verbal abuse (odds ratio = 0.58, 95% CI = 0.35 to 0.96, p = 0.04) post-intervention, the low survey response rates (40% and 24% for the pre- and post-intervention surveys, respectively) require conservative interpretation. The main limitations of this study are as follows: (1) that the study may have been under-powered to detect differences in routinely collected data outcomes as statistically significant, and (2) the low survey response rates.

CONCLUSIONS

This RCT provides little evidence that community action significantly reduces risky alcohol consumption and alcohol-related harms, other than potential reductions in self-reported average weekly consumption and experience of alcohol-related verbal abuse. Complementary legislative action may be required to more effectively reduce alcohol harms.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry ACTRN12607000123448.

Allocation techniques for balance at baseline in cluster randomized trials: a methodological review

Reviews have repeatedly noted important methodological issues in the conduct and reporting of cluster randomized controlled trials (C-RCTs). These reviews usually focus on whether the intracluster correlation was explicitly considered in the design and analysis of the C-RCT. However, another important aspect requiring special attention in C-RCTs is the risk for imbalance of covariates at baseline. Imbalance of important covariates at baseline decreases statistical power and precision of the results. Imbalance also reduces face validity and credibility of the trial results. The risk of imbalance is elevated in C-RCTs compared to trials randomizing individuals because of the difficulties in recruiting clusters and the nested nature of correlated patient-level data. A variety of restricted randomization methods have been proposed as way to minimize risk of imbalance. However, there is little guidance regarding how to best restrict randomization for any given C-RCT. The advantages and limitations of different allocation techniques, including stratification, matching, minimization, and covariate-constrained randomization are reviewed as they pertain to C-RCTs to provide investigators with guidance for choosing the best allocation technique for their trial.

Use of a social and character development program to prevent substance use, violent behaviors, and sexual activity among elementary-school students in Hawaii

OBJECTIVES

We assessed the effectiveness of a 5-year trial of a comprehensive school-based program designed to prevent substance use, violent behaviors, and sexual activity among elementary-school students.

METHODS

We used a matched-pair, cluster-randomized, controlled design, with 10 intervention schools and 10 control schools. Fifth-graders (N = 1714) self-reported on lifetime substance use, violence, and voluntary sexual activity. Teachers of participant students reported on student (N = 1225) substance use and violence.

RESULTS

Two-level random-effects count models (with students nested within schools) indicated that student-reported substance use (rate ratio [RR] = 0.41; 90% confidence interval [CI] = 0.25, 0.66) and violence (RR = 0.42; 90% CI = 0.24, 0.73) were significantly lower for students attending intervention schools. A 2-level random-effects binary model indicated that sexual activity was lower (odds ratio = 0.24; 90% CI = 0.08, 0.66) for intervention students. Teacher reports substantiated the effects seen for student-reported data. Dose-response analyses indicated that students exposed to the program for at least 3 years had significantly lower rates of all negative behaviors.

CONCLUSIONS

Risk-related behaviors were substantially reduced for students who participated in the program, providing evidence that a comprehensive school-based program can have a strong beneficial effect on student behavior.