Interplay between design and analysis for behavioral intervention trials with community as the unit of randomization

Review Article: Learning and knowledge-production for public health: a review of approaches to evidence-based public health

Public health needs to be evidence-based if it is to be done correctly, which means that learning and knowledge-production for public health must be comprehensive and include knowledge from four different domains: distribution of health, determinants or causal web, consequences, and intervention methods. Specification of development trends in cardiovascular prevention points at four generations of preventive programs; these include a clinical, a bioepidemiological, a socioepidemiological, and an environmental and policy-oriented generation. Generations differ in focus and strategy, in knowledge base - the art of making an impact - and evaluation. Comprehensive public health programs often comprise components from different generations, leading to the need for an expanded model for research and evaluation. There is an urgent need for learning and knowledge-production using both quantitative and qualitative approaches for developing the evidence base for public health action. In addition, epidemiological knowledge is necessary for making appropriate priorities.

Effectiveness of health education teachers and school nurses teaching sexually transmitted infections/human immunodeficiency virus prevention knowledge and skills in high school

BACKGROUND

We examined the differential impact of a well-established human immunodeficiency virus (HIV)/sexually transmitted infections (STIs) curriculum, Be Proud! Be Responsible!, when taught by school nurses and health education classroom teachers within a high school curricula.

METHODS

Group-randomized intervention study of 1357 ninth and tenth grade students in 10 schools. Twenty-seven facilitators (6 nurses, 21 teachers) provided programming; nurse-led classrooms were randomly assigned.

RESULTS

Students taught by teachers were more likely to report their instructor to be prepared, comfortable with the material, and challenged them to think about their health than students taught by a school nurse. Both groups reported significant improvements in HIV/STI/condom knowledge immediately following the intervention, compared to controls. Yet, those taught by school nurses reported significant and sustained changes (up to 12 months after intervention) in attitudes, beliefs, and efficacy, whereas those taught by health education teachers reported far fewer changes, with sustained improvement in condom knowledge only.

CONCLUSIONS

Both classroom teachers and school nurses are effective in conveying reproductive health information to high school students; however, teaching the technical (eg, condom use) and interpersonal (eg, negotiation) skills needed to reduce high-risk sexual behavior may require a unique set of skills and experiences that health education teachers may not typically have.

A community-based participatory diabetes prevention and management intervention in rural India using community health workers

PURPOSE

The purpose of this study was to test the effectiveness of a 6-month community-based diabetes prevention and management program in rural Gujarat, India.

METHODS

A community-based participatory research method was used to plan and tailor the intervention by engaging trained community health workers as change agents to provide lifestyle education, serve as community advocates, and collect data from 1638 rural Indians (81.9% response rate). Ten culturally and linguistically appropriate health education messages were provided in face-to-face individual and group sessions (demonstrations of model meals and cooking techniques).

RESULTS

Mean age was 41.9 ± 15.9 years. Overall point prevalence of diabetes, prediabetes, obesity, and hypertension were 7.2%, 19.3%, 16.7%, and 28%, respectively, with significant differences between the low socioeconomic status (SES) participants (agricultural workers) and the high SES participants (business community) due to differing diet and activity levels. The intervention significantly reduced blood glucose levels by 5.7 and 14.9 mg/dL for individuals with prediabetes and diabetes, respectively, and systolic and diastolic blood pressure by 8 mm Hg and 4 mm Hg, respectively, in the overall population. Knowledge of diabetes and cardiovascular disease improved by 50% in the high SES group and doubled in the low SES group; general and abdominal obesity also decreased by ≤ 1%. High rates of undiagnosed hypertension (26.1%) were surprising. Among individuals with diabetes, metabolic complications such as diabetic nephropathy and chronic kidney disease were noted.

CONCLUSIONS

Through collective engagement of the community, participatory programs can serve as a prototype for future prevention and management efforts, which are rare and underutilized in India.

Imputation strategies for missing binary outcomes in cluster randomized trials

Background

Attrition, which leads to missing data, is a common problem in cluster randomized trials (CRTs), where groups of patients rather than individuals are randomized. Standard multiple imputation (MI) strategies may not be appropriate to impute missing data from CRTs since they assume independent data. In this paper, under the assumption of missing completely at random and covariate dependent missing, we compared six MI strategies which account for the intra-cluster correlation for missing binary outcomes in CRTs with the standard imputation strategies and complete case analysis approach using a simulation study.

Method

We considered three within-cluster and three across-cluster MI strategies for missing binary outcomes in CRTs. The three within-cluster MI strategies are logistic regression method, propensity score method, and Markov chain Monte Carlo (MCMC) method, which apply standard MI strategies within each cluster. The three across-cluster MI strategies are propensity score method, random-effects (RE) logistic regression approach, and logistic regression with cluster as a fixed effect. Based on the community hypertension assessment trial (CHAT) which has complete data, we designed a simulation study to investigate the performance of above MI strategies.

Results

The estimated treatment effect and its 95% confidence interval (CI) from generalized estimating equations (GEE) model based on the CHAT complete dataset are 1.14 (0.76 1.70). When 30% of binary outcome are missing completely at random, a simulation study shows that the estimated treatment effects and the corresponding 95% CIs from GEE model are 1.15 (0.76 1.75) if complete case analysis is used, 1.12 (0.72 1.73) if within-cluster MCMC method is used, 1.21 (0.80 1.81) if across-cluster RE logistic regression is used, and 1.16 (0.82 1.64) if standard logistic regression which does not account for clustering is used.

Conclusion

When the percentage of missing data is low or intra-cluster correlation coefficient is small, different approaches for handling missing binary outcome data generate quite similar results. When the percentage of missing data is large, standard MI strategies, which do not take into account the intra-cluster correlation, underestimate the variance of the treatment effect. Within-cluster and across-cluster MI strategies (except for random-effects logistic regression MI strategy), which take the intra-cluster correlation into account, seem to be more appropriate to handle the missing outcome from CRTs. Under the same imputation strategy and percentage of missingness, the estimates of the treatment effect from GEE and RE logistic regression models are similar.

Use of group-randomized trials in pet population research

Communities invest considerable resources to address the animal welfare and public health concerns resulting from unwanted pet animals. Traditionally, research in this area has enumerated the pet-owning population, described pet population dynamics in individual communities, and estimated national euthanasia figures. Recent research has investigated the human-animal bond and explored the community implications of managed feral cat colonies. These reports have utilized traditional epidemiologic study designs to generate observational data to describe populations and measure associations. However, rigorous scientific evaluations of potential interventions at the group level have been lacking. Group-randomized trials have been used extensively in public health research to evaluate interventions that change a population's behavior, not just the behavior of selected individuals. We briefly describe the strengths and limitations of group-randomized trials as they are used to evaluate interventions that promote social and behavioral changes in the human public health field. We extend these examples to suggest the appropriate application of group-randomized trials for pet population dynamics research.

Limitations of the randomized controlled trial in evaluating population-based health interventions

Population- and systems-based interventions need evaluation, but the randomized controlled trial (RCT) research design has significant limitations when applied to their complexity. After some years of being largely dismissed in the ranking of evidence in medicine, alternatives to the RCT have been debated recently in public health and related population and social service fields to identify the trade-offs in their use when randomization is impractical or unethical. This review summarizes recent debates and considers the pragmatic and economic issues associated with evaluating whole-population interventions while maintaining scientific validity and credibility.

Effect evaluation of a multifactor community intervention to reduce falls among older persons

The objective of the study was to evaluate the effectiveness of a multifactor and multimethod community intervention programme to reduce falls among older persons by at least 20%. In a pre-test-post test design, self-reported falls were registered for 10 months in the intervention community and two control communities. After the pre-test registration, participants followed the intervention programme (Information and education, Training and exercise and Environmental modifications) for 14 months. All communities were situated in the Province of Friesland in the north of The Netherlands. The study ran from November 1999 to November 2002. The participants (intervention 1122; control 630) were aged 65 years and older and lived independently. There was no significant decrease in total falls, except outside the home, among women (odds ratio = 0.54, 95% CI = 0.30-0.98; p = 0.041). The multifactor intervention programme was not effective. The reduction of falls outside the home among women could be due to reduced outdoor physical activity.

Longitudinal and repeated cross-sectional cluster-randomization designs using mixed effects regression for binary outcomes: bias and coverage of frequentist and Bayesian methods

As medical applications for cluster randomization designs become more common, investigators look for guidance on optimal methods for estimating the effect of group-based interventions over time. This study examines two distinct cluster randomization designs: (1) the repeated cross-sectional design in which centres are followed over time but patients change, and (2) the longitudinal design in which individual patients are followed over time within treatment clusters. Simulations of each study design stipulated a multiplicative treatment effect (on the log odds scale), between 5 and 15 clusters in each of two treatment arms, and followed over two time periods. Estimation options included linear mixed effects models using restricted maximum likelihood (REML), generalized estimating equations (GEE), mixed effects logistic regression using both penalized quasi likelihood (PQL) and numerical integration, and Bayesian Monte Carlo analysis. For the repeated cross-sectional designs, most methods performed well in terms of bias and coverage when clusters were numerous (30) and variability across clusters of baseline risk and treatment effect was modest. With few clusters (two groups of five) and higher variability, only the Bayesian methods maintained coverage. In the longitudinal designs, the common methods of REML, GEE, or PQL performed poorly when compared to numerical integration, while Bayesian methods demonstrated less bias and better coverage for estimates of both log odds ratios and risk differences. The performance of common statistical tools for the analysis of cluster randomization designs depends heavily on the precise design, the number of clusters, and the variability of baseline outcomes and treatment effects across centres.

A comparison of permutation and mixed-model regression methods for the analysis of simulated data in the context of a group-randomized trial

Our first purpose was to determine whether, in the context of a group-randomized trial (GRT) with Gaussian errors, permutation or mixed-model regression methods fare better in the presence of measurable confounding in terms of their Monte Carlo type I error rates and power. Our results indicate that given a proper randomization, the type I error rate is similar for both methods, whether unadjusted or adjusted, even in small studies. However, our results also show that should the investigator face the unfortunate circumstance in which modest confounding exists in the only realization available, the unadjusted analysis risks a type I error; in this regard, there was little to distinguish the two methods. Finally, our results show that power is similar for the two methods and, not surprisingly, better for the adjusted tests. Our second purpose was to examine the relative performance of permutation and mixed-model regression methods in the context of a GRT when the normality assumptions underlying the mixed model are violated. Published studies have examined the impact of violation of this assumption at the member level only. Our findings indicate that both methods perform well when the assumption is violated so long as the ICC is very small and the design is balanced at the group level. However, at ICC>or=0.01, the permutation test carries the nominal type I error rate while the model-based test is conservative and so less powerful. Binomial group- and member-level errors did not otherwise change the relative performance of the two methods with regard to confounding.

Public health impact of community-based nutrition and lifestyle interventions

Community-based interventions have increasingly received attention since researchers and public health professionals have come to acknowledge the importance of an environment that makes the healthy choice the easy choice. All stakeholders including the target community are involved to achieve changes in legislation, in people's social and physical context, and in individual characteristics that support healthy diets and other lifestyles. Some early large-scale community-based heart health interventions showed promising results. The Stanford Five City Project, for example, showed net improvements in knowledge of coronary heart disease risk factors of approximately 12%. Net declines in smoking prevalence (14%), cholesterol (2%), and systolic (3%) and diastolic (5%) blood pressure were also observed. Most later studies did not replicate these findings and it was therefore suggested that community-based interventions, which require substantial commitment and resources, may be less effective than approaches targeting high-risk groups. We present the rationale and theories for community-based interventions, and then elaborate on the methodological challenges in the design and the outcome and process evaluation of community-based interventions. We provide an overview of some of the evidence on the effectiveness of community-based heart health interventions and conclude with the perspectives for community-based interventions in future research and practice.

Reliability, effect size, and responsiveness of health status measures in the design of randomized and cluster-randomized trials

BACKGROUND

New health status survey instruments are often described by their psychometric (measurement) properties, such as Validity, Reliability, Effect Size, and Responsiveness. For cluster-randomized trials, another important statistic is the Intraclass Correlation (ICC) for the instrument within clusters. Studies using better instruments can be performed with smaller sample sizes, but better instruments may be more expensive in terms of dollars, opportunity cost, or poorer data quality due to the response burden of longer instruments.

METHODS

We defined the psychometric statistics in terms of a mathematical model, and examined the power of a two-sample test as a function of the test-retest Reliability, Effect Size, Responsiveness, and Intraclass Correlation of the instrument. We examined the "cost-effectiveness" of using a one-item versus a five-item measure of mental health status.

FINDINGS

Under the standard model for measurement error, the psychometric statistics are all functions of the same error term. They are also functions of the setting in which they were estimated. In randomized trials, power is a function of Reliability and sample size, and a less reliable instrument can achieve the desired power if N is increased. In cluster-randomized trials, adequate power may be obtained by increasing the number of clusters per treatment group (and often the number of persons per cluster), as well as by choosing a more reliable instrument. The one-item measure of mental health status may be more cost-effective than the five-item measure in some situations.

CONCLUSION

If the goal is to diagnose or refer individual patients, an instrument with high Validity and Reliability is needed. In settings where the sample sizes are large or can be increased easily, any valid instrument may be cost-effective. It is likely that many published values of psychometric statistics are accurate only in settings similar to that in which they were estimated.

A naturalistic inquiry on the impact of interventions aiming to improve health and the quality of life in the community

The goal of this study is to identify and describe variables contributing to the efficiency of health promotion interventions, and to assess whether these variables can serve as reliable and early indicators of the success of such interventions. The study sample includes 44 interventions selected through a network of key informants from five cities--Liverpool, Sandwell, Vienna, Pula, and Rijeka--by using a chain technique. Data on each intervention are collected through an in-depth interview with a program leader, the collection of project-related documents, and on-site observation. Qualitative analysis of data performed with content analysis and computer-assisted free-text analysis reveals different characteristics of interventions depending on whether they are initiated by the city government sector, health-care system, or citizens sector (independent of the city or country). The assessment of the efficiency of these three groups of interventions also differs because of varying features, scope (activity potentials) and impact they are able to accomplish. We have identified ways in which the efficiency of all three groups of interventions can be improved. The efficiency of the interventions within the city sector can be increased through an improved process of delegation to other sectors, higher involvement of user groups, and higher receptivity and organizational flexibility. The efficiency of the interventions within the citizens sector can be improved through professional, organizational, and financial support. Support from the professional community is important for citizens sector interventions in confirming the importance of the problem they address and legitimizing the actions they propose and undertake.

Implementing evidence-based guidelines for radiography in acute low back pain: a pilot study in a chiropractic community

OBJECTIVE

To evaluate the ability of a systematic educational intervention strategy to change the plain radiography ordering behavior of chiropractors toward evidence-based practice for patients with acute low back pain (LBP).

DESIGN

A quasi-experimental method was used comparing outcomes before and after the intervention with those of a control community.

SETTING

Two communities in southern Ontario.

DATA SOURCE

Mailed survey data on the management of acute LBP. Outcome Measures Plain radiography use rates for acute LBP based on responses to mailed surveys.

RESULTS

Following the intervention, there was a 42% reduction in the self-report need for plain radiography for uncomplicated acute LBP (P <.025) and a 50% reduction for patients with acute LBP < 1 month (P <.025) in the intervention community. There was no significant change in the self-report need for plain radiography in the control community (P >.05).

CONCLUSIONS

The educational intervention strategy used in this study appeared to have an effect in reducing the perceived need for plain radiography in acute LBP.

Mounting a community-randomized trial: sample size, matching, selection, and randomization issues in PRISM

This paper discusses some of the processes for establishing a large cluster-randomized trial of a community and primary care intervention in 16 local government areas in Victoria, Australia. The development of the trial in terms of design factors such as sample size estimates and the selection and randomization of communities to intervention or comparison is described. The intervention program to be implemented in Program of Resources, Information and Support for Mothers (PRISM) was conceived as a whole community approach to improving support for all mothers in the first 12 months after birth. A cluster-randomized trial was thus the design of choice from the outset. With a limited number of communities available, a matched-pair design with eight pairs was chosen. Sample size estimates, adjusting for the cluster randomization and the pair-matched design, showed that with eight pairs, on average, 800 women from each community would need to respond to provide sufficient power to determine a 3% reduction in the prevalence of maternal depression 6 months after birth-a reduction deemed to be a worthwhile impact of the intervention to be reliably detected at 80% power. The process of selecting suitable communities and matching them into pairs required careful collection of data on numbers of births, size of the local government areas (LGAs), and an assessment of the capacity of communities to implement the intervention. Ways of dealing with boundary issues associated with potential contamination are discussed. Methods for the selection of feasible configurations of sets of pairs and the ultimate allocation to intervention or comparison are provided in detail. Ultimately, all such studies are a balancing act between selecting the minimum number of communities to detect a meaningful outcome effect of an intervention and the maximum size budget and other resources allow.

Evidence-based behavioral medicine: what is it and how do we achieve it?

The goal of evidence-based medicine is ultimately to improve patient outcomes and quality of care. Systematic reviews of the available published evidence are required to identify interventions that lead to improvements in behavior, health, and well-being. Authoritative literature reviews depend on the quality of published research and research reports. The Consolidated Standards for Reporting Trials (CONSORT) Statement (www.consort-statement.org) was developed to improve the design and reporting of interventions involving randomized clinical trials (RCTs) in medical journals. We describe the 22 CONSORT guidelines and explain their application to behavioral medicine research and to evidence-based practice. Additional behavioral medicine-specific guidelines (e.g., treatment adherence) are also presented. Use of these guidelines by clinicians, educators, policymakers, and researchers who design, report, and evaluate or review RCTs will strengthen the research itself and accelerate efforts to apply behavioral medicine research to improve the processes and outcomes of behavioral medicine practice.

Statistical and Substantive Inferences in Public Health: Issues in the Application of Multilevel Models

Multilevel statistical models have become increasingly popular among public health researchers over the past decade. Yet the enthusiasm with which these models are being adopted may obscure rather than solve some problems of statistical and substantive inference. We discuss the three most common applications of multilevel models in public health: (a) cluster-randomized trials, (b) observational studies of the multilevel etiology of health and disease, and (c) assessments of health care provider performance. In each area of investigation, we describe how multilevel models are being applied, comment on the validity of the statistical and substantive inferences being drawn, and suggest ways in which the strengths of multilevel models might be more fully exploited. We conclude with a call for more careful thinking about multilevel causal inference.

Methodologic advances and ongoing challenges in designing community-based health promotion programs

Community intervention trials continue to attract researchers as potential ways to achieve widespread, long-term change in health behaviors. The first generations of community studies were somewhat unsophisticated in design and analysis, and their promise may have been overstated. As design and analysis issues were better defined, as secular trends caught up with the behaviors that researchers were trying to change, or as other unknown variables affected community studies, small effects of interventions were observed in community trials. Discussions were held in professional meetings and reported in the literature: Should community trials be discontinued? In general, the answer was a qualified no. In this paper, we briefly review some of the many advances made in community intervention trials, and address in more detail the challenges ahead.

Development and Evaluation of “Aging Well and Healthily”: A Health-Education and Exercise Program for Community-Living Older Adults

The Aging Well and Healthily (AWH) program consists of health education by peers and low-intensity exercise. It was evaluated via a small randomized controlled trial and a community intervention trial involving older adults in the Netherlands. Reasons stated for participation were to exercise (35%), to acquire information about health (28%), and for social reasons (12%). The program was rated 8.2 on a 10-point scale. Twenty-five percent of participants joined exercise groups after the program ended, and 28% intended to do so. The mean physical activity score improved from 2.6 to 4.6 at follow-up (F = 16.9, p = .00) and was for the least active participants significantly different from that of the control group (F = 22.9, p = .02). Four to 6 months later, 60% of respondents reported still doing the exercises regularly at home. It is concluded that AWH is a potentially effective program for older adults.

Application of the development stages of a cluster randomized trial to a framework for valuating complex health interventions

INTRODUCTION

Trials of complex health interventions often pose difficult methodologic challenges. The objective of this paper is to assess the extent to which the various development steps of a cluster randomized trial to optimize antibiotic use in nursing homes are represented in a recently published framework for the design and evaluation of complex health interventions. In so doing, the utility of the framework for health services researchers is evaluated.

METHODS

Using the five phases of the framework (theoretical, identification of components of the intervention, definition of trial and intervention design, methodological issues for main trial, promoting effective implementation), corresponding stages in the development of the cluster randomized trial using diagnostic and treatment algorithms to optimize the use of antibiotics in nursing homes are identified and described.

RESULTS

Synthesis of evidence needed to construct the algorithms, survey and qualitative research used to define components of the algorithms, a pilot study to assess the feasibility of delivering the algorithms, methodological issues in the main trial including choice of design, allocation concealment, outcomes, sample size calculation, and analysis are adequately represented using the stages of the framework.

CONCLUSIONS

The framework is a useful resource for researchers planning a randomized clinical trial of a complex intervention.

Balance in cluster randomized trials

This paper explores the role of balancing covariates between treatment groups in the design of cluster randomized trials. General expressions are obtained for two criteria to evaluate designs for parallel group studies with two treatments. The first is the variance of the estimated treatment effect and the second is the extent to which the estimated treatment effect is changed by adjusting for covariates. It is argued that the second of these is more important for cluster randomized trials. Methods of obtaining balanced designs from covariates which are available at the start of a study are proposed. An imbalance measure is used to compare the extent to which designs balance important covariates between the arms of a trial. Several approaches to selecting a well balanced design are possible. A method that randomly selects one member from the class of designs with acceptable bias will allow randomization inference as well as model-based inference. The methods are illustrated with data from a trial of school-based sex education.

Dealing with diversity: recruiting churches and women for a randomized trial of mammography promotion

There is little documentation about the recruitment process for church-based health education programs. In this study, the authors recruit African American, Latino, and white churches and women members (age 50 to 80) for a randomized church-based trial of mammography promotion in Los Angeles County. Efforts to enhance recruitment began 10 months before churches were invited to participate and included a variety of community-based strategies. Subsequently, 45 churches were recruited over a 5-month period through group pastor breakfast meetings and church-specific follow-up. In close collaboration with the 45 churches, the authors administered church-based surveys over 6 months and identified 1,967 age-eligible women who agreed to be contacted by the program team. It was found that an extended resource intensive period of relationship-building and community-based activities were necessary to conduct church-based programs effectively, particularly among older and ethnically diverse urban populations.

The female condom and STDs: design of a community intervention trial

OBJECTIVES

The main purpose of this study is to compare sexually transmitted disease (STD) prevalence in cohorts of women with and without access to female condoms.

METHODS

Six matched pairs of communities were identified from Kenya tea, coffee and flower plantations. One community within each pair was randomly selected to receive the female condom intervention. Approximately 160 eligible women were enrolled at each site. Female condom communities underwent an education program on use of female and male condoms and STDs, comprising group meetings, puppetry and other folk media, and training of clinic service providers and community outreach workers. Control communities received similar information on use of male condoms (freely available at all sites). At baseline, participants were tested for cervical gonorrhea and chlamydia and vaginal trichomoniasis, to be repeated at 6 and 12 months. The study has 80% power to detect a 10% prevalence difference, assuming an aggregate STD prevalence of 20% with 25% loss to follow-up and intracluster correlation of 0.03.

RESULTS

Among 1929 women at baseline, the mean age was 33.1 years; 78% had never used a male condom. The prevalences of gonorrhea, chlamydia and trichomoniasis were 2.6%, 3. 2% and 20.4%, respectively (23.9% overall). The intracluster correlation based on these data was near zero.

CONCLUSIONS

Comparable pairs of study sites have been selected. STD prevalence is sufficiently high, and the variation between sites is acceptably low. The study is feasible as designed.

Hypothesis testing in clinical trials

In designing and analyzing any clinical trial, two issues related to patient heterogeneity must be considered: (1) the effect of chance and (2) the effect of bias. These issues are addressed by enrolling adequate numbers of patients in the study and using randomization for treatment assignment. An "intention-to-treat" analysis of outcome data includes all individuals randomized and counted in the group to which they are randomized. There is an increased risk of spurious results with a greater number of subgroup analyses, particularly when these analyses are data derived. Factorial designs are sometimes appropriate and can lead to efficiencies by addressing more than one comparison of interventions in a single trial.

Firearm safety counseling in primary care pediatrics: a randomized, controlled trial

CONTEXT

Physicians have been encouraged to counsel families about risks associated with gun ownership, but the effectiveness of physician counseling regarding gun safety is unknown.

OBJECTIVE

To determine the effectiveness of gun safety counseling during well-child care visits.

DESIGN

Group randomized, controlled trial. Physicians, nurse practitioners, and physician assistants were randomized to either the intervention group or a control group.

SETTING

Group Health Cooperative, a staff-model health maintenance organization. Patients. Consecutive sample of families (n = 1295) seen for a scheduled appointment for well-child care for a child <18 years of age. Of the families originally scheduled for a visit, 80. 3% were seen and completed the outcomes surveys.

INTERVENTION

Each family in the intervention group was given a 60-second message by their practitioner that depended on the presence of guns in the home. Families without guns were informed of the health risks associated with gun ownership and given a standard information pamphlet. Families with guns were given the same information about risks and were told that if they chose to keep a gun, they should store it locked and unloaded. They were given instructions on storage and a folder with material, including the same pamphlet, a letter from the police department, written storage guidelines, and discount coupons for gun storage devices.

MAIN OUTCOME MEASURE

Changes in the following self-reported events: 1) acquisition of a safe storage device; 2) removal of firearms from the home; and 3) acquisition of firearms. Results. There were no important differences between intervention and control groups in the rate of acquisition of new guns (intervention: 1.3% vs control:.9%) after the intervention. Among households with guns at baseline, there were also no differences between groups in the removal of guns (intervention: 6.7% vs control: 5.7%), but there was a fairly large nonsignificant difference in the proportion who purchased trigger locks (intervention: 8.0% vs control: 2.5%).

CONCLUSIONS

A single firearm safety-counseling session during well-child care, combined with economic incentives to purchase safe storage devices, did not lead to changes in household gun ownership and did not lead to statistically significant overall changes in storage patterns.

Using mass media to promote healthy eating: A community-based demonstration project

BACKGROUND

Many question whether mass media, in the absence of other programming, can produce significant and sustained behavior change.

METHODS

The 1% Or Less campaign in Wheeling, West Virginia (population 35,000), used paid advertising and public relations to encourage members of one community to switch from whole or 2% milk (high-fat milk) to 1% or fat-free milk (low-fat milk). The study used a quasi-experimental research design with one intervention city and one comparison city. The effectiveness of the campaign was evaluated by collecting milk sales data from supermarkets and conducting pre- and post-intervention telephone surveys in intervention and comparison cities.

RESULTS

In the intervention city, low-fat milk sales increased from 29% of overall milk sales before the campaign to 46% of sales in the month following the campaign. The increase was maintained at the 6-month follow up. According to the telephone surveys, 34.1% of high-fat-milk drinkers reported switching to low-fat milk in the intervention community compared with 3.6% in the comparison community (z = 13.1, P < 0.0001).

CONCLUSIONS

A media-only approach was sufficient to encourage a significant proportion of the people in one community to alter the dietary habit targeted by the intervention.

Rapid early action for coronary treatment: rationale, design, and baseline characteristics. REACT Research Group

OBJECTIVE

Early reperfusion for acute myocardial infarction (AMI) can reduce morbidity and mortality, yet there is often delay in accessing medical care after symptom onset. This report describes the design and baseline characteristics of the Rapid Early Action for Coronary Treatment (REACT) community trial, which is testing community intervention to reduce delay.

METHODS

Twenty U.S. communities were pair-matched and randomly assigned within pairs to intervention or comparison. Four months of baseline data collection was followed by an 18-month intervention of community organization and public, patient, and health professional education. Primary cases were community residents seen in the ED with chest pain, admitted with suspected acute cardiac ischemia, and discharged with a diagnosis related to coronary heart disease. The primary outcome was delay time from symptom onset to ED arrival. Secondary outcomes included delay time in patients with MI/unstable angina, hospital case-fatality rate and length of stay, receipt of reperfusion, and ED/emergency medical services utilization. Impact on public and patient knowledge, attitudes, and intentions was measured by telephone interviews. Characteristics of communities and cases and comparability of paired communities at baseline were assessed.

RESULTS

Baseline cases are 46% female, 14% minorities, and 73% aged > or =55 years, and paired communities have similar demographics characteristics. Median delay time (available for 72% of cases) is 2.3 hours and does not vary between treatment conditions (p > 0.86).

CONCLUSIONS

REACT communities approximate the demographic distribution of the United States and there is baseline comparability between the intervention and comparison groups. The REACT trial will provide valuable information for community educational programs to reduce patient delay for AMI symptoms.

Implications of the results of community intervention trials

This paper examines the results of population-level interventions conducted in three settings: entire communities, worksites, and schools. Four major conclusions are discussed: (a) Directions for the next generation of community-based interventions include targeting multiple levels of influence; addressing social inequalities in disease risk; involving communities in program planning and implementation; incorporating approaches for "tailoring" interventions; and utilizing rigorous process evaluation. (b) In addition to randomized controlled trials, it is time to use the full range of research phases available, from hypothesis generation and methods development to dissemination research. (c) The public health research agenda may have contributed to observed secular trends by placing behavioral risk factors on the social and media agendas. (d) The magnitude of the results of community intervention trials must be judged according to their potential public health or population-level effects. Small changes at the individual level may result in large benefits at the population level.

Designing an evaluation for a multiple-strategy community intervention: the North Coast Stay on Your Feet program

Evaluation of the North Coast Stay on Your Feet falls prevention program is described as a case study of a comprehensive evaluation design for multi-strategic community interventions. Qualitative and quantitative methods were used to evaluate the program at formative, process and outcome levels. Formative evaluation used literature review, focus groups, mail-out and telephone survey methods to gather evidence from publications, older people, health workers, local business, media and government bodies. It included an analysis of demographic and hospital databases and identified incidence, causal pathways, knowledge, attitudes, behaviour, consequences and effectiveness of potential strategies. Process evaluation employed auditing, monitoring and telephone surveys to maintain an inventory of intervention activities and to track the reach of the program. Outcome evaluation involved a longitudinal study of intervention and control cohorts, surveyed before, during and after the intervention by telephone to monitor changes in knowledge, attitudes, risk and falls incidence. The survey instrument was designed for both formative and outcome evaluation, and analysis reflected the research design by incorporating repeat measures and adjusting for bias and confounding. Outcome validity was cross-checked via hospital admission rates. A novel, integrated framework for presenting inputs, activities and outcomes from all stages of the program is described. This framework facilitated feedback to stakeholders and enabled subsequent rapid adjustment of the intervention. Rigorous evaluation combined with clear presentation of findings helped to engender intersectoral support and obtain funding grants for extended implementation and evaluation. It also helped Stay on Your Feet to become a model for other falls prevention programs within Australia and internationally.

The advantages of community-randomized trials for evaluating lifestyle modification

Observational studies may provide suggestive evidence for the results of behavior change and lifestyle modification, but they do not replace randomized trials for comparing interventions. To obtain a valid comparison of competing intervention strategies, randomized trials of adequate size are the recommended approach. Randomization avoids bias, achieves balance (on average) of both known and unknown predictive factors between intervention and comparison groups, and provides the basis of statistical tests. The value of randomization is as relevant when investigating community interventions as it is for studies that are directed at individuals. Randomization by group is less efficient statistically than randomization by individual, but there are reasons why randomization by group (such as community) may be chosen, including feasibility of delivery of the intervention, political and administrative considerations, avoiding contamination between individuals allocated to competing interventions, and the very nature of the intervention. One example is the Community Intervention Trial for Smoking Cessation (COMMIT), which involved 11 matched pairs of communities and randomized within these pairs to active community-level intervention versus comparison. For analysis of results, community-level permutation tests (and corresponding test-based confidence intervals) can be designed based on the randomization distribution. The advantages of this approach are that it is robust, and the unit of randomization is the unit of analysis, yet it can incorporate individual-level covariates. Such covariates can play a role in imputation for missing values, adjustment for imbalances, and separate analyses in demographic subsets (with appropriate tests for interaction). A community-randomized trial can investigate a multichannel community-based approach to lifestyle modification, thus providing generalizability coupled with a rigorous evaluation of the intervention.

How should interventions to reduce inequalities in health be evaluated?

OBJECTIVE

The effectiveness of interventions which have been proposed or are currently in progress to reduce socioeconomic inequalities in health is largely unknown. This paper aims to develop guidelines for evaluating these interventions.

APPROACH

Starting from a set of general guidelines which was recently proposed by a group of experts reporting to the national Programme Committee on Socioeconomic Inequalities in Health in The Netherlands, an analysis was made of the appropriateness of different study designs which could be used to assess the effectiveness of interventions to reduce inequalities in health.

RESULTS

A "full" study design requires the measurement, in one or more experimental populations and one or more control populations, of changes over time in the magnitude of socioeconomic inequalities in health. This will usually imply a community intervention trial. Five alternative study designs are distinguished which require less complex measurements but also require more assumptions to be made. Several examples are given.

CONCLUSIONS

Building up a systematic knowledge base on the effectiveness of interventions to reduce socioeconomic inequalities in health will be a major enterprise. Elements of a strategy to increase learning speed are discussed. Although the guidelines and design recommendations developed in this paper apply to the evaluation of specific interventions where rigorous evaluation methods can often be used, they may also be useful for the interpretation of the results of less rigorous evaluation studies, for example of broader policies to reduce socioeconomic inequalities in health.

Batch sampling to improve power in a community trial. Experience from the Pawtucket Heart Health Program

Experiments involving large social units, such as schools, work sites, or whole cities, are commonly limited in statistical power because the number of randomized units is small, leaving few degrees of freedom for residual (between-unit) error. The authors describe a method for increasing residual degrees of freedom in a community experiment without substantially increasing cost or difficulty. In brief, they propose that the experimental units should be divided into random subsamples (batches). Batch sampling can improve statistical power if the community endpoint means are stable over time or if their temporal variation is comparable in period to the batch-sampling schedule. The authors demonstrate the theoretical advantages of the batch system and illustrate its use with data from the Pawtucket Heart Health Program, in which such a design was implemented.