Design and analysis of group-randomized trials: a review of recent practices

Study design and baseline description of the BMI2 trial: reducing paediatric obesity in primary care practices

OBJECTIVES

This study will test the efficacy of motivational interviewing (MI) conducted by primary care providers and dieticians among children ages 2-8 years old with a body mass index (BMI) ≥ 85th and ≤ 97th percentile.

METHODS

Forty-two practices from the American Academy of Pediatrics, Pediatric Research in Office Settings Network were assigned to one of three groups. Group 1 (usual care) measures BMI percentile at baseline, and at 1- and 2-year follow-ups and receives standard health education materials. Group 2 providers deliver three proactive MI counselling sessions with a parent of the index child in Year 1 and one additional 'booster' visit in Year 2. Group 3 adds six MI counselling sessions from a trained dietician. The primary outcome is the child's BMI percentile at 2-year follow-up. Secondary outcomes include parent report of the child's screen time, physical activity, intake of fruits and vegetables, and sugar-sweetened beverages.

RESULTS

We enrolled 633 eligible children whose mean BMI percentile was 92.0 and mean age of 5.1. The cohort was 57% female. Almost 70% of parents reported a household income of ≥ $40,000 per year, and 39% had at least a college education. The cohort was 63% white, 23% Hispanic, 7% black and 7% Asian. Parent self-reported confidence that their child will achieve a healthy weight was on average an 8 (out of 10).

CONCLUSION

To date, several aspects of the study can inform similar efforts including our ability to use volunteer clinicians to recruit participants and their willingness to dedicate their time, without pay, to receive training in MI.

Guidance for evidence-informed policies about health systems: assessing how much confidence to place in the research evidence

In the third paper in a three-part series on health systems guidance, Simon Lewin and colleagues explore the challenge of assessing how much confidence to place in evidence on health systems interventions.

Outcome expectancy as a predictor of treatment response in cognitive behavioral therapy for public speaking fears within social anxiety disorder

Outcome expectancy, the extent that clients anticipate benefiting from therapy, is theorized to be an important predictor of treatment response for cognitive-behavioral therapy. However, there is a relatively small body of empirical research on outcome expectancy and the treatment of social anxiety disorder. This literature, which has examined the association mostly in group-based interventions, has yielded mixed findings. The current study sought to further evaluate the effect of outcome expectancy as a predictor of treatment response for public-speaking fears across both individual virtual reality and group-based cognitive-behavioral therapies. The findings supported outcome expectancy as a predictor of the rate of change in public-speaking anxiety during both individual virtual reality exposure therapy and group cognitive-behavioral therapy. Furthermore, there was no evidence to suggest that the impact of outcome expectancy differed across virtual reality or group treatments.

Impact of CONSORT extension for cluster randomised trials on quality of reporting and study methodology: review of random sample of 300 trials, 2000-8

OBJECTIVE

To assess the impact of the 2004 extension of the CONSORT guidelines on the reporting and methodological quality of cluster randomised trials.

DESIGN

Methodological review of 300 randomly sampled cluster randomised trials. Two reviewers independently abstracted 14 criteria related to quality of reporting and four methodological criteria specific to cluster randomised trials. We compared manuscripts published before CONSORT (2000-4) with those published after CONSORT (2005-8). We also investigated differences by journal impact factor, type of journal, and trial setting.

DATA SOURCES

A validated Medline search strategy. Eligibility criteria for selecting studies Cluster randomised trials published in English language journals, 2000-8.

RESULTS

There were significant improvements in five of 14 reporting criteria: identification as cluster randomised; justification for cluster randomisation; reporting whether outcome assessments were blind; reporting the number of clusters randomised; and reporting the number of clusters lost to follow-up. No significant improvements were found in adherence to methodological criteria. Trials conducted in clinical rather than non-clinical settings and studies published in medical journals with higher impact factor or general medical journals were more likely to adhere to recommended reporting and methodological criteria overall, but there was no evidence that improvements after publication of the CONSORT extension for cluster trials were more likely in trials conducted in clinical settings nor in trials published in either general medical journals or in higher impact factor journals.

CONCLUSION

The quality of reporting of cluster randomised trials improved in only a few aspects since the publication of the extension of CONSORT for cluster randomised trials, and no improvements at all were observed in essential methodological features. Overall, the adherence to reporting and methodological guidelines for cluster randomised trials remains suboptimal, and further efforts are needed to improve both reporting and methodology.

Efficacy of a pilot Internet-based weight management program (H.E.A.L.T.H.) and longitudinal physical fitness data in Army Reserve soldiers

BACKGROUND

The primary aims of this article are to describe the utilization of an Internet-based weight management Web site [Healthy Eating, Activity, and Lifestyle Training Headquarters (H.E.A.L.T.H.)] over a 12-27 month period and to describe concurrent weight and fitness changes in Army Reserve soldiers.

METHODS

The H.E.A.L.T.H. Web site was marketed to Army Reserve soldiers via a Web site promotion program for 27 months (phase I) and its continued usage was observed over a subsequent 12-month period (phase II). Web site usage was obtained from the H.E.A.L.T.H. Web site. Weight and fitness data were extracted from the Regional Level Application Software (RLAS).

RESULTS

A total of 1499 Army Reserve soldiers registered on the H.E.A.L.T.H. Web site. There were 118 soldiers who returned to the H.E.A.L.T.H. Web site more than once. Registration rate reduced significantly following the removal of the Web site promotion program. During phase I, 778 Army Reserve soldiers had longitudinal weight and fitness data in RLAS. Men exceeding the screening table weight gained less weight compared with men below it (p < .007). Percentage change in body weight was inversely associated with change in fitness scores.

CONCLUSIONS

The Web site promotion program resulted in 52% of available Army Reserve soldiers registering onto the H.E.A.L.T.H. Web site, and 7.9% used the Web site more than once. The H.E.A.L.T.H. Web site may be a viable population-based weight and fitness management tool for soldier use.

Cluster randomized trials of cancer screening interventions: are appropriate statistical methods being used?

The design and analysis of cluster randomized trials can require more sophistication than individually randomized trials. However, the need for statistical methods that account for the clustered design has not always been appreciated, and past reviews have found widespread deficiencies in methodology and reporting. We reviewed cluster randomized trials of cancer screening interventions published in 1995-2010 to determine whether the use of appropriate statistical methods had increased over time. Literature searches yielded 50 articles reporting outcome analyses of cluster randomized trials of breast, cervix and colorectal cancer screening interventions. Of studies published in 1995-1999, 2000-2002, 2003-2006 and 2007-2010, 55% (6/11), 82% (9/11), 92% (12/13) and 60% (9/15) used appropriate analytic methods, respectively. Results were suggestive of a peak in 2003-2006 (p =.06) followed by a decline in 2007-2010 (p =.08). While the sample of studies was small, these results indicate that many cluster randomized trials of cancer screening interventions have had deficiencies in the application of correct statistical procedures for the outcome analysis, and that increased adoption of appropriate methods in the early and mid-2000's may not have been sustained.

Comparison of dynamic block randomization and minimization in randomized trials: a simulation study

BACKGROUND

Minimizing the imbalance of key baseline covariates between treatments is known to be very important to the precision of the estimate of treatment effect in clinical research. Dynamic randomization allocation techniques have been used to achieve balance across multiple baseline characteristics. However, empirical data are limited on how these techniques compare in terms of balance and efficiency. We are motivated by a newly funded randomized controlled trial, in which we have the option of choosing between two methods of randomization at the subject level: (1) randomizing individual subjects consecutively as they are enrolled, using Pocock and Simon's minimization method, and (2) simultaneously randomizing blocks of subjects once all subjects in a block have been enrolled, using a balance algorithm originally developed for cluster randomized trials.

PURPOSE

To compare dynamic block randomization and minimization in terms of balance on baseline covariates and statistical efficiency. Simple randomization was included as a reference.

METHODS

A simulation study using data from a previous randomized controlled trial was conducted to compare balance statistics and the accuracy and power of hypothesis testing among the randomization methods.

RESULTS

Dynamic block randomization consistently produced the best balance and highest power for various sample and treatment effect sizes, even after post-adjustment of the pre-specified baseline covariates in all three methods. Consistent with previous reports, minimization performed better in balance and power than simple randomization; however, the differences were noticeably smaller compared to those between dynamic block randomization and simple randomization.

LIMITATIONS

In this simulation study, we considered three sample sizes and two block sizes for a two-arm randomized trial. We assumed no interactions among the multiple baseline covariates. It is necessary to evaluate how the results may vary when the simulation conditions are changed before drawing broader conclusions regarding comparisons between the randomization methods.

CONCLUSIONS

This study demonstrates that dynamic block randomization outperforms minimization with regard to achieving balance and maximizing efficiency. Nevertheless, the differences across the three randomization strategies are modest. The statistical advantages associated with dynamic block randomization need to be considered in relation to the planned sample size and the practical issues for its implementation in deciding the preferred method of randomization for a given trial (e.g., the time required to accrue blocks of subjects of adequate size as balanced against the need to commence intervention/treatment immediately in those randomized to that experimental condition).

Ignoring the group in group-level HIV/AIDS intervention trials: a review of reported design and analytic methods

OBJECTIVES

Studies evaluating the efficacy of HIV/AIDS interventions often involve the random assignment of groups of participants or the treatment of participants in groups. These studies require analytic methods that take within-group correlation into account. We reviewed published studies to determine the extent to which within-group correlation was dealt with properly.

DESIGN

We reviewed group-randomized trials (GRTs) and individually randomized group treatment (IRGT) trials published in HIV/AIDS and general public health journals 2005-2009.

METHODS

At least two of the authors reviewed each article, recording descriptive characteristics, sample size estimation methods, analytic methods, and judgments about whether the methods took intraclass correlation into account properly.

RESULTS

Of those articles including sufficient information to judge whether analytic methods were correct, only 24% used only appropriate methods for dealing with the intraclass correlation. The percentages differed substantially for GRTs (41.7%) and IRGT trials (8.0%). Most of the articles (69.2%) also made no mention of a priori sample size estimation.

CONCLUSION

A majority of the articles in our review reported analyses ignoring the intraclass correlation. This practice may result in underestimated variance, inappropriately small P values, and incorrect conclusions about the effectiveness of interventions. Previous trials that were analyzed incorrectly need to be re-analyzed, and future trials should be designed and analyzed with appropriate methods. Also, journal reviewers and editors need to be aware of the special requirements for design and analysis of GRTs and IRGT trials and judge the quality of articles reporting on such trials according to appropriate standards.

Impact of commercials on food preferences of low-income, minority preschoolers

OBJECTIVE

To determine whether fruit and vegetable (FV) commercials have an impact on preschool children's preferences for specific FV.

DESIGN

A year of extensive formative assessment was conducted to develop 2 30-second commercials: "Judy Fruity" promoted apples and bananas and "Reggie Veggie" promoted broccoli and carrots. The commercials were embedded into a 15-minute TV program. Fruit and vegetable preferences were assessed before and after 4 exposures to each of the commercials.

SETTING

Four Head Start centers in Houston, Texas.

PARTICIPANTS

One hundred eighty-three preschool children (39% African American; 61% Hispanic American).

MAIN OUTCOME MEASURES

Assessment of whether FV preferences were significantly higher in the treatment group than the control group, controlling for baseline FV preferences, age, race, and intervention dose in the model.

ANALYSIS

A general linear model was used.

RESULTS

There was a significantly higher preference for broccoli and carrots (P = .02) in the intervention group compared to the control group after multiple exposures to the vegetable commercial.

CONCLUSIONS AND IMPLICATIONS

Data suggest that commercials promoting vegetables may be an effective strategy to influence young children's preferences for vegetables. This may not be the case with fruit preferences, which are already high in this age group.

Intraclass correlation associated with therapists: estimates and applications in planning psychotherapy research

It is essential that outcome research permit clear conclusions to be drawn about the efficacy of interventions. The common practice of nesting therapists within conditions can pose important methodological challenges that affect interpretation, particularly if the study is not powered to account for the nested design. An obstacle to the optimal design of these studies is the lack of data about the intraclass correlation coefficient (ICC), which measures the statistical dependencies introduced by nesting. To begin the development of a public database of ICC estimates, the authors investigated ICCs for a variety outcomes reported in 20 psychotherapy outcome studies. The magnitude of the 495 ICC estimates varied widely across measures and studies. The authors provide recommendations regarding how to select and aggregate ICC estimates for power calculations and show how researchers can use ICC estimates to choose the number of patients and therapists that will optimize power. Attention to these recommendations will strengthen the validity of inferences drawn from psychotherapy studies that nest therapists within conditions.

Efficacy of a single-session HIV prevention intervention for black women: a group randomized controlled trial

SisterLove Inc., a community-based organization (CBO) in Atlanta, Georgia, evaluated the efficacy of its highly interactive, single-session HIV prevention intervention for black women, the Healthy Love Workshop (HLW). HLW is delivered to pre-existing groups of women (e.g., friends, sororities) in settings of their choosing. Eligible groups of women were randomly assigned to receive the intervention (15 groups; 161 women) or a comparison workshop (15 groups; 152 women). Behavioral assessments were conducted at baseline and at 3- and 6-month follow-ups. Among sexually active women at the 3-month follow-up, HLW participants were more likely than comparison participants to report having used condoms during vaginal sex with any male partner or with a primary male partner, and to have used condoms at last vaginal, anal or oral sex with any male partner. At the 6-month follow-up, HLW participants were more likely to report condom use at last vaginal, anal or oral sex with any male partner, and having an HIV test and receiving their test results. The study findings suggest that a single-session intervention delivered to pre-existing groups of black women is an efficacious approach to HIV prevention. This study also demonstrates that a CBO can develop and deliver a culturally appropriate, effective HIV prevention intervention for the population it serves and, with adequate resources and technical assistance, rigorously evaluate its intervention.

Electronic search strategies to identify reports of cluster randomized trials in MEDLINE: low precision will improve with adherence to reporting standards

Background

Cluster randomized trials (CRTs) present unique methodological and ethical challenges. Researchers conducting systematic reviews of CRTs (e.g., addressing methodological or ethical issues) require efficient electronic search strategies (filters or hedges) to identify trials in electronic databases such as MEDLINE. According to the CONSORT statement extension to CRTs, the clustered design should be clearly identified in titles or abstracts; however, variability in terminology may make electronic identification challenging. Our objectives were to (a) evaluate sensitivity ("recall") and precision of a well-known electronic search strategy ("randomized controlled trial" as publication type) with respect to identifying CRTs, (b) evaluate the feasibility of new search strategies targeted specifically at CRTs, and (c) determine whether CRTs are appropriately identified in titles or abstracts of reports and whether there has been improvement over time.

Methods

We manually examined a wide range of health journals to identify a gold standard set of CRTs. Search strategies were evaluated against the gold standard set, as well as an independent set of CRTs included in previous systematic reviews.

Results

The existing strategy (randomized controlled trial.pt) is sensitive (93.8%) for identifying CRTs, but has relatively low precision (9%, number needed to read 11); the number needed to read can be halved to 5 (precision 18.4%) by combining with cluster design-related terms using the Boolean operator AND; combining with the Boolean operator OR maximizes sensitivity (99.4%) but would require 28.6 citations read to identify one CRT. Only about 50% of CRTs are clearly identified as cluster randomized in titles or abstracts; approximately 25% can be identified based on the reported units of randomization but are not amenable to electronic searching; the remaining 25% cannot be identified except through manual inspection of the full-text article. The proportion of trials clearly identified has increased from 28% between the years 2000-2003, to 60% between 2004-2007 (absolute increase 32%, 95% CI 17 to 47%).

Conclusions

CRTs should include the phrase "cluster randomized trial" in titles or abstracts; this will facilitate more accurate indexing of the publication type by reviewers at the National Library of Medicine, and efficient textword retrieval of the subset employing cluster randomization.

Analysis of group randomized trials with multiple binary endpoints and small number of groups

The group randomized trial (GRT) is a common study design to assess the effect of an intervention program aimed at health promotion or disease prevention. In GRTs, groups rather than individuals are randomized into intervention or control arms. Then, responses are measured on individuals within those groups. A number of analytical problems beset GRT designs. The major problem emerges from the likely positive intraclass correlation among observations of individuals within a group. This paper provides an overview of the analytical method for GRT data and applies this method to a randomized cancer prevention trial, where multiple binary primary endpoints were obtained. We develop an index of extra variability to investigate group-specific effects on response. The purpose of the index is to understand the influence of individual groups on evaluating the intervention effect, especially, when a GRT study involves a small number of groups. The multiple endpoints from the GRT design are analyzed using a generalized linear mixed model and the stepdown Bonferroni method of Holm.

Ethical challenges in cluster randomized controlled trials: experiences from public health interventions in Africa and Asia

Public health interventions usually operate at the level of groups rather than individuals, and cluster randomized controlled trials (RCTs) are one means of evaluating their effectiveness. Using examples from six such trials in Bangladesh, India, Malawi and Nepal, we discuss our experience of the ethical issues that arise in their conduct. We set cluster RCTs in the broader context of public health research, highlighting debates about the need to reconcile individual autonomy with the common good and about the ethics of public health research in low-income settings in general. After a brief introduction to cluster RCTs, we discuss particular challenges we have faced. These include the nature of - and responsibility for - group consent, and the need for consent by individuals within groups to intervention and data collection. We discuss the timing of consent in relation to the implementation of public health strategies, and the problem of securing ethical review and approval in a complex domain. Finally, we consider the debate about benefits to control groups and the standard of care that they should receive, and the issue of post-trial adoption of the intervention under test.

Evidence-based Research in Community Rehabilitation: Design Issues and Strategies

This review highlights a number of methodological issues that arise when a randomised controlled trial (RCT) is conducted on community rehabilitation programmes. These methodological issues are discussed with reference to examples of evidence-based studies conducted with the Hong Kong Society for Rehabilitation. In conducting RCTs of community rehabilitation programmes, we recommend using randomisation, a control or comparison group, at least single-blinding, and objective outcome measures. We also discuss strategies used to control inter-subject differences, the importance of pilot testing, and follow-up assessments. Qualitative evaluation and process evaluation can provide important evidence for enhancing the quality of programmes and examining why and how programmes either work or do not work. In view of the resources available to community rehabilitation settings, we recommend a combination of four strategies in community trials: (a) quantitative evaluation using experimental or quasi-experimental designs, (b) subjective outcome evaluation, (c) qualitative evaluation, and (d) process outcome evaluation.

Estimates of intraclass correlation for variables related to behavioral HIV/STD prevention in a predominantly African American and Hispanic sample of young women

Studies designed to evaluate HIV and STD prevention interventions often involve random assignment of groups such as neighborhoods or communities to study conditions (e.g., to intervention or control). Investigators who design group-randomized trials (GRTs) must take the expected intraclass correlation coefficient (ICC) into account in sample size estimation to have adequate power; however, few published ICC estimates exist for outcome variables related to HIV and STD prevention. The Prevention Options for Women Equal Rights (POWER) study was a GRT designed to evaluate a campaign to increase awareness and use of condoms among young African American and Hispanic women. The authors used precampaign and postcampaign data from the POWER study to estimate ICCs (unadjusted and adjusted for covariates) for a variety of sexual behavior and other variables. To illustrate the impact of ICCs on power, the authors present sample-size calculations and demonstrate how ICCs of differing magnitude will affect estimates of required sample size.

The methodological quality of cluster randomised controlled trials for managing tropical parasitic disease: a review of trials published from 1998 to 2007

The aim of this review was to assess the methodological quality of cluster randomised controlled trials (CRCT) for the management of tropical parasitic disease published between 1998 and 2007. A literature survey was conducted using Medline for CRCTs of interventions aimed at managing any one of the six major tropical parasitic diseases: malaria, leishmaniasis, lymphatic filariasis, onchocerciasis, schistosomiasis and trypanosomiasis (Chagas disease). Information was extracted from the published articles in order that, for each trial, categorical responses could be made to a pre-specified list of 12 questions concerning issues relating to the methodological quality of the trial, including choice of design, generalisability, baseline assessment, blinding, use or non-use of a matched design, and accounting for the intraclass correlation in both design and analysis. The literature survey found 38 CRCTs. Of the 35 CRCTs that reported at least one human outcome, 27 were for interventions in the management of malaria whilst the rest were for managing leishmaniasis (4 trials), lymphatic filariasis (2 trials) and schistosomiasis (2 trials). For every one of the pre-specified questions that concerned an issue associated with methodological quality, the responses were consistent with the practice of trialists in relation to the given issue being generally poor.

Statistical analysis of group-administered intervention data: reanalysis of two randomized trials

Group-administered interventions often create statistical dependencies, which, if ignored, increase the rate of Type I errors. The authors analyzed data from two randomized trials involving group interventions to document the impact of statistical dependency on tests of intervention effects and to provide estimates of statistical dependency. Intraclass correlations ranged from .02 to .12. Adjusting for dependencies increased p values for the tests of intervention effects. The increase in the p values depended on the magnitude of the statistical dependence and available degrees of freedom. Results suggest that the literature may overstate the efficacy of group interventions and imply that it will be important to study why groups create dependencies. The authors discuss how dependencies impact statistical power and how researchers can address this concern.

Individually randomized group treatment trials: a critical appraisal of frequently used design and analytic approaches

OBJECTIVES

We reviewed published individually randomized group treatment (IRGT) trials to assess researchers' awareness of within-group correlation and determine whether appropriate design and analytic methods were used to test for treatment effectiveness.

METHODS

We assessed sample size and analytic methods in IRGT trials published in 6 public health and behavioral health journals between 2002 and 2006.

RESULTS

Our review included 34 articles; in 32 (94.1%) of these articles, inappropriate analytic methods were used. In only 1 article did the researchers claim that expected intraclass correlations (ICCs) were taken into account in sample size estimation; in most articles, sample size was not mentioned or ICCs were ignored in the reported calculations.

CONCLUSIONS

Trials in which individuals are randomly assigned to study conditions and treatments administered in groups may induce within-group correlation, violating the assumption of independence underlying commonly used statistical methods. Methods that take expected ICCs into account should be used in reexamining past studies and planning future studies to ensure that interventions are not judged effective solely on the basis of statistical artifacts. We strongly encourage investigators to report ICCs from IRGT trials and describe study characteristics clearly to aid these efforts.

Trial of computerized screening for adolescent behavioral concerns

OBJECTIVE

Injury risk, depressive symptoms, and substance use are the leading causes of adolescent morbidity and death. The goal of this randomized, controlled trial was to determine whether computerized screening with real-time printing of results for pediatricians increased the identification of these adolescent behavioral concerns.

METHODS

A total of 878 primary care patients 11 to 20 years of age participated in computerized behavioral screening (the Health eTouch system) in waiting rooms of 9 urban clinics. These clinics all served predominantly low-income patients. The clinics were randomly assigned to have pediatricians receive screening results either just before face-to-face encounters with patients (immediate-results condition) or 2 to 3 business days later (delayed-results condition).

RESULTS

Fifty-nine percent of Health eTouch respondents had positive results for >/=1 of the following behavioral concerns: injury risk behaviors, significant depressive symptoms, or substance use. Sixty-eight percent of youths in the immediate-results condition who screened positive were identified as having a problem by their pediatrician. This was significantly higher than the recognition rate of 52% for youths in the delayed-results condition.

CONCLUSION

Immediate provision of an adolescent's self-report of behavioral concerns to a pediatrician increased recognition of those problems, compared with the delayed provision of results.

Internal and external validity of cluster randomised trials: systematic review of recent trials

OBJECTIVES

To assess aspects of the internal validity of recently published cluster randomised trials and explore the reporting of information useful in assessing the external validity of these trials.

DESIGN

Review of 34 cluster randomised trials in primary care published in 2004 and 2005 in seven journals (British Medical Journal, British Journal of General Practice, Family Practice, Preventive Medicine, Annals of Internal Medicine, Journal of General Internal Medicine, Pediatrics).

DATA SOURCES

National Library of Medicine (Medline) via PubMed.

DATA EXTRACTION

To assess aspects of internal validity we extracted data on appropriateness of sample size calculations and analyses, methods of identifying and recruiting individual participants, and blinding. To explore reporting of information useful in assessing external validity we extracted data on cluster eligibility, cluster inclusion and retention, cluster generalisability, and the feasibility and acceptability of the intervention to health providers in clusters.

RESULTS

21 (62%) trials accounted for clustering in sample size calculations and 30 (88%) in the analysis; about a quarter were potentially biased because of procedures surrounding recruitment and identification of patients; individual participants were blind to allocation status in 19 (56%) and outcome assessors were blind in 15 (44%). In almost half the reports, information relating to generalisability of clusters was poorly reported, and in two fifths there was no information about the feasibility and acceptability of the intervention.

CONCLUSIONS

Cluster randomised trials are essential for evaluating certain types of interventions. Issues affecting their internal validity, such as appropriate sample size calculations and analysis, have been widely disseminated and are now better addressed by researchers. Blinding of those identifying and recruiting patients to allocation status is recommended but is not always carried out. There may be fewer barriers to internal validity in trials in which individual participants are not recruited. External validity seems poorly addressed in many trials, yet is arguably as important as internal validity in judging quality as a basis for healthcare intervention.

Design and analysis of group-randomized trials in cancer: a review of current practices

BACKGROUND

Previous reviews have identified problems in the design and analysis of group-randomized trials in a number of areas. Similar problems may exist in cancer research, but there have been no comprehensive reviews.

METHODS

We searched Medline and PubMed for group-randomized trials focused on cancer prevention and control that were published between 2002 and 2006. We located and reviewed 75 articles to determine whether articles included evidence of taking group randomization into account in establishing the size of the trial, such as reporting the expected intraclass correlation, the group component of variance, or the variance inflation factor. We also examined the analytical approaches to determine their appropriateness.

RESULTS

Only 18 (24%) of the 75 articles documented appropriate methods for sample size calculations. Only 34 (45%) limited their reports to analyses judged to be appropriate. Fully 26 (34%) failed to report any analyses that were judged to be appropriate. The most commonly used inappropriate analysis was an analysis at the individual level that ignored the groups altogether. Nine articles (12%) did not provide sufficient information.

CONCLUSIONS

Many investigators who use group-randomized trials in cancer research do not adequately attend to the special design and analytic challenges associated with these trials. Failure to do so can lead to reporting type I errors as real effects, mislead investigators and policy-makers, and slow progress toward control and prevention of cancer. A collaborative effort by investigators, statisticians, and others will be required to ensure that group-randomized trials are planned and analyzed using appropriate methods so that the scientific community can have confidence in the published results.

Cluster Randomized Trials

BACKGROUND

Cluster randomized trials (CRTs) offer unique advantages over standard randomized controlled clinical trials (RCTs) and observational methodologies, and may provide a cost-efficient alternative for answering questions about the best treatments for common conditions.

OBJECTIVES

To describe health plan leaders' views on CRTs, identify barriers to conducting CRTs, and solicit recommendations for increasing the acceptability of CRTs.

RESEARCH DESIGN

Qualitative in-depth telephone interviews with leaders from 8 health plans.

SUBJECTS

: Thirty-four health plan leaders (medical directors, pharmacy directors, Institutional Review Board leaders, ethics leaders, compliance leaders, and others).

MEASURES

Qualitative analysis of interview transcripts to identify barriers, factors influencing leaders' views, ethical issues, aspects of CRTs that appeal to leaders, and recommendations for increasing acceptability of CRTs.

RESULTS

Multiple barriers were identified, including financial costs, concerns about stakeholders' perceptions of CRTs, impact on physicians' prescribing habits, and formulary changes. Most leaders recognized the potential value of studying the comparative effectiveness of therapeutics, and many stressed the need for head-to-head trials. Leaders' views would be influenced by variations in study design and implementation. Recommendations for increasing acceptability of CRTs included ensuring that the fiscal impact of a CRT be budget neutral, and that researchers educate stakeholders and decision-makers about CRTs.

CONCLUSIONS

Overall, health plan leaders recognized the need for studies of the comparative effectiveness of therapeutics under real world conditions, and many expressed support for CRTs. However, researchers seeking to conduct CRTs in health plans are likely to face numerous barriers, and preparatory work will be essential.

Advances and controversies in the design of obesity prevention trials

Because randomized controlled trial designs are used more frequently to evaluate obesity prevention programs, nuances in the methodology used for this application become apparent. Areas of attention are the selection of outcome measures with high validity, attention to the description of the intervention, the use of analyses that match the sampling design, and dealing with loss to follow-up. We recommend increased use of preliminary or evidentiary research designed to develop and test intervention components and hypothesized mediators before fully powered, randomized, obesity prevention trials are attempted. Both randomized and observational designs can be used for the purpose. Attention to design issues will ultimately lead to more successful, cost-effective randomized trials, and more rapid movement toward efficacious and effective obesity prevention programs.

Sizing a trial to alter the trajectory of health behaviours: methods, parameter estimates, and their application

Group-randomized trials often involve repeat observations on the same participants. When there are no more than two observations from each participant, standard mixed-model regression methods for a pretest-posttest design can be used. When there are more than two observations from each participant, random coefficients models may be useful. This paper describes the random coefficients analysis appropriate to data from an extended nested cohort design and presents the methods for power analysis and sample size calculations based on that analysis. We provide estimates for the parameters required for those calculations for a number of adolescent health behaviours. We also show how the estimates can be used to plan a future trial.

Estimates of intraclass correlation coefficient and design effect for surveys and cluster randomized trials on injection use in Pakistan and developing countries

BACKGROUND

To assess injection practices and to test interventions aimed at reducing unsafe injections in developing countries, cluster surveys and cluster randomized trials are needed. The design of cluster-based studies requires estimates of intraclass correlation coefficients that have to be obtained from previous studies. This study presents such estimates.

METHODS

Data were derived from a cross-sectional study of injection use and health seeking in Pakistan that used 34 clusters to select 1150 study subjects aged > or =3 months. We analysed variance to separate its components.

RESULTS

Most of intraclass correlation coefficients were in the range of 0.01-0.05. For proportion of injections received during last 3 months, mean number of injections received and health seeking during the past 3 months the intraclass correlation coefficients were 0.02, 0.04 and 0.02, respectively.

CONCLUSION

These estimates can be useful in designing cluster surveys and cluster randomized trials for injection safety in Pakistan and other developing countries.

Longitudinal and repeated cross-sectional cluster-randomization designs using mixed effects regression for binary outcomes: bias and coverage of frequentist and Bayesian methods

As medical applications for cluster randomization designs become more common, investigators look for guidance on optimal methods for estimating the effect of group-based interventions over time. This study examines two distinct cluster randomization designs: (1) the repeated cross-sectional design in which centres are followed over time but patients change, and (2) the longitudinal design in which individual patients are followed over time within treatment clusters. Simulations of each study design stipulated a multiplicative treatment effect (on the log odds scale), between 5 and 15 clusters in each of two treatment arms, and followed over two time periods. Estimation options included linear mixed effects models using restricted maximum likelihood (REML), generalized estimating equations (GEE), mixed effects logistic regression using both penalized quasi likelihood (PQL) and numerical integration, and Bayesian Monte Carlo analysis. For the repeated cross-sectional designs, most methods performed well in terms of bias and coverage when clusters were numerous (30) and variability across clusters of baseline risk and treatment effect was modest. With few clusters (two groups of five) and higher variability, only the Bayesian methods maintained coverage. In the longitudinal designs, the common methods of REML, GEE, or PQL performed poorly when compared to numerical integration, while Bayesian methods demonstrated less bias and better coverage for estimates of both log odds ratios and risk differences. The performance of common statistical tools for the analysis of cluster randomization designs depends heavily on the precise design, the number of clusters, and the variability of baseline outcomes and treatment effects across centres.

Improving Rescreening In Community Clinics: Does A System Approach Work?

Community clinics provide inadequate breast cancer screening services to low-income, racially- and ethnically-diverse communities. This study develops and evaluates the effectiveness of multifaceted organizational system interventions--operational assessments, tracking systems, reminder calls, tailored education, physician prompts and a tailored counseling call--on mammography rescreening rates within three community clinics. We used the Chronic Care Model and Put Prevention Into Practice framework to redesign breast screening delivery services within the California Cancer Detection Programs: Every Woman Counts (CDP:EWC), community clinic settings. We used a quasi-experimental design with a random selection of 400 patients at pre-intervention. To establish a post-intervention clinic's rescreening rate a new comparable cross-sectional random sample of 347 women was drawn. Measures A chart abstraction instrument was used to establish clinics' rescreening rates. Subjects participants at pre and post-intervention were low-income women 50 years of age and older who had received normal mammography results and had not been diagnosed with breast cancer in the last five years. General linear mixed model analysis revealed significant improvements for the organizational system redesign condition [pre-intervention rescreening rate: 32.1 percent v. post-intervention rescreening rate 50.2 percent, (p < .001)]. For the organizational system redesign plus tailored counseling call condition, there was maintenance in the rescreening rate following the intervention [pre-intervention: 44.4 percent v. post-intervention: 45.1 percent, (p > 0.05)]. Multilevel interventions directed at redesigning community clinics primary care breast cancer screening services, can improve mammography rescreening rates.

Measuring improved targeting of health interventions to the poor in the context of a community-randomised trial in rural India

In spite of growing interest in socioeconomic differentials in health outcomes and access to health services, little has been written about methodologies for assessing the impact of equity-enhancing policies or programs. This paper describes three methodological challenges involved in designing a randomised trial with an equity outcome, and how these were met in a trial of alternative strategies to improving the uptake of benefits of a health insurance scheme among its poorest members. The Vimo SEWA trial is nested within a community-based insurance scheme in rural India. While conducting this trial, three methodological problems were encountered: (i) measuring poverty (or "wealth", or "socioeconomic status") (ii) assessing beneficiaries against an appropriate reference standard population and (iii) settling on an appropriate equity measure as an outcome indicator. These problems are likely to arise in any policy or program assessment that has an equity outcome. In the Vimo SEWA trial, the socioeconomic status of beneficiaries (claimants) is assessed relative to that of all scheme members living in same sub-district by applying a rapid assessment questionnaire--which reduces to an integrated index of socioeconomic status--to both a random sample of members in each sub-district, and to all claimants. The results are used to estimate the full distribution of socioeconomic status of members in each sub-district, with each member given a rank score between 0 and 100. Interpolation is used to estimate the rank scores of claimants relative to the membership base. The primary outcome measure for the trial is the mean socioeconomic rank score of claimants. In developing country settings, using an index of socioeconomic status is simpler than assessing household income or the value of household consumption. It is also relatively straightforward to compare the socioeconomic status of health program beneficiaries with a relevant reference population, although two independent surveys are required. Expressing relative wealth on a scale from zero to 100 is conceptually appealing, and the mean value of this rank score provides an equity-specific outcome measure readily integrated into the usual analytic framework for cluster-randomised trials.

Multilevel analysis of group-randomized trials with binary outcomes

OBJECTIVES

Many dental studies have assessed the effectiveness of community- or group-based interventions such as community water fluoridation. These cluster trials, of which group-randomized trials (GRTs) are one type, have design and analysis considerations not found in studies with randomization of treatments to individuals (randomized controlled trials, RCTs). The purpose of this paper is to review analytic methods used for the analysis of binary outcomes from cluster trials and to illustrate these concepts and analytical methods using a school-based GRT.

METHODS

We examine characteristics of GRTs including intra-class correlation (ICC), their most distinctive feature, and review analytical methods for GRTs including group-level analysis, adjusted chi-square test and multivariable analysis (mixed effect models and generalized estimating equations) for correlated binary data. We consider two- and three-level modeling of data from a cross-sectional cluster design. We apply the concepts reviewed using a GRT designed to determine the effect of incentives on response rates in a school-based dental study. We compare the results of analyses using methods for correlated binary data with those from traditional methods that do not account for ICC.

RESULTS

Application of traditional analytic methods to the dental GRT used as an example for this paper led to a substantial overstatement of the effectiveness of the intervention.

CONCLUSIONS

Ignoring the ICC among members of the same group in the analysis of public health intervention studies can lead to erroneous conclusions where groups are the unit of assignment. Special consideration is needed in the analysis of data from these cluster trials. Randomization of treatments to groups also should receive more consideration in the design of cluster trials in dental public health.

Analysis strategies for a community trial to reduce adolescent ATOD use: a comparison of random coefficient and ANOVA/ANCOVA models

The Community Youth Development Study (CYDS) will evaluate the Communities That Care (CTC) operating system for its effects on alcohol, tobacco, drug use, and other outcomes among adolescents resident in the 24 participating communities. The CYDS employs a combination of both cross-sectional and cohort designs. We use data from an earlier study that included the CYDS communities to estimate power for CYDS intervention effects given several analytic models that might be applied to the multiple baseline and follow-up surveys that define the CYDS cross-sectional design. We compare pre-post mixed-model ANCOVA models against random coefficients models, both in one- and two-stage versions. The two-stage pre-post mixed-model ANCOVA offers the best power for the primary outcomes and will provide adequate power for detection of modest but important intervention effects.

A Comparison of Single Sample and Bootstrap Methods to Assess Mediation in Cluster Randomized Trials

A Monte Carlo study examined the statistical performance of single sample and bootstrap methods that can be used to test and form confidence interval estimates of indirect effects in two cluster randomized experimental designs. The designs were similar in that they featured random assignment of clusters to one of two treatment conditions and included a single intervening variable and outcome, but they differed in whether the mediator was measured at the participant or site level. A bias-corrected bootstrap had the best statistical performance for each design and was closely followed by the empirical-Mtest, either of which is recommended for testing and estimating indirect effects in multilevel designs. In addition, consistent with previous research, the commonly used z test had relatively poor performance.

Statistical Aspects of Design and Analysis of Clinical Trials for the Prevention of Caries

This paper considers the methods used in design and analysis of recent clinical trials of topical fluoride interventions designed to prevent the development of dental caries in children, with particular consideration given to issues related to cluster-randomized trials. Studies which met the inclusion criteria were recent clinical trials of topical fluoride interventions published since 1990, conducted in children under 16 years of age, with caries as the outcome variable. Papers not published in English were translated. Information was extracted from the published trial reports on the units of randomization and analysis. The papers were also studied to assess if reporting allowed the assessment of potential consent bias in cluster-randomized trials and the reproduction of sample size calculations. Fifteen trials published since 1990 were included, of which five were cluster randomized. Only 1 of the 5 accounted for the clustering in the analysis. For the other four trials, it was possible to calculate that values from 0.002 (for DMFS) and 0.08 (for being caries free) for the intracluster correlation coefficient within schools could result in statistically non-significant findings. 3 of the 5 cluster-randomized trials did not report the consenting procedure in enough detail to judge whether consent bias could be present. Only 1 of the total 15 trials reported a sample size calculation. In summary, researchers should be aware of the importance of correctly analyzing cluster-randomized data and thorough reporting of clinical trials according to the CONSORT guidelines.

The Multilevel Structure of Four Adolescent Problems

This paper examines variability in adolescent self-reported behavior at the individual, cohort, and school levels for 8th and 11th graders. We examine four adolescent behaviors: substance use, antisocial behavior, depression, and academic performance. Research staff collected the data as part of the Oregon Healthy Teens survey of a population-based sample of 60,837 adolescents over three years in 92 communities. The results indicate that schools vary over time, but not necessarily systematically, and grade-level cohorts account for important variance within schools. The school and cohort combined, however, accounted for at most 4% of the overall variance. The results have implications for research and practice in schools and communities. For example, selection of communities for interventions based on high levels of adolescent problems may be unproductive if individuals account for at least 96% of the variance. Furthermore, in non-experimental designs, cohort variability, not an intervention, may account for apparent improvement across

Empirically supported treatments or type I errors? Problems with the analysis of data from group-administered treatments

When treatments are administered in groups, clients interact in ways that lead to violations of a key assumption of most statistical analyses-the assumption of independence of observations. The resulting dependencies, when not properly accounted for, can increase Type I errors dramatically. Of the 33 studies of group-administered treatment on the empirically supported treatments list, none appropriately analyzed their data. The current authors provide corrections that can be applied to improper analyses. After the corrections, only 12.4% to 68.2% of tests that were originally reported as significant remained significant, depending on what assumptions were made about how large the dependencies among observations really are. Of the 33 studies, 6-19 studies no longer had any significant results after correction. The authors end by providing recommendations for researchers planning group-administered treatment research.

A comparison of permutation and mixed-model regression methods for the analysis of simulated data in the context of a group-randomized trial

Our first purpose was to determine whether, in the context of a group-randomized trial (GRT) with Gaussian errors, permutation or mixed-model regression methods fare better in the presence of measurable confounding in terms of their Monte Carlo type I error rates and power. Our results indicate that given a proper randomization, the type I error rate is similar for both methods, whether unadjusted or adjusted, even in small studies. However, our results also show that should the investigator face the unfortunate circumstance in which modest confounding exists in the only realization available, the unadjusted analysis risks a type I error; in this regard, there was little to distinguish the two methods. Finally, our results show that power is similar for the two methods and, not surprisingly, better for the adjusted tests. Our second purpose was to examine the relative performance of permutation and mixed-model regression methods in the context of a GRT when the normality assumptions underlying the mixed model are violated. Published studies have examined the impact of violation of this assumption at the member level only. Our findings indicate that both methods perform well when the assumption is violated so long as the ICC is very small and the design is balanced at the group level. However, at ICC>or=0.01, the permutation test carries the nominal type I error rate while the model-based test is conservative and so less powerful. Binomial group- and member-level errors did not otherwise change the relative performance of the two methods with regard to confounding.

CRTs--cluster randomized trials or "courting real troubles": challenges of running a CRT in rural Gujarat, India

This paper addresses the logistical challenges of implementing public health interventions in the setting of cluster randomized trials (CRTs), drawing on the experience of carrying out a CRT within a community-based health insurance (CBHI) scheme in rural India. Our CRT is seeking to improve the equity impact--i.e., reduce the differential in claims submission for hospitalization between poor and less poor--of this CBHI in rural areas. Five main challenges are identified and discussed: 1) assigning control clusters, 2) blinding, 3) implementing interventions simultaneously, 4) minimizing leakage, and 5) piggy-backing on a changing scheme. These challenges are not likely to be unique to low-income settings, although the fifth challenge is particularly likely when working with relatively small and resource-constrained programs. While compromises to methodological best-practice may reduce internal validity, they make the intervention more 'real', and potentially more applicable, to other programs and settings. Further, careful documentation of compromises allows them to be considered in the final analysis.

Information-motivation-behavioral skills (IMB) model: Testing direct and mediated treatment effects on condom use among women in low-income housing

BACKGROUND

The Information-Motivation-Behavioral Skills (IMB) model of HIV preventive behavior (1-4) specifies that treatment effects on behavior occur largely as the result of treatment effects on behavioral skills, which follow from effects on information and motivation.

PURPOSE

The objective was to determine whether the variables specified by the IMB model of HIV preventive behavior (1-4) accounted for the relation between an IMB-based treatment and resulting HIV preventive behavior (condom use).

METHOD

Women (n = 557) living in 18 low-income housing developments in 5 geographically dispersed cities were recruited to participate in an HIV-prevention study. Women (within housing developments) were randomly assigned to receive an IMB-based, HIV risk-avoidance intervention or a comparison intervention. Baseline and posttreatment (16 months after baseline) data were collected on condom use information, motivation (social norms, attitudes, intentions, and perceived risk), enactment of behavioral skills (condom negotiation and procurement), and rates of condom use in the past 2 months.

RESULTS

The IMB intervention led to a 12% to 16% increase in condom use rates over the course, whereas the comparison intervention led to 2% decrease. In addition, the IMB treatment led to greater increases in condom use information, in the intentions and social norms components of motivation and the condom procurement and condom conversations components of behavioral skills. The IMB model provided an acceptable fit to the data (root mean square error of approximation < .05) and accounted for 50% of the variance in posttreatment condom use among the sample. Treatment effects on condom use were almost entirely mediated by the IMB variables; specifically, motivation and enactment of behavioral skills mediated the intervention's impact on condom use.

CONCLUSIONS

These results provide supporting evidence as to how theoretical variables operate to effect change within a theory-based intervention and provide evidence as to the applicability of a prevailing theory of HIV risk behavior among low-income minority women.

CRTs--cluster randomized trials or "courting real troubles": challenges of running a CRT in rural Gujarat, India

This paper addresses the logistical challenges of implementing public health interventions in the setting of cluster randomized trials (CRTs), drawing on the experience of carrying out a CRT within a community-based health insurance (CBHI) scheme in rural India. Our CRT is seeking to improve the equity impact--i.e., reduce the differential in claims submission for hospitalization between poor and less poor--of this CBHI in rural areas. Five main challenges are identified and discussed: 1) assigning control clusters, 2) blinding, 3) implementing interventions simultaneously, 4) minimizing leakage, and 5) piggy-backing on a changing scheme. These challenges are not likely to be unique to low-income settings, although the fifth challenge is particularly likely when working with relatively small and resource-constrained programs. While compromises to methodological best-practice may reduce internal validity, they make the intervention more 'real', and potentially more applicable, to other programs and settings. Further, careful documentation of compromises allows them to be considered in the final analysis.

Increasing the degrees of freedom in future group randomized trials: the df* approach

This article builds on the previous article by Blitstein et al. (2005), which showed how external estimates of intraclass correlation can be used to improve the precision for the analysis of an existing group randomized trial. The authors extend that work to sample size estimation and power analysis for future group-randomized trials. Often this approach will allow a smaller study than would otherwise be possible without sacrificing statistical power. Such studies are needed, for example, as pilot studies to help plan for a full-scale efficacy trial, as replication studies, or in situations in which resource constraints prohibit a larger trial. The authors discuss the circumstances under which this strategy will be most helpful and the risks associated with conducting smaller studies.

Increasing the degrees of freedom in existing group randomized trials: the df* approach

This study describes a method for incorporating external estimates of intraclass correlation to improve the precision for the analysis of an existing group-randomized trial. The authors use a random-effects meta-analytic approach to pool the information across studies, which takes into account any interstudy heterogeneity that may exist. This approach can be used in several different situations to estimate the degrees of freedom available for an adjusted test of the intervention effect in a study where the challenges of group-randomized trials were not fully considered when the study was planned. The authors discuss the limitations of this approach and the circumstances in which it is likely to be helpful.

Expert searching in public health

OBJECTIVE

The article explores the characteristics of public health information needs and the resources available to address those needs that distinguish it as an area of searching requiring particular expertise.

METHODS

Public health searching activities from reference questions and literature search requests at a large, urban health department library were reviewed to identify the challenges in finding relevant public health information.

RESULTS

The terminology of the information request frequently differed from the vocabularies available in the databases. Searches required the use of multiple databases and/or Web resources with diverse interfaces. Issues of the scope and features of the databases relevant to the search questions were considered.

CONCLUSION

Expert searching in public health differs from other types of expert searching in the subject breadth and technical demands of the databases to be searched, the fluidity and lack of standardization of the vocabulary, and the relative scarcity of high-quality investigations at the appropriate level of geographic specificity. Health sciences librarians require a broad exposure to databases, gray literature, and public health terminology to perform as expert searchers in public health.

Observed intra-cluster correlation coefficients in a cluster survey sample of patient encounters in general practice in Australia

Background

Cluster sample study designs are cost effective, however cluster samples violate the simple random sample assumption of independence of observations. Failure to account for the intra-cluster correlation of observations when sampling through clusters may lead to an under-powered study. Researchers therefore need estimates of intra-cluster correlation for a range of outcomes to calculate sample size. We report intra-cluster correlation coefficients observed within a large-scale cross-sectional study of general practice in Australia, where the general practitioner (GP) was the primary sampling unit and the patient encounter was the unit of inference.

Methods

Each year the Bettering the Evaluation and Care of Health (BEACH) study recruits a random sample of approximately 1,000 GPs across Australia. Each GP completes details of 100 consecutive patient encounters. Intra-cluster correlation coefficients were estimated for patient demographics, morbidity managed and treatments received. Intra-cluster correlation coefficients were estimated for descriptive outcomes and for associations between outcomes and predictors and were compared across two independent samples of GPs drawn three years apart.

Results

Between April 1999 and March 2000, a random sample of 1,047 Australian general practitioners recorded details of 104,700 patient encounters. Intra-cluster correlation coefficients for patient demographics ranged from 0.055 for patient sex to 0.451 for language spoken at home. Intra-cluster correlations for morbidity variables ranged from 0.005 for the management of eye problems to 0.059 for management of psychological problems. Intra-cluster correlation for the association between two variables was smaller than the descriptive intra-cluster correlation of each variable. When compared with the April 2002 to March 2003 sample (1,008 GPs) the estimated intra-cluster correlation coefficients were found to be consistent across samples.

Conclusions

The demonstrated precision and reliability of the estimated intra-cluster correlations indicate that these coefficients will be useful for calculating sample sizes in future general practice surveys that use the GP as the primary sampling unit.

Impact of a national rural youth health and safety initiative: results from a randomized controlled trial

OBJECTIVES

We conducted a comprehensive evaluation of a rural youth health and safety initiative implemented in 4000 National FFA (formerly Future Farmers of America) chapters across the United States.

METHODS

Data were collected from high school students and their FFA advisers at 3 time intervals (preintervention, immediate postintervention, and 1 year postintervention) with a 3-group (standard, enhanced, and control), cluster-randomized, controlled trial design.

RESULTS

Matched data from 3081 students and 81 advisers revealed no significant effect of this initiative on agricultural health and safety knowledge, safety attitudes, leadership, self-concept, and self-reported injuries of project participants. Data from 30 public health nurses following the intervention confirmed the program's failure to develop sustainable community partnerships.

CONCLUSIONS

This nationally coordinated initiative was funded with more than $1 million donated by agribusinesses. Program implementation was inconsistent, and desired outcomes were not achieved. Future efforts should better guide effective use of private sector resources aimed at reducing agricultural disease and injury among rural youths.

Evidence-based public health: an evolving concept

Evidence-based public health (EBPH) has been proposed as a practice model that builds upon the success of evidence-based medicine (EBM). EBM has been described as a more scientific and systematic approach to the practice of medicine. It has enhanced medical training and practice in many settings. Both EBM and EBPH systematically use data, information, and scientific principles to enhance clinical care and population health, respectively. In this paper, we review the evolution of EBPH, propose a new definition for EBPH, and discuss developments that may support its further advancement.