1
|
Paskett ED, Kruse-Diehr AJ, Oliveri JM, Vanderpool RC, Gray DM, Pennell ML, Huang B, Young GS, Fickle D, Cromo M, Katz ML, Reiter PL, Rogers M, Gross DA, Fairchild V, Xu W, Carman A, Walunis JM, McAlearney AS, Huerta TR, Rahurkar S, Biederman E, Dignan M. Accelerating Colorectal Cancer Screening and Follow-up through Implementation Science (ACCSIS) in Appalachia: protocol for a group randomized, delayed intervention trial. Transl Behav Med 2023; 13:748-756. [PMID: 37202831 PMCID: PMC10538475 DOI: 10.1093/tbm/ibad017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/20/2023] Open
Abstract
Appalachian regions of Kentucky and Ohio are hotspots for colorectal cancer (CRC) mortality in the USA. Screening reduces CRC incidence and mortality; however, screening uptake is needed, especially in these underserved geographic areas. Implementation science offers strategies to address this challenge. The aim of the current study was to conduct multi-site, transdisciplinary research to evaluate and improve CRC screening processes using implementation science strategies. The study consists of two phases (Planning and Implementation). In the Planning Phase, a multilevel assessment of 12 health centers (HC) (one HC from each of the 12 Appalachian counties) was conducted by interviewing key informants, creating community profiles, identifying HC and community champions, and performing HC data inventories. Two designated pilot HCs chose CRC evidence-based interventions to adapt and implement at each level (i.e., patient, provider, HC, and community) with evaluation relative to two matched control HCs. During the Implementation Phase, study staff will repeat the rollout process in HC and community settings in a randomized, staggered fashion in the remaining eight counties/HCs. Evaluation will include analyses of electronic health record data and provider and county surveys. Rural HCs have been reluctant to participate in research because of concerns about capacity; however, this project should demonstrate that research does not need to be burdensome and can adapt to local needs and HC abilities. If effective, this approach could be disseminated to HC and community partners throughout Appalachia to encourage the uptake of effective interventions to reduce the burden of CRC.
Collapse
Affiliation(s)
- Electra D Paskett
- The Ohio State University Comprehensive Cancer Center, Columbus, OH, USA
- The Ohio State University College of Medicine, Department of Internal Medicine, Columbus, OH, USA
| | - Aaron J Kruse-Diehr
- University of Kentucky College of Medicine, Department of Family and Community Medicine, Lexington, KY, USA
- University of Kentucky Markey Cancer Center, Cancer Prevention and Control Research Program, Lexington, KY, USA
| | - Jill M Oliveri
- The Ohio State University Comprehensive Cancer Center, Columbus, OH, USA
| | - Robin C Vanderpool
- University of Kentucky College of Public Health, Department of Health, Behavior and Society, Lexington, KY, USA
| | - Darrell M Gray
- The Ohio State University Comprehensive Cancer Center, Columbus, OH, USA
- The Ohio State University College of Medicine, Department of Internal Medicine, Columbus, OH, USA
| | - Michael L Pennell
- The Ohio State University College of Public Health, Division of Biostatistics, Columbus, OH, USA
| | - Bin Huang
- University of Kentucky Markey Cancer Center, Division of Biostatistics, Biostatistics and Bioinformatics Shared Resource Facility, Lexington, KY, USA
| | | | - Darla Fickle
- The Ohio State University Comprehensive Cancer Center, Columbus, OH, USA
| | - Mark Cromo
- University of Kentucky College of Medicine, Department of Internal Medicine, Lexington, KY, USA
| | - Mira L Katz
- The Ohio State University Comprehensive Cancer Center, Columbus, OH, USA
- The Ohio State University College of Public Health, Division of Health Behavior and Health Promotion, Columbus, OH, USA
| | - Paul L Reiter
- The Ohio State University Comprehensive Cancer Center, Columbus, OH, USA
- The Ohio State University College of Public Health, Division of Health Behavior and Health Promotion, Columbus, OH, USA
| | - Melinda Rogers
- University of Kentucky Markey Cancer Center, Community Impact Office, Lexington, KY, USA
| | - David A Gross
- Northeast Kentucky Area Health Education Center, Morehead, KY, USA
| | - Vickie Fairchild
- Northeast Kentucky Area Health Education Center, Morehead, KY, USA
| | - Wendy Xu
- The Ohio State University College of Public Health, Division of Health Services Management and Policy, Columbus, OH, USA
| | - Angela Carman
- University of Kentucky Markey Cancer Center, Cancer Prevention and Control Research Program, Lexington, KY, USA
| | - Jean M Walunis
- The Ohio State University Comprehensive Cancer Center, Columbus, OH, USA
| | - Ann Scheck McAlearney
- The Ohio State University Comprehensive Cancer Center, Columbus, OH, USA
- The Ohio State University College of Public Health, Division of Health Services Management and Policy, Columbus, OH, USA
- The Ohio State University College of Medicine, Department of Family and Community Medicine, Columbus, OH, USA
| | - Timothy R Huerta
- The Ohio State University Comprehensive Cancer Center, Columbus, OH, USA
- The Ohio State University College of Public Health, Division of Health Services Management and Policy, Columbus, OH, USA
- The Ohio State University College of Medicine, Department of Family and Community Medicine, Columbus, OH, USA
| | | | - Erika Biederman
- The Ohio State University Comprehensive Cancer Center, Columbus, OH, USA
| | - Mark Dignan
- University of Kentucky College of Medicine, Department of Internal Medicine, Lexington, KY, USA
| |
Collapse
|
2
|
The effect of missing data on design efficiency in repeated cross-sectional multi-period two-arm parallel cluster randomized trials. Behav Res Methods 2021; 53:1731-1745. [PMID: 33528816 PMCID: PMC8367915 DOI: 10.3758/s13428-020-01529-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/17/2020] [Indexed: 11/08/2022]
Abstract
The reduced efficiency of the cluster randomized trial design may be compensated by implementing a multi-period design. The trial then becomes longitudinal, with a risk of intermittently missing observations and dropout. This paper studies the effect of missing data on design efficiency in trials where the periods are the days of the week and clusters are followed for at least one week. The multilevel model with a decaying correlation structure is used to relate outcome to period and treatment condition. The variance of the treatment effect estimator is used to measure efficiency. When there is no data loss, efficiency increases with increasing number of subjects per day and number of weeks. Different weekly measurement schemes are used to evaluate the impact of planned missing data designs: the loss of efficiency due to measuring on fewer days is largest for few subjects per day and few weeks. Dropout is modeled by the Weibull survival function. The loss of efficiency due to dropout increases when more clusters drop out during the course of the trial, especially if the risk of dropout is largest at the beginning of the trial. The largest loss is observed for few subjects per day and a large number of weeks. An example of the effect of waiting room environments in reducing stress in dental care shows how different design options can be compared. An R Shiny app allows researchers to interactively explore various design options and to choose the best design for their trial.
Collapse
|
3
|
Moerbeek M. The cluster randomized crossover trial: The effects of attrition in the AB/BA design and how to account for it in sample size calculations. Clin Trials 2020; 17:420-429. [PMID: 32191129 PMCID: PMC7472836 DOI: 10.1177/1740774520913042] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/18/2022]
Abstract
Background/Aims: This article studies the effect of attrition in the cluster randomized crossover trial. The focus is on the two-treatment two-period AB/BA design where attrition occurs during the washout period. Attrition may occur at either the subject level or the cluster level. In the latter case, clusters drop out entirely and provide no measurements in the second period. Subject attrition can only occur in the cohort design, where each subject receives both treatments. Cluster attrition can also occur in the cross-sectional design, where different subjects are measured in the two time periods. Furthermore, this article explores two different strategies to account for potential levels of attrition: increasing sample size and replacing those subjects who drop out by others. Methods: The statistical model that takes into account the nesting of subjects within clusters, and the nesting of repeated measurements within subjects is presented. The effect of attrition is evaluated on the basis of the efficiency of the treatment effect estimator. Matrix algebra is used to derive the relation between efficiency, the degree of attrition, cluster size and the intraclass correlations: the within-cluster within-period correlation, the within-cluster between-period correlation and (in the case of a cohort design) the within-subject correlation. The methodology is implemented in two Shiny Apps. Results: Attrition in a cluster randomized crossover trial implies a loss of efficiency. Efficiency decreases with an increase of the attrition rate. The loss of efficiency due to attrition of subjects in a cohort design is largest for small number of subjects per cluster-period, but it may be repaired to a large degree by increasing the number of subjects per cluster-period or by replacing those subjects who drop out by others. Attrition of clusters results in a larger loss of efficiency, but this loss does not depend on the number of subjects per cluster-period. Repairing for this loss requires a large increase in the number of subjects per cluster-period. The methodology of this article is illustrated by an example on the effect of lavender scent on dental patients’ anxiety. Conclusion: This article provides the methodology of exploring the effect of attrition in cluster randomized crossover trials, and to repair for attrition. As such, it helps researchers plan their trial in an appropriate way and avoid underpowered trials. To use the methodology, prior estimates of the degree of attrition and intraclass correlation coefficients are needed. It is advocated that researchers clearly report the estimates of these quantities to help facilitate planning future trials.
Collapse
Affiliation(s)
- Mirjam Moerbeek
- Department of Methodology and Statistics, Utrecht University, Utrecht, The Netherlands
| |
Collapse
|
4
|
Brown CH, Curran G, Palinkas LA, Aarons GA, Wells KB, Jones L, Collins LM, Duan N, Mittman BS, Wallace A, Tabak RG, Ducharme L, Chambers DA, Neta G, Wiley T, Landsverk J, Cheung K, Cruden G. An Overview of Research and Evaluation Designs for Dissemination and Implementation. Annu Rev Public Health 2017; 38:1-22. [PMID: 28384085 PMCID: PMC5384265 DOI: 10.1146/annurev-publhealth-031816-044215] [Citation(s) in RCA: 282] [Impact Index Per Article: 40.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Abstract
The wide variety of dissemination and implementation designs now being used to evaluate and improve health systems and outcomes warrants review of the scope, features, and limitations of these designs. This article is one product of a design workgroup that was formed in 2013 by the National Institutes of Health to address dissemination and implementation research, and whose members represented diverse methodologic backgrounds, content focus areas, and health sectors. These experts integrated their collective knowledge on dissemination and implementation designs with searches of published evaluations strategies. This article emphasizes randomized and nonrandomized designs for the traditional translational research continuum or pipeline, which builds on existing efficacy and effectiveness trials to examine how one or more evidence-based clinical/prevention interventions are adopted, scaled up, and sustained in community or service delivery systems. We also mention other designs, including hybrid designs that combine effectiveness and implementation research, quality improvement designs for local knowledge, and designs that use simulation modeling.
Collapse
Affiliation(s)
- C Hendricks Brown
- Department of Psychiatry and Behavioral Sciences, Feinberg School of Medicine, Northwestern University, Chicago, Illinois 60611;
| | - Geoffrey Curran
- Division of Health Services Research, Psychiatric Research Institute, University of Arkansas for Medical Sciences, Little Rock, Arkansas 72205;
| | - Lawrence A Palinkas
- Department of Children, Youth and Families, School of Social Work, University of Southern California, Los Angeles, California 90089;
| | - Gregory A Aarons
- Department of Psychiatry, University of California, San Diego, School of Medicine, La Jolla, California 92093;
| | - Kenneth B Wells
- Center for Health Services and Society, Semel Institute for Neuroscience and Human Behavior, University of California, Los Angeles, California 90024;
| | - Loretta Jones
- Healthy African American Families, Los Angeles, California 90008;
| | - Linda M Collins
- The Methodology Center and Department of Human Development & Family Studies, Pennsylvania State University, University Park, Pennsylvania 16802;
| | - Naihua Duan
- Department of Psychiatry, Columbia University Medical Center, Columbia University, New York, NY 10027;
| | - Brian S Mittman
- VA Center for Implementation Practice and Research Support, Virginia Greater Los Angeles Healthcare System, North Hills, California 91343;
| | - Andrea Wallace
- College of Nursing, The University of Iowa, Iowa City, Iowa 52242;
| | - Rachel G Tabak
- Prevention Research Center, George Warren Brown School, Washington University, St. Louis, Missouri 63105;
| | - Lori Ducharme
- National Institute of Alcohol Abuse and Alcoholism, National Institutes of Health, Bethesda, Maryland 20814;
| | - David A Chambers
- Division of Cancer Control and Population Sciences, National Cancer Institute, Rockville, Maryland 20850; ,
| | - Gila Neta
- Division of Cancer Control and Population Sciences, National Cancer Institute, Rockville, Maryland 20850; ,
| | - Tisha Wiley
- National Institute on Drug Abuse, National Institutes of Health, Bethesda, Maryland 20814;
| | | | - Ken Cheung
- Mailman School of Public Health, Columbia University, New York, NY 10032;
| | - Gracelyn Cruden
- Department of Psychiatry and Behavioral Sciences, Feinberg School of Medicine, Northwestern University, Chicago, Illinois 60611;
- Department of Health Policy and Management, University of North Carolina, Chapel Hill, North Carolina 27514;
| |
Collapse
|
5
|
Evaluation of Community-Level Effects of Communities That Care on Adolescent Drug Use and Delinquency Using a Repeated Cross-Sectional Design. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2016; 17:177-87. [PMID: 26462492 DOI: 10.1007/s11121-015-0613-4] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
Abstract
The Communities That Care (CTC) prevention system has shown effects on reducing incidence and prevalence of problem behaviors among a panel of youth followed from 5th through 12th grade. The present report examines whether similar intervention effects could be observed using a repeated cross-sectional design in the same study. Data were from a community-randomized trial of 24 US towns. Cross-sectional samples of sixth, eighth, and tenth graders were surveyed at four waves. Two-stage ANCOVA analyses estimated differences between CTC and control communities in community-level prevalence of problem behaviors for each grade, adjusting for baseline prevalence. No statistically significant reductions in prevalence of problem behaviors were observed at any grade in CTC compared to control communities. Secondary analyses examined intervention effects within a “pseudo cohort” where cross-sectional data were used from sixth graders at baseline and tenth graders 4 years later. When examining effects within the pseudo cohort, CTC compared to control communities showed a significantly slower increase from sixth to tenth grade in lifetime smokeless tobacco use but not for other outcomes. Exploratory analyses showed significantly slower increases in lifetime problem behaviors within the pseudo cohort for CTC communities with high, but not low, prevention program saturation compared to control communities. Although CTC demonstrated effects in a longitudinal panel from the same community-randomized trial, we did not find similar effects on problem behaviors using a repeated cross-sectional design. These differences may be due to a reduced ability to detect effects because of potential cohort effects, accretion of those who were not exposed, and attrition of those who were exposed to CTC programming in the repeated cross-sectional sample.
Collapse
|
6
|
Van Horn ML, Fagan AA, Hawkins JD, Oesterle S. Effects of the Communities That Care system on cross-sectional profiles of adolescent substance use and delinquency. Am J Prev Med 2014; 47:188-97. [PMID: 24986217 PMCID: PMC4106992 DOI: 10.1016/j.amepre.2014.04.004] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/29/2013] [Revised: 03/13/2014] [Accepted: 04/12/2014] [Indexed: 11/19/2022]
Abstract
INTRODUCTION Adolescent substance use and delinquency are major public health problems. Although community-based prevention strategies have been recommended to produce population-level reductions in rates of substance use and delinquency, few models show evidence of effectiveness. PURPOSE To test the efficacy of a community-based prevention system, Communities That Care (CTC), in reducing community rates of problem behaviors, particularly effects on specific profiles of adolescent substance use and delinquency in eighth- and tenth-graders. METHODS Twenty-four communities were randomized to CTC intervention or control groups. Data were collected from 14,099 8th- and 10th-grade students in these communities using anonymous cross-sectional surveys in 2004 and 2010 and analyzed in 2012. Outcomes were four different profiles of self-reported substance use and delinquency in 8th grade and five profiles in 10th grade. RESULTS In the cross-sectional 2010 data, there was no intervention effect on the probability of experimenting with substances or of substance use coupled with delinquent activities for either grade. However, tenth-graders in intervention communities were significantly less likely to be alcohol users than those in control communities (OR=0.69, CI=0.48, 1.00). CONCLUSIONS Cross-sectional population surveys showed evidence of CTC effects in reducing tenth-grade alcohol users but not experimenters. A community-wide reduction in adolescent alcohol use is important because alcohol is the most commonly used illicit substance during adolescence, and early initiation of alcohol use has been associated with alcohol-related disorders in adulthood. Failure to find hypothesized effects on experimenters qualifies these results.
Collapse
Affiliation(s)
- M Lee Van Horn
- Department of Psychology, University of South Carolina, Columbia, South Carolina.
| | - Abigail A Fagan
- Department of Sociology, Criminology & Law, University of Florida, Gainesville, Florida
| | - J David Hawkins
- Social Development Research Group, University of Washington, Seattle, Washington
| | - Sabrina Oesterle
- Social Development Research Group, University of Washington, Seattle, Washington
| |
Collapse
|
7
|
Aitken ME, Miller BK, Anderson BL, Swearingen CJ, Monroe KW, Daniels D, O Neil J, Tres Scherer LR, Hafner J, Mullins SH. Promoting use of booster seats in rural areas through community sports programs. J Rural Health 2013; 29 Suppl 1:s70-8. [PMID: 23944283 PMCID: PMC3752700 DOI: 10.1111/jrh.12000] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2022]
Abstract
BACKGROUND Booster seats reduce mortality and morbidity for young children in car crashes, but use is low, particularly in rural areas. This study targeted rural communities in 4 states using a community sports-based approach. OBJECTIVE The Strike Out Child Passenger Injury (Strike Out) intervention incorporated education about booster seat use in children ages 4-7 years within instructional baseball programs. We tested the effectiveness of Strike Out in increasing correct restraint use among participating children. METHODS Twenty communities with similar demographics from 4 states participated in a nonrandomized, controlled trial. Surveys of restraint use were conducted before and after baseball season. Intervention communities received tailored education and parents had direct consultation on booster seat use. Control communities received only brochures. RESULTS One thousand fourteen preintervention observation surveys for children ages 4-7 years (Intervention Group [I]: N = 511, Control [C]: N = 503) and 761 postintervention surveys (I: N = 409, C: N = 352) were obtained. For 3 of 4 states, the intervention resulted in increases in recommended child restraint use (Alabama +15.5%, Arkansas +16.1%, Illinois +11.0%). Communities in 1 state (Indiana) did not have a positive response (-9.2%). Overall, unadjusted restraint use increased 10.2% in intervention and 1.7% in control communities (P = .02). After adjustment for each state in the study, booster seat use was increased in intervention communities (Cochran-Mantel-Haenszel odds ratio 1.56, 95% confidence interval [1.16-2.10]). CONCLUSIONS A tailored intervention using baseball programs increased appropriate restraint use among targeted rural children overall and in 3 of 4 states studied. Such interventions hold promise for expansion into other sports and populations.
Collapse
Affiliation(s)
- Mary E Aitken
- Department of Pediatrics, University of Arkansas for Medical Sciences, Little Rock, AR 72202, USA.
| | | | | | | | | | | | | | | | | | | |
Collapse
|
8
|
Vuchinich S, Flay BR, Aber L, Bickman L. Person mobility in the design and analysis of cluster-randomized cohort prevention trials. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2012; 13:300-13. [PMID: 22249907 DOI: 10.1007/s11121-011-0265-y] [Citation(s) in RCA: 21] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/14/2022]
Abstract
Person mobility is an inescapable fact of life for most cluster-randomized (e.g., schools, hospitals, clinic, cities, state) cohort prevention trials. Mobility rates are an important substantive consideration in estimating the effects of an intervention. In cluster-randomized trials, mobility rates are often correlated with ethnicity, poverty and other variables associated with disparity. This raises the possibility that estimated intervention effects may generalize to only the least mobile segments of a population and, thus, create a threat to external validity. Such mobility can also create threats to the internal validity of conclusions from randomized trials. Researchers must decide how to deal with persons who leave study clusters during a trial (dropouts), persons and clusters that do not comply with an assigned intervention, and persons who enter clusters during a trial (late entrants), in addition to the persons who remain for the duration of a trial (stayers). Statistical techniques alone cannot solve the key issues of internal and external validity raised by the phenomenon of person mobility. This commentary presents a systematic, Campbellian-type analysis of person mobility in cluster-randomized cohort prevention trials. It describes four approaches for dealing with dropouts, late entrants and stayers with respect to data collection, analysis and generalizability. The questions at issue are: 1) From whom should data be collected at each wave of data collection? 2) Which cases should be included in the analyses of an intervention effect? and 3) To what populations can trial results be generalized? The conclusions lead to recommendations for the design and analysis of future cluster-randomized cohort prevention trials.
Collapse
Affiliation(s)
- Sam Vuchinich
- School of Social and Behavioral Health Sciences, Oregon State University, 314 Milam Hall, Corvallis, OR 97331, USA.
| | | | | | | |
Collapse
|
9
|
Gloppen KM, Arthur MW, Hawkins JD, Shapiro VB. Sustainability of the Communities That Care prevention system by coalitions participating in the Community Youth Development Study. J Adolesc Health 2012; 51:259-64. [PMID: 22921136 PMCID: PMC3428591 DOI: 10.1016/j.jadohealth.2011.12.018] [Citation(s) in RCA: 40] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/28/2011] [Revised: 12/08/2011] [Accepted: 12/09/2011] [Indexed: 11/26/2022]
Abstract
PURPOSE Community prevention coalitions are a common strategy to mobilize stakeholders to implement tested and effective prevention programs to promote adolescent health and well-being. This article examines the sustainability of Communities That Care (CTC) coalitions approximately 20 months after study support for the intervention ended. METHODS The Community Youth Development Study is a community-randomized trial of the CTC prevention system. Using data from 2007 and 2009 coalition leader interviews, this study reports changes in coalition activities from a period of study support for CTC (2007) to 20 months following the end of study support for CTC (2009), measured by the extent to which coalitions continued to meet specific benchmarks. RESULTS Twenty months after study support for CTC implementation ended, 11 of 12 CTC coalitions in the Community Youth Development Study still existed. The 11 remaining coalitions continued to report significantly higher scores on the benchmarks of phases 2 through 5 of the CTC system than did prevention coalitions in the control communities. At the 20-month follow-up, two-thirds of the CTC coalitions reported having a paid staff person. CONCLUSIONS This study found that the CTC coalitions maintained a relatively high level of implementation fidelity to the CTC system 20 months after the study support for the intervention ended. However, the downward trend in some of the measured benchmarks indicates that continued high-quality training and technical assistance may be important to ensure that CTC coalitions maintain a science-based approach to prevention, and continue to achieve public health impacts on adolescent health and behavior outcomes.
Collapse
Affiliation(s)
- Kari M Gloppen
- Social Development Research Group, School of Social Work, University of Washington, Seattle, USA.
| | | | | | | |
Collapse
|
10
|
Konstantopoulos S. The Impact of Covariates on Statistical Power in Cluster Randomized Designs: Which Level Matters More? MULTIVARIATE BEHAVIORAL RESEARCH 2012; 47:392-420. [PMID: 26814604 DOI: 10.1080/00273171.2012.673898] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
Field experiments with nested structures are becoming increasingly common, especially designs that assign randomly entire clusters such as schools to a treatment and a control group. In such large-scale cluster randomized studies the challenge is to obtain sufficient power of the test of the treatment effect. The objective is to maximize power without adding many clusters that make the study much more expensive. In this article I discuss how power estimates of tests of treatment effects in balanced cluster randomized designs are affected by covariates at different levels. I use third-grade data from Project STAR, a field experiment about class size, to demonstrate how covariates that explain a considerable proportion of variance in outcomes increase power significantly. When lower level covariates are group-mean centered and clustering effects are larger, top-level covariates increase power more than lower level covariates. In contrast, when clustering effects are smaller and lower level covariates are grand-mean centered or uncentered, lower level covariates increase power more than top-level covariates.
Collapse
|
11
|
Pennell ML, Hade EM, Murray DM, Rhoda DA. Cutoff designs for community-based intervention studies. Stat Med 2011; 30:1865-82. [PMID: 21500240 DOI: 10.1002/sim.4237] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2010] [Accepted: 01/24/2011] [Indexed: 11/10/2022]
Abstract
Public health interventions are often designed to target communities defined either geographically (e.g. cities, counties) or socially (e.g. schools or workplaces). The group randomized trial (GRT) is regarded as the gold standard for evaluating these interventions. However, community leaders may object to randomization as some groups may be denied a potentially beneficial intervention. Under a regression discontinuity design (RDD), individuals may be assigned to treatment based on the levels of a pretest measure, thereby allowing those most in need of the treatment to receive it. In this article, we consider analysis, power, and sample size issues in applying the RDD and related cutoff designs in community-based intervention studies. We examine the power of these designs as a function of intraclass correlation, number of groups, and number of members per group and compare results to the traditional GRT.
Collapse
Affiliation(s)
- Michael L Pennell
- Division of Biostatistics, College of Public Health, The Ohio State University, Columbus, OH 43210, U.S.A.
| | | | | | | |
Collapse
|
12
|
Ip EH, Wasserman R, Barkin S. Comparison of intraclass correlation coefficient estimates and standard errors between using cross-sectional and repeated measurement data: the Safety Check cluster randomized trial. Contemp Clin Trials 2010; 32:225-32. [PMID: 21070889 DOI: 10.1016/j.cct.2010.11.001] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2010] [Revised: 10/27/2010] [Accepted: 11/04/2010] [Indexed: 10/18/2022]
Abstract
Designing cluster randomized trials in clinical studies often requires accurate estimates of intraclass correlation, which quantifies the strength of correlation between units, such as participants, within a cluster, such as a practice. Published ICC estimates, even when available, often suffer from the problem of wide confidence intervals. Using data from a national, randomized, controlled study concerning violence prevention for children--the Safety Check--we compare the ICC values derived from two approaches only baseline data and using both baseline and follow-up data. Using a variance component decomposition approach, the latter method allows flexibility in handling complex data sets. For example, it allows for shifts in the outcome variable over time and for an unbalanced cluster design. Furthermore, we evaluate the large-sample formula for ICC estimates and standard errors using the bootstrap method. Our findings suggest that ICC estimates range from 0.012 to 0.11 for providers within practice and range from 0.018 to 0.11 for families within provider. The estimates derived from the baseline-only and repeated-measurements approaches agree quite well except in cases in which variation over repeated measurements is large. The reductions in the widths of ICC confidence limits from using repeated measurement over baseline only are, respectively, 62% and 42% at the practice and provider levels. The contribution of this paper therefore includes two elements, which are a methodology for improving the accuracy of ICC, and the reporting of such quantities for pediatric and other researchers who are interested in designing clustered randomized trials similar to the current study.
Collapse
Affiliation(s)
- Edward H Ip
- Department of Biostatistical Sciences and Department of Social Sciences and Health Policy, Division of Public Health Sciences, Wake Forest University School of Medicine, Winston-Salem, NC 27012, USA.
| | | | | |
Collapse
|
13
|
Arthur MW, Hawkins JD, Brown EC, Briney JS, Oesterle S, Abbott RD. Implementation of the Communities That Care Prevention System by Coalitions in the Community Youth Development Study. JOURNAL OF COMMUNITY PSYCHOLOGY 2010; 38:245-258. [PMID: 22199409 PMCID: PMC3244354 DOI: 10.1002/jcop.20362] [Citation(s) in RCA: 24] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/28/2023]
Abstract
While advances in prevention science over the past 2 decades have produced a growing list of tested and effective programs and policies for preventing adolescent delinquency and drug use, widespread dissemination and high-quality implementation of effective programs and policies in communities has not been achieved. The Community Youth Development Study (CYDS) is a randomized, community-level trial of the Communities That Care (CTC) system for promoting science-based prevention in communities. This paper compares 12 community prevention coalitions implementing the CTC system in 12 intervention communities as part of the CYDS to prevention coalitions located in the 12 control communities. As hypothesized, the CYDS coalitions implemented significantly more of the CTC core intervention elements, and also implemented significantly greater numbers of tested, effective prevention programs than the prevention coalitions in the control communities. Implications of the findings for efforts to achieve widespread dissemination of effective prevention programs, policies, and practices are discussed.
Collapse
|
14
|
Brown EC, Graham JW, Hawkins JD, Arthur MW, Baldwin MM, Oesterle S, Briney JS, Catalano RF, Abbott RD. Design and analysis of the Community Youth Development Study longitudinal cohort sample. EVALUATION REVIEW 2009; 33:311-34. [PMID: 19509119 PMCID: PMC2714913 DOI: 10.1177/0193841x09337356] [Citation(s) in RCA: 47] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/11/2023]
Abstract
Communities That Care (CTC) is a prevention system designed to reduce adolescent substance use and delinquency through the selection of effective preventive interventions tailored to a community's specific profile of risk and protection. A community-randomized trial of CTC, the Community Youth Development Study, is currently being conducted in 24 communities across the United States. This article describes the rationale, multilevel analyses, and baseline comparability for the study's longitudinal cohort design. The cohort sample consists of 4,407 fifth- and sixth-grade students recruited in 2004 and 2005 and surveyed annually through ninth grade. Results of mixed-model ANOVAs indicated that students in CTC and control communities exhibited no significant differences (ps > .05) in baseline levels of student outcomes.
Collapse
|
15
|
Konstantopoulos S. Incorporating cost in power analysis for three-level cluster-randomized designs. EVALUATION REVIEW 2009; 33:335-357. [PMID: 19509118 DOI: 10.1177/0193841x09337991] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/27/2023]
Abstract
In experimental designs with nested structures, entire groups (such as schools) are often assigned to treatment conditions. Key aspects of the design in these cluster-randomized experiments involve knowledge of the intraclass correlation structure, the effect size, and the sample sizes necessary to achieve adequate power to detect the treatment effect. However, the units at each level of the hierarchy have a cost associated with them and thus researchers need to decide on sample sizes given a certain budget, when designing their studies. This article provides methods for computing power within an optimal design framework that incorporates costs of units in all three levels for three-level cluster-randomized balanced designs with two levels of nesting at the second and third level. The optimal sample sizes are a function of the variances at each level and the cost of each unit. Overall, larger effect sizes, smaller intraclass correlations at the second and third level, and lower cost of Level 3 and Level 2 units result in higher estimates of power.
Collapse
|
16
|
Testing communities that care: the rationale, design and behavioral baseline equivalence of the community youth development study. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2008; 9:178-90. [PMID: 18516681 DOI: 10.1007/s11121-008-0092-y] [Citation(s) in RCA: 114] [Impact Index Per Article: 7.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2007] [Accepted: 04/11/2008] [Indexed: 10/22/2022]
Abstract
Recent advances in prevention science provide evidence that adolescent health and behavior problems can be prevented by high-quality prevention services. However, many communities continue to use prevention strategies that have not been shown to be effective. Studying processes for promoting the dissemination and high-quality implementation of prevention strategies found to be effective in controlled research trials has become an important focus for prevention science. The Communities That Care prevention operating system provides manuals, tools, training, and technical assistance to activate communities to use advances in prevention science to plan and implement community prevention services to reduce adolescent substance use, delinquency, and related health and behavior problems. This paper describes the rationale, aims, intervention, and design of the Community Youth Development Study, a randomized controlled community trial of the Communities That Care system, and investigates the baseline comparability of the 12 intervention and 12 control communities in the study. Results indicate baseline similarity of the intervention and control communities in levels of adolescent drug use and antisocial behavior prior to the Communities That Care intervention. Strengths and limitations of the study's design are discussed.
Collapse
|
17
|
The impact on tobacco use of branded youth anti-tobacco activities and family communications about tobacco. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2008; 9:73-87. [PMID: 18478333 DOI: 10.1007/s11121-008-0089-6] [Citation(s) in RCA: 20] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2007] [Accepted: 03/28/2008] [Indexed: 10/22/2022]
Abstract
In a randomized controlled trial, we evaluated the effect on tobacco use onset among middle school students of Family Communications (FC) activities designed to mobilize parental influences against tobacco use and Youth Anti-tobacco Activities (YAT) designed to market anti-tobacco norms to adolescents. We conducted a simple, two-condition experimental design in which 40 middle schools, with a prevalence of tobacco use at or above the Oregon median, received, by random assignment, either the intervention or no intervention. State, county, and local prevention coordinators around Oregon served as liaisons to schools. To generate interest, staff made presentations to these groups and distributed marketing packets at several conferences. Dependent variables were indices of smoking prevalence and use of smokeless tobacco (ST) in the prior month. Additionally, we created an intervention manual so that other communities could replicate this study. The findings suggest that efforts to influence parents to discourage their children's tobacco use and efforts to market an anti-tobacco perspective to teens are effective in preventing smoking. The impact of YAT is consistent with experimental and nonexperimental evaluations of media campaigns to influence young people not to smoke.
Collapse
|
18
|
Van Horn ML, Fagan AA, Jaki T, Brown EC, Hawkins JD, Arthur MW, Abbott RD, Catalano RF. Using Multilevel Mixtures to Evaluate Intervention Effects in Group Randomized Trials. MULTIVARIATE BEHAVIORAL RESEARCH 2008; 43:289-326. [PMID: 26765664 DOI: 10.1080/00273170802034893] [Citation(s) in RCA: 13] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
There is evidence to suggest that the effects of behavioral interventions may be limited to specific types of individuals, but methods for evaluating such outcomes have not been fully developed. This study proposes the use of finite mixture models to evaluate whether interventions, and, specifically, group randomized trials, impact participants with certain characteristics or levels of problem behaviors. This study uses latent classes defined by clustering of individuals based on the targeted behaviors and illustrates the model by testing whether a preventive intervention aimed at reducing problem behaviors affects experimental users of illicit substances differently than problematic substance users or those individuals engaged in more serious problem behaviors. An illustrative example is used to demonstrate the identification of latent classes, specification of random effects in a multilevel mixture model, independent validation of latent classes, and the estimation of power for the proposed models to detect intervention effects. This study proposes specific steps for the estimation of multilevel mixture models and their power and suggests that this model can be applied more broadly to understand the effectiveness of interventions.
Collapse
Affiliation(s)
- M Lee Van Horn
- a Department of Psychology , University of South Carolina
| | | | - Thomas Jaki
- c Department of Mathematics and Statistics , Lancaster University
| | - Eric C Brown
- d Social Development Research Group , University of Washington
| | - J David Hawkins
- d Social Development Research Group , University of Washington
| | | | - Robert D Abbott
- d Social Development Research Group , University of Washington
| | | |
Collapse
|
19
|
Konstantopoulos S. Computing Power of Tests of the Variance of Treatment Effects in Designs With Two Levels of Nesting. MULTIVARIATE BEHAVIORAL RESEARCH 2008; 43:327-352. [PMID: 26765665 DOI: 10.1080/00273170802034901] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
Experiments that involve nested structures may assign treatment conditions either to entire groups (such as classrooms or schools) or individuals within groups (such as students). Although typically the interest in field experiments is in determining the significance of the overall treatment effect, it is equally important to examine the inconsistency of the treatment effect in different groups. This study provides methods for computing power of tests for the variability of treatment effects across level-2 and level-3 units in three-level designs, where, for example, students are nested within classrooms and classrooms are nested within schools and random assignment takes place at the first or the second level. The power computations take into account nesting effects at the second (e.g., classroom) and at the third (e.g., school) level as well as sample size effects (e.g., number of level-1 and level-2 units). The methods can also be applied to quasi-experimental studies that examine the significance of the variation of group differences in an outcome or associations between predictors and outcomes across level-2 and level-3 units.
Collapse
|
20
|
Maxwell SE, Kelley K, Rausch JR. Sample Size Planning for Statistical Power and Accuracy in Parameter Estimation. Annu Rev Psychol 2008; 59:537-63. [DOI: 10.1146/annurev.psych.59.103006.093735] [Citation(s) in RCA: 267] [Impact Index Per Article: 16.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Affiliation(s)
- Scott E. Maxwell
- Department of Psychology, University of Notre Dame, Notre Dame, Indiana 46556;
| | - Ken Kelley
- Inquiry Methodology Program, Indiana University, Bloomington, Indiana 47405;
| | - Joseph R. Rausch
- Department of Psychology, University of Minnesota, Minneapolis, Minnesota 55455;
| |
Collapse
|
21
|
Preisser JS, Reboussin BA, Song EY, Wolfson M. The Importance and Role of Intracluster Correlations in Planning Cluster Trials. Epidemiology 2007; 18:552-60. [PMID: 17879427 PMCID: PMC2567827 DOI: 10.1097/ede.0b013e3181200199] [Citation(s) in RCA: 26] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
There is increasing recognition of the critical role of intracluster correlations of health behavior outcomes in cluster intervention trials. This study examines the estimation, reporting, and use of intracluster correlations in planning cluster trials. We use an estimating equations approach to estimate the intracluster correlations corresponding to the multiple-time-point nested cross-sectional design. Sample size formulae incorporating 2 types of intracluster correlations are examined for the purpose of planning future trials. The traditional intracluster correlation is the correlation among individuals within the same community at a specific time point. A second type is the correlation among individuals within the same community at different time points. For a "time x condition" analysis of a pretest-posttest nested cross-sectional trial design, we show that statistical power considerations based upon a posttest-only design generally are not an adequate substitute for sample size calculations that incorporate both types of intracluster correlations. Estimation, reporting, and use of intracluster correlations are illustrated for several dichotomous measures related to underage drinking collected as part of a large nonrandomized trial to enforce underage drinking laws in the United States from 1998 to 2004.
Collapse
Affiliation(s)
- John S Preisser
- Department of Biostatistics, University of North Carolina School of Public Health, Chapel Hill, North Carolina 27599-7420, USA.
| | | | | | | |
Collapse
|
22
|
Murray DM, Blitstein JL, Hannan PJ, Baker WL, Lytle LA. Sizing a trial to alter the trajectory of health behaviours: methods, parameter estimates, and their application. Stat Med 2007; 26:2297-316. [PMID: 17044139 DOI: 10.1002/sim.2714] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
Group-randomized trials often involve repeat observations on the same participants. When there are no more than two observations from each participant, standard mixed-model regression methods for a pretest-posttest design can be used. When there are more than two observations from each participant, random coefficients models may be useful. This paper describes the random coefficients analysis appropriate to data from an extended nested cohort design and presents the methods for power analysis and sample size calculations based on that analysis. We provide estimates for the parameters required for those calculations for a number of adolescent health behaviours. We also show how the estimates can be used to plan a future trial.
Collapse
Affiliation(s)
- David M Murray
- Division of Epidemiology, School of Public Health, The Ohio State University, B222 Starling Loving Hall, 320 West 10th Street, Columbus, OH 43210, USA.
| | | | | | | | | |
Collapse
|
23
|
Murray DM, Stevens J, Hannan PJ, Catellier DJ, Schmitz KH, Dowda M, Conway TL, Rice JC, Yang S. School-level intraclass correlation for physical activity in sixth grade girls. Med Sci Sports Exerc 2006; 38:926-36. [PMID: 16672847 PMCID: PMC2034369 DOI: 10.1249/01.mss.0000218188.57274.91] [Citation(s) in RCA: 32] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022]
Abstract
PURPOSE The Trial for Activity in Adolescent Girls (TAAG) is a group-randomized trial (GRT) to reduce the usual decline in moderate to vigorous physical activity (MVPA) among middle school girls. We report the school-level intraclass correlation (ICC) for MVPA from the TAAG baseline survey of sixth grade girls and describe the relationship between the schedule of data collection and the ICC. METHODS Each of six sites recruited six schools and randomly selected 60 sixth grade girls from each school; 74.2% participated. Girls were grouped in waves defined by the date measurements began and asked to wear an Actigraph accelerometer for 6 d. Occasional missing data were replaced by imputation, and counts above 1500 per 30 s were treated as MVPA, converted into metabolic equivalents (METs), and summed over 6 a.m.-midnight to provide MET-minutes per 18-h day. Mixed-model regression was used to estimate ICC. RESULTS The school-level ICC were higher when estimated from a single wave compared with three waves (e.g., 0.057 vs 0.022) and across weekdays compared with weekend days (e.g., 0.024 vs 0.012). Power in a new trial would be greater with some schedules (e.g., 88% given three waves and 6 d) than with others (e.g., 23% given one wave and Tuesday only). CONCLUSIONS The schedule of data collection can have a dramatic effect on the ICC for MVPA. In turn, this can have a dramatic effect on the standard error for an intervention effect and on power. Investigators will need to consider the expected magnitude of the ICC and the validity of the MVPA estimates associated with their data collection schedule in planning a new study.
Collapse
Affiliation(s)
- David M Murray
- Division of Epidemiology, School of Public Health, The Ohio State University, Columbus, 43210, USA.
| | | | | | | | | | | | | | | | | |
Collapse
|