Power in randomized group comparisons: the value of adding a single intermediate time point to a traditional pretest-posttest design

Effects of intense assessment on statistical power in randomized controlled trials: Simulation study on depression

Smartphone-based devices are increasingly recognized to assess disease symptoms in daily life (e.g. ecological momentary assessment, EMA). Despite this development in digital psychiatry, clinical trials are mainly based on point assessments of psychopathology. This study investigated expectable increases in statistical power by intense assessment in randomized controlled trials (RCTs). A simulation study, based on three scenarios and several empirical data sets, estimated power gains of two- or fivefold pre-post-assessment. For each condition, data sets of various effect sizes were generated, and AN(C)OVAs were applied to the sample of interest (N = 50-N = 200). Power increases ranged from 6% to 92%, with higher gains in more underpowered scenarios and with higher number of repeated assessments. ANCOVA profited from a more precise estimation of the baseline covariate, resulting in additional gains in statistical power. Fivefold pre-post EMA resulted in highest absolute statistical power and clearly outperformed traditional questionnaire assessments. For example, ANCOVA of automatized PHQ-9 questionnaire data resulted in absolute power of 55 (for N = 200 and d = 0.3). Fivefold EMA, however, resulted in power of 88.9. Non-parametric and multi-level analyses resulted in comparable outcomes. Besides providing psychological treatment, digital mental health can help optimizing sensitivity in RCT-based research. Intense assessment appears advisable whenever psychopathology needs to be assessed with high precision at pre- and post-assessment (e.g. small sample sizes, small treatment effects, or when applying optimization problems like machine learning). First empiric studies are promising, but more evidence is needed. Simulations for various effects and a short guide for popular power software are provided for study planning.

Student Integration into STEM Careers and Culture: A Longitudinal Examination of Summer Faculty Mentors and Project Ownership

It is widely recognized that the United States needs to attract and retain more people in science, technology, engineering, and mathematics (STEM) careers. Intensive undergraduate research experiences (UREs) are one of the few strategies shown to improve longitudinal student interest and persistence in STEM-related career pathways; however, less is known about the underlying process linking activities to positive outcomes. The tripartite integration model of social influences (TIMSI) provides a framework for understanding the social influence processes by which students integrate into STEM careers and culture. The current study used a longitudinal design and latent growth curve modeling to examine and predict the development of scientific research career persistence intentions over the course of an intensive summer URE. The latent growth curve analysis showed that student persistence intentions declined and rebounded over the course of the summer. Furthermore, the positive impact of faculty mentor role modeling on growth trajectories was mediated through internalization of science community values. In addition, project ownership was found to buffer students from the typical trend of declining and rebounding persistence intentions. The TIMSI framework illuminates the contextual features and underlying psychological processes that link UREs to student integration into STEM careers and culture.

Using Simulations to Investigate the Longitudinal Stability of Alternative Schemes for Classifying and Identifying Children with Reading Disabilities

The present study employed data simulation techniques to investigate the one-year stability of alternative classification schemes for identifying children with reading disabilities. Classification schemes investigated include low performance, unexpected low performance, dual-discrepancy, and a rudimentary form of constellation model of reading disabilities that included multiple criteria. Data from Spencer et al. (2014) were used to construct a growth model of reading development. The parameters estimated from this model were then used to construct three simulated datasets wherein the growth parameters were manipulated in one of three ways: A stable-growth pattern, a mastery learning pattern and a fan-spread pattern. Results indicated that overall the constellation model provided the most stable classifications across all conditions of the simulation, and that classification schemes were most stable in the fan-spread condition, and were the least stable under the mastery learning growth pattern. These results also demonstrate the utility of data simulations in reading research.

When the test of mediation is more powerful than the test of the total effect

Although previous research has studied power in mediation models, the extent to which the inclusion of a mediator will increase power has not been investigated. To address this deficit, in a first study we compared the analytical power values of the mediated effect and the total effect in a single-mediator model, to identify the situations in which the inclusion of one mediator increased statistical power. The results from this first study indicated that including a mediator increased statistical power in small samples with large coefficients and in large samples with small coefficients, and when coefficients were nonzero and equal across models. Next, we identified conditions under which power was greater for the test of the total mediated effect than for the test of the total effect in the parallel two-mediator model. These results indicated that including two mediators increased power in small samples with large coefficients and in large samples with small coefficients, the same pattern of results that had been found in the first study. Finally, we assessed the analytical power for a sequential (three-path) two-mediator model and compared the power to detect the three-path mediated effect to the power to detect both the test of the total effect and the test of the mediated effect for the single-mediator model. The results indicated that the three-path mediated effect had more power than the mediated effect from the single-mediator model and the test of the total effect. Practical implications of these results for researchers are then discussed.

Evidence base on outpatient behavioral treatments for adolescent substance use: updates and recommendations 2007-2013

This article updates the evidence base on outpatient behavioral treatments for adolescent substance use (ASU) since publication of the previous review completed for this journal by Waldron and Turner ( 2008 ). It first summarizes the Waldron and Turner findings as well as those from more recent literature reviews and meta-analytic studies of ASU treatment. It then presents study design and methods criteria used to select 19 comparative studies subjected to Journal of Clinical Child & Adolescent Psychology level of support evaluation. These 19 studies are grouped by study category (efficacy or effectiveness) and described for sample characteristics, methodological quality, and substance use outcomes. Cumulative level of support designations are then made for each identified treatment approach: ecological family-based treatment, group cognitive-behavioral therapy, and individual cognitive-behavioral therapy are deemed Well Established; behavioral family-based treatment and motivational interviewing are deemed Probably Efficacious; drug counseling is deemed Possibly Efficacious; and four integrated treatment models combining more than one approach are deemed Well Established or Probably Efficacious. The remainder of the article (a) articulates fidelity, mediator, and moderator effects reported for evidence-based approaches since 2008 and (b) recommends four enhancements to the prevailing business model of ASU outpatient services to accelerate penetration of evidence-based approaches into the underserved consumer base: pursue partnerships with influential governmental systems, utilize web-based technology to extend reach and control costs, adapt effective methods for linking services across sectors of care, and promote uptake and sustainability by emphasizing return on investment.

Prediction of gait recovery in spinal cord injured individuals trained with robotic gait orthosis

Background

Motor impairment is a major consequence of spinal cord injury (SCI). Earlier studies have shown that robotic gait orthosis (e.g., Lokomat) can improve an SCI individual’s walking capacity. However, little is known about the differential responses among different individuals with SCI. The present longitudinal study sought to characterize the distinct recovery patterns of gait impairment for SCI subjects receiving Lokomat training, and to identify significant predictors for these patterns.

Methods

Forty SCI subjects with spastic hypertonia at their ankles were randomly allocated to either control or intervention groups. Subjects in the intervention group participated in twelve 1-hour Lokomat trainings over one month, while control subjects received no interventions. Walking capacity was evaluated in terms of walking speed, functional mobility, and endurance four times, i.e. baseline, 1, 2, and 4 weeks after training, using the 10-Meter-Walking, Timed-Up-and-Go, and 6-Minute-Walking tests. Growth Mixture Modeling, an analytical framework for stratifying subjects based on longitudinal changes, was used to classify subjects, based on their gait impairment recovery patterns, and to identify the effects of Lokomat training on these improvements.

Results

Two recovery classes (low and high walking capacity) were identified for each clinical evaluation from both the control and intervention groups. Subjects with initial high walking capacity (i.e. shorter Timed-Up-and-Go time, higher 10-Meter-Walking speed and longer 6-Minute-Walking distance) displayed significant improvements in speed and functional mobility (0.033 m/s/week and–0.41 s/week respectively); however no significant change in endurance was observed. Subjects with low walking capacity exhibited no significant improvement. The membership in these two classes—and thus prediction of the subject’s gait improvement trajectory over time—could be determined by the subject’s maximum voluntary torque at the ankle under both plantar-and dorsi-flexion contractions determined prior to any training.

Conclusion

Our findings demonstrate that subjects responded to Lokomat training non-uniformly, and should potentially be grouped based on their likely recovery patterns using objective criteria. Further, we found that the subject’s ankle torque can predict whether he/she would benefit most from Lokomat training prior to the therapy. These findings are clinically significant as they can help individualize therapeutic programs that maximize patient recovery while minimizing unnecessary efforts and costs.

Individual music therapy for mental health care clients with low therapy motivation: multicentre randomised controlled trial

BACKGROUND

Music therapy (MT) has been shown to be efficacious for mental health care clients with various disorders such as schizophrenia, depression and substance abuse. Referral to MT in clinical practice is often based on other factors than diagnosis. We aimed to examine the effectiveness of resource-oriented MT for mental health care clients with low motivation for other therapies.

METHOD

This was a pragmatic parallel trial. In specialised centres in Norway, Austria and Australia, 144 adults with non-organic mental disorders and low therapy motivation were randomised to 3 months of biweekly individual, resource-oriented MT plus treatment as usual (TAU) or TAU alone. TAU was typically intensive (71% were inpatients) and included the best combination of therapies available for each participant, excluding MT. Blinded assessments of the Scale for the Assessment of Negative Symptoms (SANS) and 15 secondary outcomes were collected before randomisation and after 1, 3 and 9 months. Changes were analysed on an intention-to-treat basis using generalised estimating equations in longitudinal linear models, controlling for diagnosis, site and time point.

RESULTS

MT was superior to TAU for total negative symptoms (SANS, d = 0.54, p < 0.001) as well as functioning, clinical global impressions, social avoidance through music, and vitality (all p < 0.01).

CONCLUSION

Individual MT as conducted in routine practice is an effective addition to usual care for mental health care clients with low motivation.

Optimal Experimental Design With Nesting of Persons in Organizations

This paper introduces optimal design of randomized experiments where individuals are nested within organizations, such as schools, health centers, or companies. The focus is on nested designs with two levels (organization, individual) and two treatment conditions (treated, control), with treatment assignment to organizations, or to individuals within organizations. For each type of assignment, a multilevel model is first presented for the analysis of a quantitative dependent variable or outcome. Simple equations are then given for the optimal sample size per level (number of organizations, number of individuals) as a function of the sampling cost and outcome variance at each level, with realistic examples. Next, it is explained how the equations can be applied if the dependent variable is dichotomous, or if there are covariates in the model, or if the effects of two treatment factors are studied in a factorial nested design, or if the dependent variable is repeatedly measured. Designs with three levels of nesting and the optimal number of repeated measures are briefly discussed, and the paper ends with a short discussion of robust design.

Sample size and power calculations for medical studies by simulation when closed form expressions are not available

This paper shows how Monte Carlo simulation can be used for sample size, power or precision calculations when planning medical research studies. Standard study designs can lead to the use of analysis methods for which power formulae do not exist. This may be because complex modelling techniques with optimal statistical properties are used but power formulae have not yet been derived or because analysis models are employed that divert from the population model due to lack of availability of more appropriate analysis tools. Our presentation concentrates on the conceptual steps involved in carrying out power or precision calculations by simulation. We demonstrate these steps in three examples concerned with (i) drop out in longitudinal studies, (ii) measurement error in observational studies and (iii) causal effect estimation in randomised controlled trials with non-compliance. We conclude that the Monte Carlo simulation approach is an important general tool in the methodological arsenal for assessing power and precision.

Effective Classroom Instruction: Implications of Child Characteristics by Reading Instruction Interactions on First Graders' Word Reading Achievement

Too many children fail to learn how to read proficiently with serious consequences for their overall well-being and long term success in school. This may be because providing effective instruction is more complex than many of the current models of reading instruction portray; there are child characteristic by instruction (CXI) interactions. Here we present efficacy results for a randomized control field trial of the Individualizing Student Instruction (ISI) intervention, which relies on dynamic system forecasting intervention models to recommend amounts of reading instruction for each student, taking into account CXI interactions that consider his or her vocabulary and reading skills. The study, conducted in seven schools with 25 teachers and 396 first graders, revealed that students in the ISI intervention classrooms demonstrated significantly greater reading skill gains by spring than did students in control classrooms. Plus, they were more likely to receive differentiated reading instruction based on CXI interaction guided recommended amounts than were students in control classrooms. The precision with which students received the recommended amounts of each type of literacy instruction, the distance from recommendation, also predicted reading outcomes.

The effect of population heterogeneity on statistical power in the design and evaluation of interventions

Do interventions work and for whom? For this article, we examined the influence of population heterogeneity on power in designing and evaluating interventions. On the basis of Monte Carlo simulations in Study 1, we demonstrated that the power to detect the overall intervention effect is lower for a mixture of two subpopulations than for a homogeneous population with the same average effect size. We also examined how obtaining covariate information for the outcome and risk status (i.e., latent subclass membership) affects power. For Study 2, we moved from the simulation to assess power in the design and evaluation of an intervention for antisocial children. We illustrated the importance of considering population heterogeneity because interventions have different impacts on different ‘‘classes’’ of participants. We also illustrated that power increases by including covariates that are correlated with risk status. The results provide guidelines for the design and evaluation of interventions.

A Small-Scale Randomized Efficacy Trial of Carescapes: Enhancing Children's Social Development in Child Care Homes

The quality of the child care environment and caregiver practices can potentially have significant, lasting impact on children's social development. This study involves the development and a small-scale efficacy trial of the Carescapes program, a video-based training program that focuses on promoting positive social development in young children attending family child care. Fifty-seven caregivers who provided child care in their homes were randomly assigned to immediate intervention or waitlist control groups. Random coefficients analyses showed significant increased use of effective behavior management practices and decreased overall children's problem behavior for the intervention group. A mediation model demonstrated that increases in effective behavior management practices were associated with decreases in problem behavior. A medium intervention effect was found for caregiver's monitoring and a small effect for use of positive attention. These effects declined 5 months following the intervention. Implications for future efficacy and effectiveness studies in family child care settings that involve strategies to facilitate maintenance are discussed.

The impact on tobacco use of branded youth anti-tobacco activities and family communications about tobacco

In a randomized controlled trial, we evaluated the effect on tobacco use onset among middle school students of Family Communications (FC) activities designed to mobilize parental influences against tobacco use and Youth Anti-tobacco Activities (YAT) designed to market anti-tobacco norms to adolescents. We conducted a simple, two-condition experimental design in which 40 middle schools, with a prevalence of tobacco use at or above the Oregon median, received, by random assignment, either the intervention or no intervention. State, county, and local prevention coordinators around Oregon served as liaisons to schools. To generate interest, staff made presentations to these groups and distributed marketing packets at several conferences. Dependent variables were indices of smoking prevalence and use of smokeless tobacco (ST) in the prior month. Additionally, we created an intervention manual so that other communities could replicate this study. The findings suggest that efforts to influence parents to discourage their children's tobacco use and efforts to market an anti-tobacco perspective to teens are effective in preventing smoking. The impact of YAT is consistent with experimental and nonexperimental evaluations of media campaigns to influence young people not to smoke.

Resource-oriented music therapy for psychiatric patients with low therapy motivation: protocol for a randomised controlled trial [NCT00137189]

BACKGROUND

Previous research has shown positive effects of music therapy for people with schizophrenia and other mental disorders. In clinical practice, music therapy is often offered to psychiatric patients with low therapy motivation, but little research exists about this population. The aim of this study is to examine whether resource-oriented music therapy helps psychiatric patients with low therapy motivation to improve negative symptoms and other health-related outcomes. An additional aim of the study is to examine the mechanisms of change through music therapy.

METHODS

144 adults with a non-organic mental disorder (ICD-10: F1 to F6) who have low therapy motivation and a willingness to work with music will be randomly assigned to an experimental or a control condition. All participants will receive standard care, and the experimental group will in addition be offered biweekly sessions of music therapy over a period of three months. Outcomes will be measured by a blind assessor before and 1, 3, and 9 months after randomisation.

DISCUSSION

The findings to be expected from this study will fill an important gap in the knowledge of treatment effects for a patient group that does not easily benefit from treatment. The study's close link to clinical practice, as well as its size and comprehensiveness, will make its results well generalisable to clinical practice.