Identification and evaluation of risk of generalizability biases in pilot versus efficacy/effectiveness trials: a systematic review and meta-analysis

Effects of a 10-Week Integrated Curriculum Intervention on Physical Activity, Resting Blood Pressure, Motor Skills, and Well-Being in 6- to 7-Year-Olds

BACKGROUND

Integrated curriculum interventions have been suggested as an effective means to increase physical activity (PA) and health. The feasibility of such approaches in children living in deprivation is unknown. This study sought to pilot an integrated curriculum pedometer intervention in children living in deprivation on school-based PA, body fatness, resting blood pressure, motor skills, and well-being.

METHODS

Using a pilot cluster randomized intervention design, children (6-7 y old, n = 64) from 2 schools in central England undertook: (1) 10-week integrated curriculum intervention or (2) control (regular school-based activity). School-based PA, body fatness, resting blood pressure, motor skills, and well-being were assessed preintervention and postintervention.

RESULTS

For the intervention group, PA was higher on school days when children had physical education lessons or there were physically active integrated curriculum activities. Body fatness significantly decreased, and well-being and perceived physical competence increased, pre-post for the intervention group compared with the control group. Accelerometer-derived PA, motor skills, and resting blood pressure were not significantly different pre-post for intervention or control groups.

CONCLUSIONS

A 10-week integrated curriculum PA intervention is feasible to conduct and can positively impact aspects of health in 6- to 7-year-old children in England.

The effect of a home-based, gamified stability skills intervention on 4-5-year-old children's physical and cognitive outcomes: A pilot study

BACKGROUND

Stability skills (e.g., static/dynamic balance) are a precursor for other movement skill development (e.g., jumping, catching). However, young children consistently demonstrate low stability and movement skill ability. There is therefore a need to develop effective strategies to improve stability skills in early childhood.

AIM

To pilot the effect of a home-based gamified stability skills intervention on 4-5-year-old children's physical skills, self-perceptions and cognitions.

METHODS

One-hundred-and-eleven 4-5-year-old children participated from three schools. Two schools were allocated into the intervention group (n = 66 children, 33 boys) and one to the control group (n = 45 children, 25 boys). Stability, fundamental movement skills, perceived motor competence, and cognition were assessed at baseline and at post-intervention. The intervention group was given a booklet detailing the 12-week gamified stability skill intervention. The control group participated in their usual weekly activities.

RESULTS

A series of ANCOVAs controlling for baseline values demonstrated significantly higher stability skills (F(1,93) = 24.79, p < 0.001, partial η2 = 0.212), fundamental movement skills (F(1,94) = 15.5, p = < 0.001, partial η2 = 0.139), perceived motor competence (F(1,96) = 5.48, p = 0.021, partial η2 = 0.054) and cognition (F(1,96) = 15.5, p = < 0.001, partial η2 = 0.139) at post-test for the intervention versus control groups.

DISCUSSION

This study demonstrates that a home-based, gamified, stability skills intervention enhances stability skills, fundamental movement skills, perceived motor competence and cognition in children aged 4-5-years old.

Teamwork and its impact on chronic disease clinical outcomes in primary care: a systematic review and meta-analysis

OBJECTIVE

Teamwork positively affects staff performance and patient outcomes in chronic disease management. However, there is limited research on the impact of specific team components on clinical outcomes. This review aims to explore the impact of teamwork components on key clinical outcomes of chronic diseases in primary care.

STUDY DESIGN

Systematic review and meta-analysis.

METHODS

This systematic review and meta-analysis conducted searching EMBASE, PubMed, Cochrane Central Register of Controlled Trials. Studies included must have at least one teamwork component, conducted in primary care for selected chronic diseases, and report an impact of teamwork on clinical outcomes. Mean differences and 95% confidence intervals were used to determine pooled effects of intervention.

RESULTS

A total of 54 studies from 1988 to 2021 were reviewed. Shared decision-making, roles sharing, and leadership were missing in most studies. Team-based intervention showed a reduction in mean systolic blood pressure (MD = 5.88, 95% CI 3.29-8.46, P= <0.001, I2 = 95%), diastolic blood pressure (MD = 3.23, 95% CI 1.53 to 4.92, P = <0.001, I2 = 94%), and HbA1C (MD = 0.38, 95% CI 0.21 to 0.54, P = <0.001, I2 = 58%). More team components led to better SBP and DBP outcomes, while individual team components have no impact on HbA1C. Fewer studies limit analysis of cholesterol levels, hospitalizations, emergency visits and chronic obstructive pulmonary disease-related outcomes.

CONCLUSION

Team-based interventions improve outcomes for chronic diseases, but more research is needed on managing cholesterol, hospitalizations, and chronic obstructive pulmonary disease. Studies with 4-5 team components were more effective in reducing systolic blood pressure and diastolic blood pressure. Heterogeneity should be considered, and additional research is needed to optimize interventions for specific patient populations.

Are the Risk of Generalizability Biases Generalizable? A Meta-Epidemiological Study

Background Preliminary studies (e.g., pilot/feasibility studies) can result in misleading evidence that an intervention is ready to be evaluated in a large-scale trial when it is not. Risk of Generalizability Biases (RGBs, a set of external validity biases) represent study features that influence estimates of effectiveness, often inflating estimates in preliminary studies which are not replicated in larger-scale trials. While RGBs have been empirically established in interventions targeting obesity, the extent to which RGBs generalize to other health areas is unknown. Understanding the relevance of RGBs across health behavior intervention research can inform organized efforts to reduce their prevalence. Purpose The purpose of our study was to examine whether RGBs generalize outside of obesity-related interventions. Methods A systematic review identified health behavior interventions across four behaviors unrelated to obesity that follow a similar intervention development framework of preliminary studies informing larger-scale trials (i.e., tobacco use disorder, alcohol use disorder, interpersonal violence, and behaviors related to increased sexually transmitted infections). To be included, published interventions had to be tested in a preliminary study followed by testing in a larger trial (the two studies thus comprising a study pair). We extracted health-related outcomes and coded the presence/absence of RGBs. We used meta-regression models to estimate the impact of RGBs on the change in standardized mean difference (ΔSMD) between the preliminary study and larger trial. Results We identified sixty-nine study pairs, of which forty-seven were eligible for inclusion in the analysis (k = 156 effects), with RGBs identified for each behavior. For pairs where the RGB was present in the preliminary study but removed in the larger trial the treatment effect decreased by an average of ΔSMD=-0.38 (range - 0.69 to -0.21). This provides evidence of larger drop in effectiveness for studies containing RGBs relative to study pairs with no RGBs present (treatment effect decreased by an average of ΔSMD =-0.24, range - 0.19 to -0.27). Conclusion RGBs may be associated with higher effect estimates across diverse areas of health intervention research. These findings suggest commonalities shared across health behavior intervention fields may facilitate introduction of RGBs within preliminary studies, rather than RGBs being isolated to a single health behavior field.

Effectiveness of the Walking in ScHools (WISH) Study, a peer-led walking intervention for adolescent girls: results of a cluster randomised controlled trial

BACKGROUND

Most adolescent girls fail to meet current physical activity guidelines. Physical activity behaviours track from childhood into adulthood and providing adolescent girls with opportunities to be physically active may have health benefits beyond childhood. The effects of walking interventions on adult cardiometabolic health are known, however less is understood about the potential of walking to promote physical activity in adolescents. Following the Walking In ScHools (WISH) feasibility study, this definitive trial aimed to evaluate the effectiveness of a novel, low-cost, school-based walking intervention at increasing physical activity levels of adolescent girls (aged 12-14 years).

METHODS

Female pupils were recruited from eighteen schools across the border region of Ireland and in Northern Ireland. In intervention schools (n = 9), girls aged 15-18 years, were trained as walk leaders, and led the younger pupils in 10-15 min walks before school, at break and lunch recess. All walks took place in school grounds and pupils were encouraged to participate in as many walks as possible each week. The primary outcome measure was accelerometer determined total physical activity (counts per minutes, cpm).

RESULTS

In total, 589 pupils were recruited to the study. At baseline, pupils engaged in a median (interquartile range (IQR)) 35.7 (21.2) mins moderate-vigorous physical activity (MVPA) per day and only 12% (n = 66) of participants met physical activity guidelines (60 min MVPA per day). The intervention was delivered for a mean (standard deviation (SD)) 19.9 ± 0.97 weeks. The mean post-intervention total physical activity for the intervention group was 676 cpm and 710 cpm in the control group. Post-intervention total physical activity did not statistically differ between groups when adjusted for age, body mass index z-scores and baseline physical activity (mean difference, -33.5, 95% CI = -21.2 to 88.1; p = 0.213).

CONCLUSIONS

'Scaling-up' physical activity interventions is challenging and despite a promising feasibility study, the results of this fully powered trial suggest that in this context, the WISH intervention did not increase device measured physical activity. Since the COVID-19 pandemic, school environments have changed and although pupils enjoyed the programme, attendance at walks was low, indicating that there is a need to better understand how to implement interventions within schools.

TRIAL REGISTRATION

ISRCTN; ISRCTN12847782; Registered 2nd July 2019.

Physical activity interventions in European primary schools: a scoping review to create a framework for the design of tailored interventions in European countries

Introduction

Schools provide a unique environment to facilitate physical activity for children. However, many school-based physical activity interventions have not been effective. We propose a new approach, which allows schools to tailor interventions to their specific context. This scoping review aimed to identify intervention components from previous school-based physical activity interventions to form the basis of a tailored approach in a European setting.

Methods

Joanna Briggs Institute guidelines for conducting scoping reviews were followed. European school-based intervention studies aimed at increasing physical activity in children aged 7-11 years published in English since 2015 were included. Databases searched were Ovid Medline, Embase, PsycINFO, Web of Science Social Sciences Citation Index, ERIC and British Education Index. Data was extracted on intervention components, context-related factors (geographical location, school size, child socioeconomic status and ethnicity), feasibility, acceptability and cost-effectiveness. A data-driven framework was developed to summarize the identified intervention components.

Results

79 articles were included, constituting 45 intervention studies. We identified 177 intervention components, which were synthesized into a framework of 60 intervention component types across 11 activity opportunities: six within the school day, three within the extended school day and two within the wider school environment. Interventions most frequently targeted physical education (21%), active and outdoor learning (16%), active breaks (15%), and school-level environmewnt (12%). Of the intervention components, 41% were delivered by school staff, 31% by the research team, and 24% by external organizations. Only 19% of intervention studies reported geographical location and only 10% reported school size. Participant ethnicity and socioeconomic information was reported by 15% and 25%, respectively. Intervention acceptability was reported in 51% of studies, feasibility in 49%, and cost effectiveness in 2%.

Discussion

This review offers a first step in developing a future framework to help schools to develop context-specific, tailored interventions. However, there was a lack of reporting of contextual factors within the included studies, making it difficult to understand the role of context. Future research should seek to measure and report contextual factors, and to better understand the important aspects of context within school-based physical activity.

Does Osteopathic Heart-Focused Palpation Modify Heart Rate Variability in Stressed Participants with Musculoskeletal Pain? A Randomised Controlled Pilot Study

Heart rate variability (HRV) describes fluctuations in time intervals between heartbeats and reflects autonomic activity. HRV is reduced in stressed patients with musculoskeletal pain and improved after osteopathic manipulative treatment and mind-body interventions. Heart-focused palpation (HFP) combines manual and mind-body approaches to facilitate relaxation. This randomised controlled pilot study investigated the feasibility and sample size for a future randomised controlled trial and the effect of a single treatment with HFP or sham HFP (SHAM) on short-term HRV. A total of Thirty-three adults (47.7 ± 13.5 years old) with stress and musculoskeletal pain completed the trial with acceptable rates of recruitment (8.25 subjects per site/month), retention (100%), adherence (100%), and adverse events (0%). HFP (n = 18), but not SHAM (n = 15), significantly increased the root mean square of successive RR interval differences (p = 0.036), standard deviation of the NN intervals (p = 0.009), and ratio of the low-frequency to high-frequency power band (p = 0.026). HFP and SHAM significantly decreased the heart rate (p < 0.001, p = 0.009) but not the stress index and ratio of the Poincaré plot standard deviation along and perpendicular to the line of identity (p > 0.05). A power analysis calculated 72 participants. Taken together, the study was feasible and HFP improved HRV in stressed subjects with musculoskeletal pain, suggesting a parasympathetic effect.

Research Reviews: Advances in methods for evaluating child and adolescent mental health interventions

BACKROUND

The evidence base for interventions for child mental health and neurodevelopment is weak and the current capacity for rigorous evaluation limited. We describe some of the challenges that make this field particularly difficult and expensive for evaluation studies.

METHODS

We describe and review the use of novel study designs and analysis methodology for their potential to improve this situation.

RESULTS

While several novel designs appeared ill-suited to our field, systematic review found others that offered potential but had yet to be widely adopted, some not at all.

CONCLUSIONS

While funding is inevitably a constraint, we argue that improvements in the evidence base of both current and new treatments will only be achieved by the adoption of a number of these new technologies and study designs, the consistent application of rigorous constructive but demanding standards, and the engagement of the public, patients, clinical and research services to build a design, recruitment, and analysis infrastructure.

Re-evaluating the role of pilot trials in informing effect and sample size estimates for full-scale trials: a meta-epidemiological study

BACKGROUND

Some have argued that pilot trials have little value for informing the expected effect size of a subsequent large trial. This study aims to empirically evaluate the roles of pilot trials in informing the effect and sample size estimates of a full-scale trial.

METHODS

We conducted a search in PubMed on 19 February 2022, for all pilot trials published between 2005 and 2018 and their subsequent full-scale trials. We analysed the agreement in results by comparing the direction and magnitude of the effect size in the pilot trial and full-scale trial. Logistic regression was used to explore whether a significant pilot trial and other characteristics were associated with a significant full-scale trial.

RESULTS

A total of 248 pairs of pilot and full-scale trials were analysed. Full-scale trials with a significant pilot trial were 2.72 times more likely to find a significant result for the primary efficacy outcome than those with a non-significant pilot trial (95% CI 1.52 to 4.86, p=0.001). The association remained significant irrespective of changes made to the trial design. In 73% of the pairs, the pilot trial produced a larger point estimate than the subsequent full-scale trial, but 87% of pairs had a 95% CI estimated by the pilot trial that covered the full-scale trial point estimate. Full-scale trials with a sample size estimated using the SD from the pilot trial were less likely to yield a significant result (OR=0.26, 95% CI 0.10 to 0.65, p=0.004).

CONCLUSION

Pilot trials can provide strong signals on intervention efficacy. When determining the sample size for full-scale trials, using the CI bounds from the pilot trials instead of the point estimate may improve power estimation.

Treatment of posttraumatic stress disorder with prolonged exposure for primary care (PE-PC): Effectiveness and patient and therapist factors related to symptom change and retention

Prolonged exposure (PE) is a first-line treatment for posttraumatic stress disorder (PTSD) available in specialty mental health. PE for primary care (PE-PC) is a brief version of PE adapted for primary care mental health integration, composed of four-eight, 30-min sessions. Using retrospective data of PE-PC training cases from 155 Veterans Health Administration (VHA) providers in 99 VHA clinics who participated in a 4- to 6-month PE-PC training and consultation program, we examined patients' PTSD and depression severity across sessions via mixed effects multilevel linear modeling. Additionally, hierarchical logistic regression analysis was conducted to assess predictors of treatment dropout. Among 737 veterans, medium-to-large reductions in PTSD (intent-to-treat, Cohen's d = 0.63; completers, Cohen's d = 0.79) and small-to-medium reductions in depression (intent-to-treat, Cohen's d = 0.40; completers, Cohen's d = 0.51) were observed. The modal number of PE-PC sessions was five (SD = 1.98). Providers previously trained in both PE and cognitive processing therapy (CPT) were more likely than providers who were not trained in either PE or CPT to have veterans complete PE-PC (OR = 1.54). Veterans with military sexual trauma were less likely to complete PE-PC than veterans with combat trauma (OR = 0.42). Asian American and Pacific Islander veterans were more likely than White veterans to complete treatment (OR = 2.93). Older veterans were more likely than younger veterans to complete treatment (OR = 1.11). (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Recruitment and retention challenges and strategies in randomized controlled trials of psychosocial interventions for children with cancer and their parents: a collective case study

In pediatric oncology there are few examples of successful recruitment and retention strategies in psychosocial care research. This study aims to summarize experiences, challenges, and strategies for conducting randomized controlled trials (RCTs) of psychosocial intervention studies among children with cancer and their parent(s). We conducted a collective case study. To identify the cases, Pubmed and two trial registries were searched for ongoing and finished RCTs of psychosocial intervention studies for children with cancer and their parents. Online semi-structured expert interviews discussing recruitment and retention challenges and strategies were performed with principal investigators and research staff members of the identified cases. Nine studies were identified. Investigators and staff from seven studies participated, highlighting challenges and strategies within three major themes: eligibility, enrollment and retention. Regarding eligibility, collaborating constructively with healthcare professionals and involving them before the start of the study were essential. Being flexible, training the research staff, enabling alignment with the participants' situation, and providing consistency in contact between the research staff member and the families were important strategies for optimizing enrollment and retention. All studies followed a stepped process in recruitment.  Conclusion: Although recruitment and retention in some selected studies were successful, there is a paucity of evidence on experienced recruitment and retention challenges in pediatric psychosocial research and best practices on optimizing them. The strategies outlined in this study can help researchers optimize their protocol and trial-implementation, and contribute to better psychosocial care for children with cancer and their parents.  Trial Registration: This study is not a clinical trial. What is Known: • Performing RCTs is challenging, particularly in pediatric psychosocial research when both the child and parent are targeted. Recruitment and retention are common concerns. In pediatric oncology, there are few examples of successful recruitment and retention strategies in psychosocial care research. What is New: • Key strategies to collaborate constructively with healthcare professionals were outlined. Being flexible, training the research staff, alignment with the participant's situations and providing consistency in contact between the research staff member and the families were considered as essential strategies.

Use of guidelines, checklists, frameworks, and recommendations in behavioral intervention preliminary studies and associations with reporting comprehensiveness: a scoping bibliometric review

BACKGROUND

Guidelines, checklists, frameworks, and recommendations (GCFRs) related to preliminary studies serve as essential resources to assist behavioral intervention researchers in reporting findings from preliminary studies, but their impact on preliminary study reporting comprehensiveness is unknown. The purpose of this study was to conduct a scoping bibliometric review of recently published preliminary behavioral-focused intervention studies to (1) examine the prevalence of GCFR usage and (2) determine the associations between GCFR usage and reporting feasibility-related characteristics.

METHODS

A systematic search was conducted for preliminary studies of behavioral-focused interventions published between 2018 and 2020. Studies were limited to the top 25 journals publishing behavioral-focused interventions, text mined to identify usage of GCFRs, and categorized as either not citing GCFRs or citing ≥ 2 GCFRs (Citers). A random sample of non-Citers was text mined to identify studies which cited other preliminary studies that cited GCFRs (Indirect Citers) and those that did not (Never Citers). The presence/absence of feasibility-related characteristics was compared between Citers, Indirect Citers, and Never Citers via univariate logistic regression.

RESULTS

Studies (n = 4143) were identified, and 1316 were text mined to identify GCFR usage (n = 167 Citers). A random sample of 200 studies not citing a GCFR were selected and categorized into Indirect Citers (n = 71) and Never Citers (n = 129). Compared to Never Citers, Citers had higher odds of reporting retention, acceptability, adverse events, compliance, cost, data collection feasibility, and treatment fidelity (ORrange = 2.62-14.15, p < 0.005). Citers also had higher odds of mentioning feasibility in purpose statements, providing progression criteria, framing feasibility as the primary outcome, and mentioning feasibility in conclusions (ORrange = 6.31-17.04, p < 0.005) and lower odds of mentioning efficacy in purpose statements, testing for efficacy, mentioning efficacy in conclusions, and suggesting future testing (ORrange = 0.13-0.54, p < 0.05). Indirect Citers had higher odds of reporting acceptability and treatment fidelity (ORrange = 2.12-2.39, p < 0.05) but lower odds of testing for efficacy (OR = 0.36, p < 0.05) compared to Never Citers.

CONCLUSION

The citation of GCFRs is associated with greater reporting of feasibility-related characteristics in preliminary studies of behavioral-focused interventions. Researchers are encouraged to use and cite literature that provides guidance on design, implementation, analysis, and reporting to improve the comprehensiveness of reporting for preliminary studies.

Pilot Trial Characteristics, Postpilot Design Modifications, and Feasibility of Full-Scale Trials

Importance

Pilot trials often lead to study design changes in subsequent full-scale trials. Yet, it remains unclear whether these modifications improve the feasibility of the larger trial.

Objective

To compare feasibility estimates between pilot and full-scale trials and identify pilot trial characteristics and modifications associated with equivalent or improved feasibility in the full-scale trial.

Design, Setting, and Participants

This cohort study used pilot trials published between January 2005 and December 2018 and their corresponding full-scale trials. PubMed was searched for trials on February 19, 2022.

Exposures

Pilot trial characteristics and postpilot trial design modifications.

Main Outcomes and Measures

The outcome of interest was difference in 3 feasibility parameters: successful screening probability, enrollment rate, and retention probability. These metrics were defined as equivalent or improved if the full-scale trial's estimate was within or exceeding 10% of the pilot trial's estimate.

Results

A total of 249 pairs of pilot and full-scale trials were analyzed, with 43%, 77%, and 82% of full-scale trials having equivalent or improved successful screening probabilities, enrollment rates, and retention probabilities, respectively. When pilot trials used feasibility progression criteria (relative risk [RR], 1.94; 95% CI, 1.02-5.97) and maintained masking for participants (RR, 1.82; 95% CI, 1.04-4.33) or health care practitioners (RR, 1.81; 95% CI, 1.03-3.97) consistent with the full-scale trial, the likelihood of achieving equivalent or improved screening success in full-scale trials increased. Increasing study sites after the pilot was associated with higher likelihood of equivalent or improved enrollment rates (RR, 1.03; 95% CI, 1.01-1.08). Adding extra content to the intervention (RR, 0.82; 95% CI, 0.66-0.98), changing to active control (RR, 0.74; 95% CI, 0.48-0.99), administrating the control treatment more frequently (RR, 0.60; 95% CI, 0.29-0.93), different follow-up lengths (RR, 0.85; 95% CI, 0.73-0.97), and more follow-up visits (RR, 0.86; 95% CI, 0.75-0.98) were associated with lower likelihood of equivalent or improved retention probability in the full-scale trial.

Conclusions and relevance

In this cohort study of pilot and full-scale trial pairs, pilot trial characteristics and postpilot modifications had varying associations with full-scale trial's feasibility. If full-scale trials planned for masking, it was desirable to use it in the pilot. Modifications increasing participant burden might decrease full-scale trial feasibility. Trialists and funders should consider both pilot trial data and proposed design changes when assessing full-scale trials.

Preliminary Efficacy of a Multi-Behavioral Zoom-Based Peer Health Coaching Intervention in Young Adults: A Stepped Wedge Randomized Controlled Trial

The purpose of this study was to test the preliminary efficacy of a Zoom-based peer coaching intervention on health and risk behaviors in young adults. A convenience sample of young adults was recruited from one U.S. university (N = 89; 73.0 percent female). Participants were randomized to one of two coaching session sequences within the framework of a stepped wedge randomized controlled trial. One experimental sequence received a control condition and a single coaching session, and the second sequence received two sessions. The intervention was a 1-hour program delivered by peer health coaches in a one-on-one setting on Zoom. The program consisted of a behavior image screen, a consultation, and goal planning. Behavioral assessments were completed after each condition. Mixed-effects models were employed to test for behavior differences after coaching sessions compared with the control condition (no coaching session) adjusting for baseline scores. Participants reported significantly higher levels of vigorous physical activity (b = 750 metabolic equivalent of task minutes, p < 0.001), a lower frequency of e-cigarette use (b = -2.1 days; p < 0.001), and a lower risk of e-cigarette susceptibility after two sessions (relative risk = 0.04, p = 0.05), and higher odds of using stress reduction techniques after one session (odds ratio = 1.4, p = 0.04). A nonsignificant trend was observed for longer weekday sleep (b = 0.4 h/night, p = 0.11) after two coaching sessions. The Zoom-based peer health coaching intervention may be an efficient way to improve vigorous physical activity, lower e-cigarette use and susceptibility, and facilitate the use of stress reduction techniques in young adults. The results observed from this preliminary study warrants further investigation using powered effectiveness trials.

Pilot trials may improve the quality of full-scale trials: a meta-research study

OBJECTIVES

Evidence on the value of pilot trials for subsequent trial's quality is scarce. This study aims to determine if a pilot trial improves the quality of the full-scale trial.

STUDY DESIGN AND SETTING

We searched PubMed for pilot trials and their subsequent full-scale trials. The meta-analysis of the full-scale trials was used to identify other full-scale trials on the same research topic but without a pilot trial. Markers of trial quality included publication outcomes and Cochrane Risk of Bias (RoB) assessment.

RESULTS

Fifty-eight full-scale trials with a pilot trial and 151 full-scale trials without were identified from 47 meta-analyses. Trials with a pilot trial were published 0.9 years sooner (mean ± standard deviation: 1.7 ± 1.0 vs. 2.6 ± 2.0, P = 0.005) and in peer-reviewed journals with higher impact factors (60.9 ± 75.0 vs. 24.8 ± 50.3, P < 0.001). A pilot trial's presence was associated with lower risk of bias in full-scale trial random sequence generation (OR [95% CI]: 4.05 [1.27-12.91]), allocation concealment (2.89 [1.07-7.83]), and participants/researchers masking (4.31 [1.37-13.50]), but not outcome assessment masking (1.03 [0.49-2.18]), incomplete outcome data (1.27 [0.47-3.42]), and selective reporting (1.23 [0.44-3.46]).

CONCLUSION

Conducting a pilot trial may enhance the quality of the subsequent full-scale trial.

The mysterious case of the disappearing pilot study: a review of publication bias in preliminary behavioral interventions presented at health behavior conferences

BACKGROUND

The number of preliminary studies conducted and published has increased in recent years. However, there are likely many preliminary studies that go unpublished because preliminary studies are typically small and may not be perceived as methodologically rigorous. The extent of publication bias within preliminary studies is unknown but can prove useful to determine whether preliminary studies appearing in peer-reviewed journals are fundamentally different than those that are unpublished. The purpose of this study was to identify characteristics associated with publication in a sample of abstracts of preliminary studies of behavioral interventions presented at conferences.

METHODS

Abstract supplements from two primary outlets for behavioral intervention research (Society of Behavioral Medicine and International Society of Behavioral Nutrition and Physical Activity) were searched to identify all abstracts reporting findings of behavioral interventions from preliminary studies. Study characteristics were extracted from the abstracts including year presented, sample size, design, and statistical significance. To determine if abstracts had a matching peer-reviewed publication, a search of authors' curriculum vitae and research databases was conducted. Iterative logistic regression models were used to estimate odds of abstract publication. Authors with unpublished preliminary studies were surveyed to identify reasons for nonpublication.

RESULTS

Across conferences, a total of 18,961 abstracts were presented. Of these, 791 were preliminary behavioral interventions, of which 49% (388) were published in a peer-reviewed journal. For models with main effects only, preliminary studies with sample sizes greater than n = 24 were more likely to be published (range of odds ratios, 1.82 to 2.01). For models including interactions among study characteristics, no significant associations were found. Authors of unpublished preliminary studies indicated small sample sizes and being underpowered to detect effects as barriers to attempting publication.

CONCLUSIONS

Half of preliminary studies presented at conferences go unpublished, but published preliminary studies appearing in peer-reviewed literature are not systematically different from those that remain unpublished. Without publication, it is difficult to assess the quality of information regarding the early-stage development of interventions. This inaccessibility inhibits our ability to learn from the progression of preliminary studies.

Initial insights into the impact and implementation of Creating Active Schools in Bradford, UK

BACKGROUND

Few whole-school physical activity programmes integrate implementation science frameworks within the design, delivery, and evaluation. As a result, knowledge of the key factors that support implementation at scale is lacking. The Creating Active Schools (CAS) programme was co-designed and is underpinned by the Capability, Opportunity, Motivation and Behaviour (COM-B) model and the Consolidated Framework for Implementation Research (CFIR). The study aims to understand the initial impact and implementation of CAS in Bradford over 9 months using McKay's et al.'s (2019) implementation evaluation roadmap.

METHODS

Focus groups and interviews were conducted with school staff (n = 30, schools = 25), CAS Champions (n = 9), and the CAS strategic lead (n = 1). Qualitative data were analysed both inductively and deductively. The deductive analysis involved coding data into a priori themes based on McKay et al's implementation evaluation roadmap, using a codebook approach to thematic analysis. The inductive analysis included producing initial codes and reviewing themes before finalising.

RESULTS

Identified themes aligned into three categories: (i) key ingredients for successful adoption and implementation of CAS, (ii) CAS implementation: challenges and solutions, and (iv) the perceived effectiveness of CAS at the school level. This included the willingness of schools to adopt and implement whole-school approaches when they are perceived as high quality and aligned with current school values. The programme implementation processes were seen as supportive; schools identified and valued the step-change approach to implementing CAS long-term. Formal and informal communities of practice provided "safe spaces" for cross-school support. Conversely, challenges persisted with gaining broader reach within schools, school staff's self-competence and shifting school culture around physical activity. This resulted in varied uptake between and within schools.

CONCLUSIONS

This study provides novel insights into the implementation of CAS, with outcomes aligning to the adoption, reach, and sustainability. Successful implementation of CAS was underpinned by determinants including acceptability, intervention complexity, school culture and school stakeholders' perceived self-efficacy. The combination of McKay's evaluation roadmap and CFIR establishes a rigorous approach for evaluating activity promotion programmes underpinned by behavioural and implementation science. Resultantly this study offers originality and progression in understanding the implementation and effectiveness of whole-school approaches to physical activity.

Scientists' perception of pilot study quality was influenced by statistical significance and study design

OBJECTIVES

Preliminary studies play a key role in developing large-scale interventions but may be held to higher or lower scientific standards during the peer review process because of their preliminary study status.

STUDY DESIGN AND SETTING

Abstracts from 5 published obesity prevention preliminary studies were systematically modified to generate 16 variations of each abstract. Variations differed by 4 factors: sample size (n = 20 vs. n = 150), statistical significance (P < 0.05 vs. P > 0.05), study design (single group vs. randomized 2 groups), and preliminary study status (presence/absence of pilot language). Using an online survey, behavioral scientists were provided with a randomly selected variation of each of the 5 abstracts and blinded to the existence of other variations. Respondents rated each abstract on aspects of study quality.

RESULTS

Behavioral scientists (n = 271, 79.7% female, median age 34 years) completed 1,355 abstract ratings. Preliminary study status was not associated with perceived study quality. Statistically significant effects were rated as more scientifically significant, rigorous, innovative, clearly written, warranted further testing, and had more meaningful results. Randomized designs were rated as more rigorous, innovative, and meaningful.

CONCLUSION

Findings suggest reviewers place a greater value on statistically significant findings and randomized control design and may overlook other important study characteristics.

How to select interventions for promoting physical activity in schools? Combining preferences of stakeholders and scientists

BACKGROUND

The failure to scale-up and implement physical activity (PA) interventions in real world contexts, which were previously successful under controlled conditions, may be attributed to the different criteria of stakeholders and scientists in the selection process of available interventions. Therefore, the aim of our study was to investigate and compare the criteria applied by local stakeholders and scientists for selecting amongst suitable school-based PA interventions for implementation.

METHODS

We conducted a three-round repeated survey Delphi study with local stakeholders (n = 7; Bremen, Germany) and international scientific PA experts (n = 6). Independently for both panels, two rounds were utilized to develop a list of criteria and the definitions of criteria, followed by a prioritization of the criteria in the third round. For each panel, a narrative analysis was used to rank-order unique criteria, list the number of scorers for the unique criteria and synthesize criteria into overarching categories.

RESULTS

The stakeholders developed a list of 53 unique criteria, synthesized into 11 categories with top-ranked criteria being 'free of costs', 'longevity' and 'integration into everyday school life'. The scientists listed 35 unique criteria, synthesized into 7 categories with the top-ranked criteria being 'efficacy', 'potential for reach' and 'feasibility'. The top ranked unique criteria in the stakeholder panel were distributed over many categories, whereas four out of the top six criteria in the scientist panel were related to 'evidence'.

CONCLUSIONS

Although stakeholders and scientists identified similar criteria, major differences were disclosed in the prioritization of the criteria. We recommend an early collaboration of stakeholders and scientists in the design, implementation, and evaluation of PA interventions.

Development and Implementation of a Pilot Transition Preparation Intervention for Young Adults With Type 1 Diabetes in an Integrated Healthcare Setting

OBJECTIVE

To evaluate the feasibility, acceptability, and preliminary efficacy of a multisystem transition preparation intervention, SHIFT, for young adults (YAs) with type 1 diabetes (T1D).

METHODS

A single-arm, clinic-based pilot was conducted with 25 YAs with T1D (M age = 18.9 ± 1.0 years; 80% female), their parents (n = 25), and their providers (n = 10). Young adults and parents participated in a 6-month intervention designed to enhance transition readiness and independent diabetes management. Providers viewed a video module highlighting their role preparing YAs for transition and received individualized reports of YA's goals and transition readiness. Intervention feasibility (i.e., recruitment, retention, and engagement) and acceptability (e.g., program satisfaction) were assessed. Assessments of transition readiness, diabetes engagement, hemoglobin A1c (HbA1c), and related psychosocial variables were conducted at baseline, post-intervention (6 m), and follow-up (9 m). Paired t-tests examined 0-6 m and 0-9 m changes in study constructs.

RESULTS

SHIFT was feasible, evidenced by recruitment (100% of sample recruited in 4 m), retention (100% at 6 m), and YA session attendance (100%). Program satisfaction was high for YAs, parents, and providers (9.12 ± 1.40, 8.79 ± 1.56, 8.20 ± 1.30, respectively, [out of 10]). Significant improvements (with effect sizes ranging from small to medium) were observed in parent and YA-reported transition readiness at 6 and 9 m (ps<.05) and diabetes engagement at 9 m (ps<.05). Although based on limited data due to COVID-19-related disruptions, a potential reduction in HbA1c was also observed.

CONCLUSION

Findings support the feasibility, acceptability, and preliminary efficacy of SHIFT (although limited by the single arm design and homogeneous sample), and suggest a larger randomized controlled trial is warranted.

Influence of pilot and small trials in meta-analyses of behavioral interventions: a meta-epidemiological study

BACKGROUND

Pilot/feasibility or studies with small sample sizes may be associated with inflated effects. This study explores the vibration of effect sizes (VoE) in meta-analyses when considering different inclusion criteria based upon sample size or pilot/feasibility status.

METHODS

Searches were to identify systematic reviews that conducted meta-analyses of behavioral interventions on topics related to the prevention/treatment of childhood obesity from January 2016 to October 2019. The computed summary effect sizes (ES) were extracted from each meta-analysis. Individual studies included in the meta-analyses were classified into one of the following four categories: self-identified pilot/feasibility studies or based upon sample size but not a pilot/feasibility study (N ≤ 100, N > 100, and N > 370 the upper 75th of sample size). The VoE was defined as the absolute difference (ABS) between the re-estimations of summary ES restricted to study classifications compared to the originally reported summary ES. Concordance (kappa) of statistical significance of summary ES between the four categories of studies was assessed. Fixed and random effects models and meta-regressions were estimated. Three case studies are presented to illustrate the impact of including pilot/feasibility and N ≤ 100 studies on the estimated summary ES.

RESULTS

A total of 1602 effect sizes, representing 145 reported summary ES, were extracted from 48 meta-analyses containing 603 unique studies (avg. 22 studies per meta-analysis, range 2-108) and included 227,217 participants. Pilot/feasibility and N ≤ 100 studies comprised 22% (0-58%) and 21% (0-83%) of studies included in the meta-analyses. Meta-regression indicated the ABS between the re-estimated and original summary ES where summary ES ranged from 0.20 to 0.46 depending on the proportion of studies comprising the original ES were either mostly small (e.g., N ≤ 100) or mostly large (N > 370). Concordance was low when removing both pilot/feasibility and N ≤ 100 studies (kappa = 0.53) and restricting analyses only to the largest studies (N > 370, kappa = 0.35), with 20% and 26% of the originally reported statistically significant ES rendered non-significant. Reanalysis of the three case study meta-analyses resulted in the re-estimated ES rendered either non-significant or half of the originally reported ES.

CONCLUSIONS

When meta-analyses of behavioral interventions include a substantial proportion of both pilot/feasibility and N ≤ 100 studies, summary ES can be affected markedly and should be interpreted with caution.

Evaluation of the peer leadership for physical literacy intervention: A cluster randomized controlled trial

PURPOSE

The purpose of this research was to develop, implement, and test the efficacy of a theory-driven, evidence-informed peer leadership program for elementary school students (Grade 6 and 7; age 11-12 years) and the Grade 3/4 students with whom they were partnered. The primary outcome was teacher ratings of their Grade 6/7 students' transformational leadership behaviors. Secondary outcomes included: Grade 6/7 students' leadership self-efficacy, as well as Grade 3/4 motivation, perceived competence, general self-concept, fundamental movement skills, school-day physical activity, and program adherence, and program evaluation.

METHODS

We conducted a two-arm cluster randomized controlled trial. In 2019, 6 schools comprising 7 teachers, 132 leaders, and 227 grade 3 and 4 students were randomly allocated to the intervention or waitlist control conditions. Intervention teachers took part in a half-day workshop (January 2019), delivered 7 x 40 minute lessons to Grade 6/7 peer leaders (February and March 2019), and these peer leaders subsequently ran a ten-week physical literacy development program for Grade 3/4 students (2x30 minutes sessions per week). Waitlist-control students followed their usual routines. Assessments were conducted at baseline (January 2019) and immediately post-intervention (June 2019).

RESULTS

The intervention had no significant effect on teacher ratings of their students' transformational leadership (b = 0.201, p = .272) after controlling for baseline and gender. There was no significant condition effect for Grade 6/7 student rated transformation leadership (b = 0.077, p = .569) or leadership self-efficacy (b = 3.747, p = .186) while controlling for baseline and gender. There were null findings for all outcomes related to Grade 3 and 4 students.

DISCUSSION

Adaptions to the delivery mechanism were not effective in increasing leadership skills of older students or components of physical literacy in younger Grade 3/4 students. However, teacher self-reported adherence to the intervention delivery was high.

TRIAL REGISTRATION

This trial was registered on December 19th, 2018 with Clinicaltrials.gov (NCT03783767), https://clinicaltrials.gov/ct2/show/NCT03783767.

Investigating the direct and indirect effects of a school-based leadership program for primary school students: Rationale and study protocol for the 'Learning to Lead' cluster randomised controlled trial

BACKGROUND

Leadership is a valuable skill that can be taught in school, and which may have benefits within and beyond the classroom. Learning to Lead (L2L) is a student-led, primary school-based leadership program whereby older 'peer leaders' deliver a fundamental movement skills (FMS) program to younger 'peers' within their own school.

AIM

The aims of the study are to determine the efficacy of a peer-led FMS intervention on: (i) peer leaders' (aged 10 to 12 years) leadership effectiveness (primary outcome), leadership self-efficacy, well-being, and time on-task in the classroom; (ii) peers' (aged 8 to 10 years) physical activity levels, actual and perceived FMS competency, cardiorespiratory fitness, muscular power, and executive functioning; and (iii) teachers' (referred to as 'school champions') work-related stress and well-being.

METHOD

L2L will be evaluated using a two-arm parallel group cluster randomised controlled trial. Twenty schools located within a two-hour drive of the University of Newcastle, Australia will be recruited. We will recruit 80 students (40 peer leaders and 40 peers) from each school (N = 1,600). L2L will be implemented in three phases: Phase 1 -school champions' training via a professional learning workshop; Phase 2 -school champions' delivery of leadership lessons to the peer leaders; and Phase 3 -peer leaders' delivery of the FMS program to their younger peers. The FMS program, consisting of 12 x 30-minute lessons, will be delivered over the course of one school term (10 weeks). Study outcomes will be assessed at baseline (between mid-March to June, Terms 1 and 2), intervention end (mid-August to September, Term 3), and follow-up (November to mid-December, Term 4. This trial was prospectively registered on the Australian New Zealand Clinical Trials Registry (ANZCTR); registration number: ACTRN12621000376842.

Rethinking children's physical activity interventions at school: A new context-specific approach

Physical activity is important for children's health. However, evidence suggests that many children and adults do not meet international physical activity recommendations. Current school-based interventions have had limited effect on physical activity and alternative approaches are needed. Context, which includes school setting, ethos, staff, and sociodemographic factors, is a key and largely ignored contributing factor to school-based physical activity intervention effectiveness, impacting in several interacting ways.

Conceptualization

Current programs focus on tightly-constructed content that ignores the context in which the program will be delivered, thereby limiting effectiveness. We propose a move away from uniform interventions that maximize internal validity toward a flexible approach that enables schools to tailor content to their specific context.

Evaluation designs

Evaluation of context-specific interventions should explicitly consider context. This is challenging in cluster randomized controlled trial designs. Thus, alternative designs such as natural experiment and stepped-wedge designs warrant further consideration.

Primary outcome

A collective focus on average minutes of moderate-to-vigorous intensity physical activity may not always be the most appropriate choice. A wider range of outcomes may improve children's physical activity and health in the long-term. In this paper, we argue that greater consideration of school context is key in the design and analysis of school-based physical activity interventions and may help overcome existing limitations in the design of effective interventions and thus progress the field. While this focus on context-specific interventions and evaluation is untested, we hope to stimulate debate of the key issues to improve future physical activity intervention development and implementation.

Scale-up of the Internet-based Professional Learning to help teachers promote Activity in Youth (iPLAY) intervention: a hybrid type 3 implementation-effectiveness trial

BACKGROUND

Whole-of-school programs have demonstrated success in improving student physical activity levels, but few have progressed beyond efficacy testing to implementation at-scale. The purpose of our study was to evaluate the scale-up of the 'Internet-based Professional Learning to help teachers promote Activity in Youth' (iPLAY) intervention in primary schools using the RE-AIM framework.

METHODS

We conducted a type 3 hybrid implementation-effectiveness study and collected data between April 2016 and June 2021, in New South Wales (NSW), Australia. RE-AIM was operationalised as: (i) Reach: Number and representativeness of students exposed to iPLAY; (ii) Effectiveness: Impact of iPLAY in a sub-sample of students (n = 5,959); (iii) Adoption: Number and representativeness of schools that received iPLAY; (iv) Implementation: Extent to which the three curricular and three non-curricular components of iPLAY were delivered as intended; (v) Maintenance: Extent to which iPLAY was sustained in schools. We conducted 43 semi-structured interviews with teachers (n = 14), leaders (n = 19), and principals (n = 10) from 18 schools (11 from urban and 7 from rural locations) to determine program maintenance.

RESULTS

Reach: iPLAY reached ~ 31,000 students from a variety of socio-economic strata (35% of students were in the bottom quartile, almost half in the middle two quartiles, and 20% in the top quartile).

EFFECTIVENESS

We observed small positive intervention effects for enjoyment of PE/sport (0.12 units, 95% CI: 0.05 to 0.20, d = 0.17), perceptions of need support from teachers (0.26 units, 95% CI: 0.16 to 0.53, d = 0.40), physical activity participation (0.28 units, 95% CI: 0.10 to 0.47, d = 0.14), and subjective well-being (0.82 units, 95% CI: 0.32 to 1.32, d = 0.12) at 24-months. Adoption: 115 schools received iPLAY.

IMPLEMENTATION

Most schools implemented the curricular (59%) and non-curricular (55%) strategies as intended. Maintenance: Based on our qualitative data, changes in teacher practices and school culture resulting from iPLAY were sustained.

CONCLUSIONS

iPLAY had extensive reach and adoption in NSW primary schools. Most of the schools implemented iPLAY as intended and effectiveness data suggest the positive effects observed in our cluster RCT were sustained when the intervention was delivered at-scale.

TRIAL REGISTRATION

ACTRN12621001132831.

Fund behavioral science like the frameworks we endorse: the case for increased funding of preliminary studies by the National Institutes of Health

Innovative, groundbreaking science relies upon preliminary studies (aka pilot, feasibility, proof-of-concept). In the behavioral sciences, almost every large-scale intervention is supported by a series of one or more rigorously conducted preliminary studies. The importance of preliminary studies was established by the National Institutes of Health (NIH) in 2014/2015 in two translational science frameworks (NIH Stage and ORBIT models). These frameworks outline the essential role preliminary studies play in developing the next generation of evidence-based behavioral prevention and treatment interventions. Data produced from preliminary studies are essential to secure funding from the NIH’s most widely used grant mechanism for large-scale clinical trials, namely the R01. Yet, despite their unquestionable importance, the resources available for behavioral scientists to conduct rigorous preliminary studies are limited. In this commentary, we discuss ways the existing funding structure at the NIH, despite its clear reliance upon high-quality preliminary studies, inadvertently discourages and disincentivizes their pursuit by systematically underfunding them. We outline how multiple complementary and pragmatic steps via a small reinvestment of funds from larger trials could result in a large increase in funding for smaller preliminary studies. We make the case such a reinvestment has the potential to increase innovative science, increase the number of investigators currently funded, and would yield lasting benefits for behavioral science and scientists alike.

Time-efficient physical activity intervention for older adolescents with disability: rationale and study protocol for the Burn 2 Learn adapted (B2La) cluster randomised controlled trial

INTRODUCTION

Physical activity declines during adolescence, with the lowest levels of activity observed among those with disability. Schools are ideal settings to address this issue; however, few school-based interventions have been specifically designed for older adolescents with disability. Our aim is to investigate the effects of a school-based physical activity programme, involving high-intensity interval training (HIIT), on physical, mental and cognitive health in older adolescents with disability.

METHODS AND ANALYSIS

We will evaluate the Burn 2 Learn adapted (B2La) intervention using a two-arm, parallel group, cluster randomised controlled trial with allocation occurring at the school level (treatment or waitlist control). Secondary schools will be recruited in two cohorts from New South Wales, Australia. We will aim to recruit 300 older adolescents (aged 15-19 years) with disability from 30 secondary schools (10 in cohort 1 and 20 in cohort 2). Schools allocated to the intervention group will deliver two HIIT sessions per week during scheduled specialist support classes. The sessions will include foundational aerobic and muscle strengthening exercises tailored to meet student needs. We will provide teachers with training, resources, and support to facilitate the delivery of the B2La programme. Study outcomes will be assessed at baseline, 6 months (primary endpoint), and 9 months. Our primary outcome is functional capacity assessed using the 6 min walk/push test. Secondary outcomes include physical activity, muscular fitness, body composition, cognitive function, quality of life, physical literacy, and on-task behaviour in the classroom. We will also conduct economic and process evaluations to determine cost-effectiveness, programme acceptability, implementation, adaptability, and sustainability in schools.

ETHICS AND DISSEMINATION

This study has received approval from the University of Newcastle (H-2021-0262) and the New South Wales Department of Education (SERAP: 2021257) human research ethics committees. Findings will be published in peer-reviewed journals, and key stakeholders will be provided with a detailed report following the study.

TRIAL REGISTRATION NUMBER

Australian New Zealand Clinical Trials Registry Number: ACTRN12621000884808.

Improving physical activity behaviors, physical fitness, cardiometabolic and mental health in adolescents - ActTeens Program: A protocol for a randomized controlled trial

This trial aims to investigate the effects of the ActTeens physical activity program, on adolescents’ physical activity level, health-related fitness, cardiometabolic and mental health. The trial will aim to recruit ~140 adolescents (aged 13–14 years). Participants will be randomized into either intervention or control groups. The intervention will be guided by social cognitive theory and self-determination theory and implemented over one school term (24-weeks). The ActTeens Program will include: (1) structured physical activity sessions delivered within physical education, including movement-based games and dynamic stretching warm-ups; resistance training skill development; high-intensity training workouts; and cool-downs; (2) self-monitoring plus goal setting for physical activity by pedometer-smart wearable; and (3) healthy lifestyle guidance (social support) by WhatsApp® messages about healthy eating and regular physical activity for the intervention and parents groups. Study outcomes will be assessed at baseline, 24-weeks from baseline, and 12-months from baseline. Physical activity (accelerometer) is the primary outcome. Secondary outcomes include muscular and cardiorespiratory fitness, cardiometabolic profile, and mental health. A process evaluation will be conducted (i.e., recruitment, retention, attendance, and program satisfaction). This project will have the potential to address many questions and debates regarding the implementation of physical activity interventions in low-and- middle-income countries.

Trial registration: ClinicalTrials.gov NCT05070377. Registered on 7 October 2021.

Scaling-Up Adolescent High-Intensity Interval Training Programs for Population Health

High-intensity interval training (HIIT) has become a polarizing form of exercise. In this article, we argue that adolescent HIIT programs can have population health impact if they are (i) integrated into existing opportunities, (ii) designed to develop physical literacy, (iii) delivered in an engaging manner, and (iv) guided and supported by an implementation framework that addresses relevant barriers and facilitators.

Guidelines, guidelines and more guidelines for haemorrhoid treatment: A review to sort the wheat from the chaff

AIM

Guidelines benefit patients and clinicians by distilling evidence into easy-to-read recommendations. The literature around the management of haemorrhoids is immense and guidelines are invaluable to improve treatment integrity and patient outcomes. We identified current haemorrhoid guidelines and assessed them for quality and consistency.

METHODS

A systematic search of the literature from January 2011 to October 2021 was carried out. Guidelines identified were assessed for quality using the AGREE II instrument and for consistency in terms of tabulated treatment recommendations.

RESULTS

During this period nine guidelines were identified worldwide. The general quality was poor with only one guideline considered of high enough quality for use. In general, expert selection criteria for guideline development groups were vaguely defined. There were inconsistencies in the interpretation of the published evidence leading to variation in treatment recommendations.

DISCUSSION

Fewer, higher quality guidelines, with more consistent results, are needed. Particular attention should be given to defining the selection of experts involved.

The Association between ADHD and Celiac Disease in Children

Controversy around the association between celiac disease (CeD) and attention deficit hyperactive disorder (ADHD) was addressed by a systematic review in 2015, ultimately showing no association. Since 2015, there have been several studies showing an association between celiac disease and attention deficit hyperactive disorder. This is an updated systematic review. Background: Most experts agree on the recommendation to not screen as part of the standard of care for ADHD in persons with CeD or vice versa. Simultaneously, they propose that untreated patients with CeD and neurological symptoms such as chronic fatigue, inattention, pain, and headache could be predisposed to ADHD-like behavior, namely inattention (which may be alleviated by following a gluten-free diet). The inattentive subtype of ADHD that encompasses the symptoms of inattention is phenotypically heterogeneous, as it includes the clinical construct of sluggish cognitive tempo (SCT). SCT symptoms overlap with the neurological manifestations of CeD. Methods: A systematic search (PRISMA) of PubMed, Google Scholar, EMBASE, Web of Science, Stanford Lane, SCOPUS, and Ovid was conducted for articles up to 21 February 2022. Of these, 23 studies met the criteria. Results: Out of the 23 studies, 13 showed a positive association between ADHD and CeD. Most studies that showed a positive association had been published in the last five years. Inconsistencies in the results remain due to the heterogeneous methodology used, specifically for ADHD and the outcome questionnaires, as well as a lack of reporting on ADHD subtypes. Conclusion: There is an association between ADHD and celiac disease. The current methodological limitations will be lessened if we examine the subtypes of ADHD.

Quality and clinical generalizability of feasibility outcomes in exercise prehabilitation before colorectal cancer surgery - A systematic review

Suboptimal quality of feasibility assessments might partially explain inconsistencies observed in the effectiveness of exercise prehabilitation before colorectal cancer (CRC) surgery. This systematic review aimed to assess the reporting quality and clinical generalizability of feasibility outcomes in feasibility studies addressing exercise prehabilitation before CRC surgery. PubMed/Medline, Embase, Cochrane, and CINAHL were searched to identify all feasibility studies focussing on exercise prehabilitation in CRC surgery. Reporting quality was assessed using the Thabane et al. checklist and the Consolidated Standards of Reporting Trials extension for feasibility studies. Clinical generalizability was evaluated by appraising patient participation in all steps of the study and intervention. Twelve studies were included. The main feasibility outcome in all studies was adherence to the intervention by the study sample. Based on adherence, 10 studies (83%) concluded exercise prehabilitation to be feasible. Six studies (50%) reported all details to assess patient participation showing retention rates between 18.4% and 58.2%, which was caused by non-participation and drop-out. Three feasibility studies (25%) discussed patient-reported barriers to participation and five additional studies (41%) described potential selection bias. Four studies (33%) reported lessons learned to solve issues hampering feasibility and clinical generalizability. Results suggest that true feasibility of exercise prehabilitation before CRC surgery remains questionable due to poor reporting quality, insufficient clarity regarding the representativeness of the study sample for the target population, and limited attention for clinical generalizability. Feasibility of exercise prehabilitation might be improved by offering supervised community- or home-based interventions tailored to the physical and mental abilities of the patient.

Peer-led physical activity intervention for girls aged 13 to 14 years: PLAN-A cluster RCT

Background

Increasing physical activity among girls is a public health priority. Peers play a central role in influencing adolescent behaviour. Peer-led interventions may increase physical activity in adolescent girls, and a feasibility trial had shown that PLAN-A (Peer-led physical Activity iNtervention for Adolescent girls) had evidence of promise to increase physical activity in adolescent girls.

Objective

The objective was to test whether or not PLAN-A can increase adolescent girls’ physical activity, relative to usual practice, and be cost-effective.

Design

This was a two-arm, cluster-randomised controlled trial, including an economic evaluation and a process evaluation.

Participants

State-funded secondary schools in the UK with girls in Year 9 (aged 13–14 years) participated in the trial. All Year 9 girls in participating schools were eligible.

Randomisation

Schools were the unit of allocation. They were randomised by an independent statistician, who was blinded to school identities, to the control or intervention arm, stratified by region and the England Index of Multiple Deprivation score.

Intervention

The intervention comprised peer nomination (i.e. identification of influential girls), train the trainers (i.e. training the instructors who delivered the intervention), peer supporter training (i.e. training the peer-nominated girls in techniques and strategies underpinned by motivational theory to support peer physical activity increases) and a 10-week diffusion period.

Outcomes

The primary outcome was accelerometer-assessed mean weekday minutes of moderate to vigorous physical activity among Year 9 girls. The follow-up measures were conducted 5–6 months after the 10-week intervention, when the girls were in Year 10 (which was also 12 months after the baseline measures). Analysis used a multivariable, mixed-effects, linear regression model on an intention-to-treat basis. Secondary outcomes included weekend moderate to vigorous physical activity, and weekday and weekend sedentary time. Intervention delivery costs were calculated for the economic evaluation.

Results

A total of 33 schools were approached; 20 schools and 1558 pupils consented. Pupils in the intervention arm had higher Index of Multiple Deprivation scores than pupils in the control arm. The numbers randomised were as follows: 10 schools (n = 758 pupils) were randomised to the intervention arm and 10 schools (n = 800 pupils) were randomised to the control arm. For analysis, a total of 1219 pupils provided valid weekday accelerometer data at both time points (intervention, n = 602; control, n = 617). The mean weekday moderate to vigorous physical activity was similar between groups at follow-up. The central estimate of time spent engaging in moderate to vigorous physical activity was 2.84 minutes lower in the intervention arm than in the control arm, after adjustment for baseline mean weekday moderate to vigorous physical activity, the number of valid days of data and the stratification variables; however, this difference was not statistically significant (95% confidence interval –5.94 to 0.25; p = 0.071). There were no between-arm differences in the secondary outcomes. The intervention costs ranged from £20.85 to £48.86 per pupil, with an average cost of £31.16.

Harms

None.

Limitations

The trial was limited to south-west England.

Conclusions

There was no evidence that PLAN-A increased physical activity in Year 9 girls compared with usual practice and, consequently, it was not cost-effective.

Future work

Future work should evaluate the utility of whole-school approaches to promote physical activity in schools.

Trial registration

This trial is registered as ISRCTN14539759.

Funding

This project was funded by the National Institute for Health Research (NIHR) Public Health Research programme and will be published in full in Public Health Research; Vol. 10, No. 6. See the NIHR Journals Library website for further project information. This trial was designed and delivered in collaboration with the Bristol Randomised Trials Collaboration (BRTC), a United Kingdom Clinical Research Commission (UKCRC)-registered Clinical Trials Unit that, as part of the Bristol Trials Centre, is in receipt of NIHR Clinical Trials Unit support funding. The sponsor of this trial was University of Bristol, Research and Enterprise Development www.bristol.ac.uk/red/. The costs of delivering the intervention were funded by Sport England.

Early-stage studies to larger-scale trials: investigators’ perspectives on scaling-up childhood obesity interventions

Background

Pilot/feasibility studies play an important role in the development and refinement of behavioral interventions by providing information about feasibility, acceptability, and potential efficacy. Despite their importance and wide-spread use, the approaches taken by behavioral scientists to scale-up early-stage studies to larger-scale trials has received little attention. The aim of our study was to understand the role that pilot studies play in the development and execution of larger-scale trials.

Methods

We conducted interviews with childhood obesity researchers who had published pilot behavioral interventions and larger-scale trials of the same or similar interventions. Questions were asked about the role of pilot studies in developing larger-scale trials and the challenges encountered when scaling-up an intervention based upon pilot findings. Data were coded and analyzed using an inductive analytic approach to identify themes.

Results

Twenty-four interventionists (54% women, 37–70 years old, mean 20 years since terminal degree) completed a total of 148 pilot studies across their careers (mean 6.4, range 1–20), of which 59% were scaled-up. Scaling was described as resource intensive and pilot work was considered essential to successfully competing for funding by 63% of the sample (n = 15). When asked to define a high-quality pilot study, interventionists described studies that allowed them to evaluate two independent factors: components of their intervention (e.g., acceptability, feasibility) and study parameters (e.g., sample size, measures). Interventionists expressed that more process implementation measures, different study designs, and additional iterations could improve decisions to scale-up. Most agreed that pilot studies were likely to produce inflated estimates of potential efficacy though only nine interventionists provided potential solutions for decreasing inflated measures of efficacy. Suggested major causes of inflated effects included high levels of oversight in pilot studies (e.g., researcher support), reliance on subjective measures, and utilizing convenience or highly motivated samples. Potential solutions included designing pilots for real-world implementation, only conducting randomized controlled pilot studies, and pre-registering pilot studies.

Conclusions

Pilot studies purposes are multifaceted and deemed essential to obtaining funding for larger-scale trials. Clarifying the form and function of preliminary, early-stage research may enhance the productive utilization of early-stage studies and reduced drops in efficacy when transitioning to larger scale studies.

Supplementary Information

The online version contains supplementary material available at 10.1186/s40814-022-00991-8.

Impact of risk of generalizability biases in adult obesity interventions: A meta-epidemiological review and meta-analysis

Biases introduced in early-stage studies can lead to inflated early discoveries. The risk of generalizability biases (RGBs) identifies key features of feasibility studies that, when present, lead to reduced impact in a larger trial. This meta-study examined the influence of RGBs in adult obesity interventions. Behavioral interventions with a published feasibility study and a larger scale trial of the same intervention (e.g., pairs) were identified. Each pair was coded for the presence of RGBs. Quantitative outcomes were extracted. Multilevel meta-regression models were used to examine the impact of RGBs on the difference in the effect size (ES, standardized mean difference) from pilot to larger scale trial. A total of 114 pairs, representing 230 studies, were identified. Overall, 75% of the pairs had at least one RGB present. The four most prevalent RGBs were duration (33%), delivery agent (30%), implementation support (23%), and target audience (22%) bias. The largest reductions in the ES were observed in pairs where an RGB was present in the pilot and removed in the larger scale trial (average reduction ES -0.41, range -1.06 to 0.01), compared with pairs without an RGB (average reduction ES -0.15, range -0.18 to -0.14). Eliminating RGBs during early-stage testing may result in improved evidence.

An embedded multiple case study: using CFIR to map clinical food security screening constructs for the development of primary care practice guidelines

BACKGROUND

Food insecurity (FI), the limited access to healthy food to live an active and healthy life, is a social determinant of health linked to poor dietary health and difficulty with disease management in the United States (U.S.). Healthcare experts support the adoption of validated screening tools within primary care practice to identify and connect FI patients to healthy and affordable food resources. Yet, a lack of standard practices limits uptake. The purpose of this study was to understand program processes and outcomes of primary care focused FI screening initiatives that may guide wide-scale program implementation.

METHODS

This was an embedded multiple case study of two primary care-focused initiatives implemented in two diverse health systems in Chicago and Suburban Cook County that routinely screened patients for FI and referred them to onsite food assistance programs. The Consolidated Framework for Implementation Research and an iterative process were used to collect/analyze qualitative data through semi-structured interviews with N = 19 healthcare staff. Intended program activities, outcomes, actors, implementation barriers/facilitators and overarching implementation themes were identified as a part of a cross-case analysis.

RESULTS

Programs outcomes included: the number of patients screened, identified as FI and that participated in the onsite food assistance program. Study participants reported limited internal resources as implementation barriers for program activities. The implementation climate that leveraged the strength of community collaborations and aligned internal, implementation climate were critical facilitators that contributed to the flexibility of program activities that were tailored to fill gaps in resources and meet patient and clinician needs.

CONCLUSION

Highly adaptable programs and the healthcare context enhanced implementation feasibility across settings. These characteristics can support program uptake in other settings, but should be used with caution to preserve program fidelity. A foundational model for the development and testing of standard clinical practice was the product of this study.

CHoosing Active Role Models to INspire Girls (CHARMING): protocol for a cluster randomised feasibility trial of a school-based, community-linked programme to increase physical activity levels in 9-10-year-old girls

Background

In the UK, there is evidence that girls’ physical activity tends to decline to a greater extent than boys as they enter adolescence. ‘Role models’ could play a vital role in inspiring girls to become or remain physically active. The CHARMING Programme is a primary school-based community linked role-model programme, co-developed in 2016, with children, parents, schools and wider stakeholders. It involves different types of physical activity delivered for 1-h each week by a community provider and peer role models (e.g. older girls from secondary schools) joining in with the sessions. The programme ultimately aims to increase and sustain physical activity levels among 9–10-year-old girls. This study aims to assess the feasibility and acceptability of the CHARMING Programme and of evaluating it using a randomised trial.

Methods

This study is a feasibility cluster randomised controlled trial, with embedded process evaluation and health economic evaluation. Approximately 90 Year 5 (i.e. 9–10-year-old) girls will be recruited across six primary schools in Mid-South Wales. Participating schools will be allocated to the programme: control on a 2:1 basis; four intervention schools will run the CHARMING Programme and two will continue with usual practice. A survey and accelerometer will be administered at baseline and repeated at 12 months. Interviews and focus groups will be conducted post-intervention delivery. The primary aim is to assess feasibility of a future randomised trial via the recruitment of schools, participants and role models; randomisation; retention; reach; data collection completion rates; programme adherence; and programme fidelity, views on intervention acceptability and programme barriers and facilitators. Secondary aims are to evaluate established physical activity outcome measures for children plus additional health economic outcomes for inclusion in a future full-scale trial.

Discussion

The results of this study will inform decisions on whether and how to proceed to a full-scale evaluation of the effectiveness and cost-effectiveness of the CHARMING Programme to improve or sustain physical activity.

Trial registration

ClinicalTrials.gov ISRCTN36223327. Registered March 29, 2021

Supplementary Information

The online version contains supplementary material available at 10.1186/s40814-021-00961-6.

School Based Motor Skill Interventions for Developmentally Delayed and Non-Delayed Children

Introduction: A mere 33% of all children meet the recommended minimum physical activity guidelines for adequate health maintenance. Available literature however suggests children are more likely to be active when they are competent with their own motor ability. This review aimed to evaluate how several regimented motor skills training courses and interventions improve motor skill competence among children compared with age matched control peers. Method: Electronic databases were searched and included Medline Complete and Psych INFO (both hosted by EBSCO Host). The search syntax examined titles and abstracts. The study aimed to create novelty by examining participants with and without developmental delays simultaneously from studies around the globe. Included interventions were aimed at the most crucial developmental years for children (between 3 and 11 years). Results: Results were found in favor of the motor skill intervention groups (from pre-to post-test). Included interventions involved weekly motor skills exposure of 60 to 120 minutes for periods of between 2 and 6 months. Over 50% of included interventions involved alterations to current school curriculums. The included studies were of moderate to high quality. Conclusion: The findings suggest that for those with and without developmental delays, several interventions can be effectively applied in once weekly 60-minute sessions (over eight or more weeks) to improve children’s motor skill abilities. Applying appropriate difficulty to interventions seems equally influential. Implications are discussed.

Physical Activity and Healthy Eating Programming in Schools to Support Student's Health-Related Fitness: An Observational Study

Centers for Disease Control (CDC) guidelines recommend schools use a coordinated health approach to support healthy eating and physical activity. This study examines whether the number of healthy eating and physical activity programs and activities used by schools and their perceived success relate to students’ health-related fitness. This observational study used data from the Healthy Zone Schools Program. Data (collected in 2017–2019) were integrated from three sources: (1) school surveys, (2) FitnessGram^®, and (3) the Texas Education Agency. Independent variables were the number of health promotion programs and activities and their perceived success; dependent variables were meeting Healthy Fitness Zone Standards (HFZ) for aerobic capacity and body mass index (BMI). We used mixed-effects logistic regression models. Fifty-six schools were in the analytic sample (n = 15,096 students with aerobic capacity data and n = 19,969 with BMI data). Results indicated the perceived success of physical activity programs/activities was significantly associated with students meeting HFZ standards for aerobic capacity (OR = 1.32, CI = 1.06–1.63). There was a significant direct association between the number of physical activity and healthy eating activities implemented (OR = 1.04, CI = 1.01–1.06) and students meeting HFZ for BMI. Schools using multiple health programs and activities need to balance the number provided with their capacity to maintain success.

Identifying effective intervention strategies to reduce children's screen time: a systematic review and meta-analysis

BACKGROUND

Excessive screen time ([Formula: see text] 2 h per day) is associated with childhood overweight and obesity, physical inactivity, increased sedentary time, unfavorable dietary behaviors, and disrupted sleep. Previous reviews suggest intervening on screen time is associated with reductions in screen time and improvements in other obesogenic behaviors. However, it is unclear what study characteristics and behavior change techniques are potential mechanisms underlying the effectiveness of behavioral interventions. The purpose of this meta-analysis was to identify the behavior change techniques and study characteristics associated with effectiveness in behavioral interventions to reduce children's (0-18 years) screen time.

METHODS

A literature search of four databases (Ebscohost, Web of Science, EMBASE, and PubMed) was executed between January and February 2020 and updated during July 2021. Behavioral interventions targeting reductions in children's (0-18 years) screen time were included. Information on study characteristics (e.g., sample size, duration) and behavior change techniques (e.g., information, goal-setting) were extracted. Data on randomization, allocation concealment, and blinding was extracted and used to assess risk of bias. Meta-regressions were used to explore whether intervention effectiveness was associated with the presence of behavior change techniques and study characteristics.

RESULTS

The search identified 15,529 articles, of which 10,714 were screened for relevancy and 680 were retained for full-text screening. Of these, 204 studies provided quantitative data in the meta-analysis. The overall summary of random effects showed a small, beneficial impact of screen time interventions compared to controls (SDM = 0.116, 95CI 0.08 to 0.15). Inclusion of the Goals, Feedback, and Planning behavioral techniques were associated with a positive impact on intervention effectiveness (SDM = 0.145, 95CI 0.11 to 0.18). Interventions with smaller sample sizes (n < 95) delivered over short durations (< 52 weeks) were associated with larger effects compared to studies with larger sample sizes delivered over longer durations. In the presence of the Goals, Feedback, and Planning behavioral techniques, intervention effectiveness diminished as sample size increased.

CONCLUSIONS

Both intervention content and context are important to consider when designing interventions to reduce children's screen time. As interventions are scaled, determining the active ingredients to optimize interventions along the translational continuum will be crucial to maximize reductions in children's screen time.

Small studies, big decisions: the role of pilot/feasibility studies in incremental science and premature scale-up of behavioral interventions

Background

Careful consideration and planning are required to establish “sufficient” evidence to ensure an investment in a larger, more well-powered behavioral intervention trial is worthwhile. In the behavioral sciences, this process typically occurs where smaller-scale studies inform larger-scale trials. Believing that one can do the same things and expect the same outcomes in a larger-scale trial that were done in a smaller-scale preliminary study (i.e., pilot/feasibility) is wishful thinking, yet common practice. Starting small makes sense, but small studies come with big decisions that can influence the usefulness of the evidence designed to inform decisions about moving forward with a larger-scale trial. The purpose of this commentary is to discuss what may constitute sufficient evidence for moving forward to a definitive trial. The discussion focuses on challenges often encountered when conducting pilot/feasibility studies, referred to as common (mis)steps, that can lead to inflated estimates of both feasibility and efficacy, and how the intentional design and execution of one or more, often small, pilot/feasibility studies can play a central role in developing an intervention that scales beyond a highly localized context.

Main body

Establishing sufficient evidence to support larger-scale, definitive trials, from smaller studies, is complicated. For any given behavioral intervention, the type and amount of evidence necessary to be deemed sufficient is inherently variable and can range anywhere from qualitative interviews of individuals representative of the target population to a small-scale randomized trial that mimics the anticipated larger-scale trial. Major challenges and common (mis)steps in the execution of pilot/feasibility studies discussed are those focused on selecting the right sample size, issues with scaling, adaptations and their influence on the preliminary feasibility and efficacy estimates observed, as well as the growing pains of progressing from small to large samples. Finally, funding and resource constraints for conducting informative pilot/feasibility study(ies) are discussed.

Conclusion

Sufficient evidence to scale will always remain in the eye of the beholder. An understanding of how to design informative small pilot/feasibility studies can assist in speeding up incremental science (where everything needs to be piloted) while slowing down premature scale-up (where any evidence is sufficient for scaling).

Physical activity behaviours in adolescence: current evidence and opportunities for intervention

Young people aged 10-24 years constitute 24% of the world's population; investing in their health could yield a triple benefit-eg, today, into adulthood, and for the next generation. However, in physical activity research, this life stage is poorly understood, with the evidence dominated by research in younger adolescents (aged 10-14 years), school settings, and high-income countries. Globally, 80% of adolescents are insufficiently active, and many adolescents engage in 2 h or more daily recreational screen time. In this Series paper, we present the most up-to-date global evidence on adolescent physical activity and discuss directions for identifying potential solutions to enhance physical activity in the adolescent population. Adolescent physical inactivity probably contributes to key global health problems, including cardiometabolic and mental health disorders, but the evidence is methodologically weak. Evidence-based solutions focus on three key components of the adolescent physical activity system: supportive schools, the social and digital environment, and multipurpose urban environments. Despite an increasing volume of research focused on adolescents, there are still important knowledge gaps, and efforts to improve adolescent physical activity surveillance, research, intervention implementation, and policy development are urgently needed.

Establishing Effectiveness of a Community-based, Physical Activity Program for Fathers and Daughters: A Randomized Controlled Trial

Background

The ‘Dads And Daughters Exercising and Empowered’ (DADEE) program significantly improved physical activity levels of fathers and their daughters in an efficacy trial. However, the effectiveness of interventions when delivered in real-world settings needs to be established.

Purpose

To evaluate the effectiveness of the DADEE intervention when delivered in community settings by trained facilitators.

Methods

We conducted a two-arm RCT, (baseline and 3-months post-intervention assessments), in Newcastle, Australia. In 2016, 155 fathers (27–60 years) and 189 primary-school-aged daughters (4–12 years) (n = 344) were randomly allocated to the intervention (78 fathers, 95 daughters) or waitlist-control (77 fathers, 94 daughters) groups. Trained facilitators delivered the 9-week DADEE program (weekly sessions plus home-based tasks). Primary outcomes were fathers’ and daughters’ physical activity (steps/day). Secondary outcomes included screen-time, weight status, daughters’ fundamental movement skill (FMS) proficiency, perceived sports competence, and fathers’ parenting practices. Effects were assessed using linear mixed models.

Results

Primary outcome follow-up data were collected from 88% of fathers and 89% of daughters. Significant group-by-time differences in mean daily steps were found for fathers’ (adjusted difference = +1,638; 95% CI: 833, 2,443, d = 0.7) and daughters’ (adjusted difference = +1,023 steps/day; 95% CI: 259, 1,787; d = 0.4) physical activity. Significant effects were observed for daughters’ screen-time, FMS, and some parenting practices. No significant effects were identified for weight status, or fathers’screen-time or self-reported MVPA. Program attendance, satisfaction and fidelity were very high.

Conclusion

This study established the effectiveness of the DADEE intervention when delivered in community settings by trained facilitators. Importantly, the findings were comparable to those of the efficacy RCT delivered by the research team. To maximize public health benefits, a larger-scale dissemination of the program appears warranted.

Trial Registration Australian New Zealand Clinical Trial Registry: ACTRN12616001270404 Human Research Ethics Committee: H-2014-0330

Brief Exercise at Work (BE@Work): A Mixed-Methods Pilot Trial of a Workplace High-Intensity Interval Training Intervention

Introduction: The efficacy of high-intensity interval training (HIIT) for improving markers of physical fitness and cardiometabolic health is promising. The workplace is one non-laboratory setting where the effectiveness of HIIT could be explored. The aim of this study was to undertake a mixed-methods exploratory pilot trial of a workplace HIIT intervention named Brief Exercise at Work (BE@Work). Methods: Fifty-four healthy employees (mean ± standard deviation [SD] age 46 ± 10 years) from two workplaces in Northeast England were allocated to 8 weeks of thrice-weekly workplace HIIT based on boxing, stair climbing and stepping, comprising 4-7 60 s high-intensity intervals interspersed with 75 s rest (n = 30), or a no-intervention control (n = 24). The primary outcome was the change SD of predicted maximal oxygen consumption (VO2max). Markers of physical fitness, cardiometabolic health and mental well-being were also measured at baseline and follow-up. Participant perceptions of the intervention were explored in post-intervention focus groups (n = 9). Results: Mean (±SD) session attendance was 82% (±15%). Mean peak heart rate across the intervention was 87% of age-predicted maximal heart rate with a within- and between-subject SD of 5.5% and 3.7%, respectively. The SD of changes in predicted VO2max was 6.6 mL·kg-1·min-1 across both groups, which can be used to inform sample size estimations for a future full trial. The control-adjusted mean increase (95% confidence interval) in predicted VO2max was 3.9 (-0.2 to 8.1) mL·kg-1·min-1, corresponding to a Cohen's D of 0.47. We also observed preliminary evidence of small to moderate effects in favour of the intervention group for non-dominant leg extensor muscle power, markers of health-related quality of life, well-being and perceived stress and small to moderate effects in favour of the controls in perceived pain, physical activity and high-density lipoprotein cholesterol. During HIIT, focus group participants reported physiological responses they perceived as unpleasant or tiring (e.g., breathlessness, local muscular fatigue), but also that they felt alert and energised afterwards. Conclusion: The findings of this exploratory pilot trial support the implementation of a definitive randomised controlled trial to quantify the effectiveness of a workplace HIIT intervention.

Implementation at-scale of school-based physical activity interventions: A systematic review utilizing the RE-AIM framework

School-based interventions can increase young people's physical activity levels, but few are implemented at-scale (i.e., the expanded delivery of efficacious interventions under real-world conditions into new/broader populations). The Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework can be used to describe the extent to which interventions have been implemented at-scale. The aim of our review was to determine the extent to which studies of school-based physical activity interventions implemented at-scale reported information across the RE-AIM dimensions. We conducted a systematic search of seven electronic databases to identify studies published up to June 2019. A total of 26 articles (representing 14 individual studies) met the inclusion criteria and were analyzed. Eleven studies reported actual or estimated number of students exposed to the intervention; however, the representativeness of these students was rarely reported. Nine studies reported the intervention effect on the primary outcome during scale-up. Ten studies reported the rate of participating schools/teachers; however, none reported on the characteristics of adopters/nonadopters. Eight studies reported intervention fidelity. Eleven studies described the extent to which the intervention was sustained in schools. There was considerable variability in the reporting of RE-AIM outcomes across studies. There is a need for greater consistency in the evaluation, and reporting of, school-based physical activity interventions implemented at-scale.

Effectiveness and cost-effectiveness of the PLAN-A intervention, a peer led physical activity program for adolescent girls: results of a cluster randomised controlled trial

BACKGROUND

Physical activity is associated with improved health. Girls are less active than boys. Pilot work showed that a peer-led physical activity intervention called PLAN-A was a promising method of increasing physical activity in secondary school age girls. This study examined the effectiveness and cost-effectiveness of the PLAN-A intervention.

METHODS

We conducted a cluster randomised controlled trial with Year 9 (13-14 year old) girls recruited from 20 secondary schools. Schools were randomly assigned to the PLAN-A intervention or a non-intervention control group after baseline data collection. Girls nominated students to be peer leaders. The top 18 % of girls nominated by their peers in intervention schools received three days of training designed to prepare them to support physical activity. Data were collected at two time points, baseline (T0) and 5-6 months post-intervention (T1). Participants wore an accelerometer for seven days to assess the primary outcome of mean weekday minutes of moderate-to-vigorous physical activity (MVPA). Multivariable mixed effects linear regression was used to estimate differences in the primary outcome between the two arms on an Intention-to-Treat (ITT) basis. Resource use and quality of life were measured and a within trial economic evaluation from a public sector perspective was conducted.

RESULTS

A total of 1558 girls were recruited to the study. At T0, girls in both arms engaged in an average of 51 min of MVPA per weekday. The adjusted mean difference in weekday MVPA at T1 was - 2.84 min per day (95 % CI = -5.94 to 0.25) indicating a slightly larger decline in weekday MVPA in the intervention group. Results were broadly consistent when repeated using a multiple imputation approach and for pre-specified secondary outcomes and sub-groups. The mean cost of the PLAN-A intervention was £2817 per school, equivalent to £31 per girl. Economic analyses indicated that PLAN-A did not lead to demonstrable cost-effectiveness in terms of cost per unit change in QALY.

CONCLUSIONS

This study has shown that the PLAN-A intervention did not result in higher levels of weekday MVPA or associated secondary outcomes among Year 9 girls. The PLAN-A intervention should not be disseminated as a public health strategy.

TRIAL REGISTRATION

ISRCTN14539759 -31 May, 2018.

Determining sample size for progression criteria for pragmatic pilot RCTs: the hypothesis test strikes back!

Background

The current CONSORT guidelines for reporting pilot trials do not recommend hypothesis testing of clinical outcomes on the basis that a pilot trial is under-powered to detect such differences and this is the aim of the main trial. It states that primary evaluation should focus on descriptive analysis of feasibility/process outcomes (e.g. recruitment, adherence, treatment fidelity). Whilst the argument for not testing clinical outcomes is justifiable, the same does not necessarily apply to feasibility/process outcomes, where differences may be large and detectable with small samples. Moreover, there remains much ambiguity around sample size for pilot trials.

Methods

Many pilot trials adopt a ‘traffic light’ system for evaluating progression to the main trial determined by a set of criteria set up a priori. We construct a hypothesis testing approach for binary feasibility outcomes focused around this system that tests against being in the RED zone (unacceptable outcome) based on an expectation of being in the GREEN zone (acceptable outcome) and choose the sample size to give high power to reject being in the RED zone if the GREEN zone holds true. Pilot point estimates falling in the RED zone will be statistically non-significant and in the GREEN zone will be significant; the AMBER zone designates potentially acceptable outcome and statistical tests may be significant or non-significant.

Results

For example, in relation to treatment fidelity, if we assume the upper boundary of the RED zone is 50% and the lower boundary of the GREEN zone is 75% (designating unacceptable and acceptable treatment fidelity, respectively), the sample size required for analysis given 90% power and one-sided 5% alpha would be around n = 34 (intervention group alone). Observed treatment fidelity in the range of 0–17 participants (0–50%) will fall into the RED zone and be statistically non-significant, 18–25 (51–74%) fall into AMBER and may or may not be significant and 26–34 (75–100%) fall into GREEN and will be significant indicating acceptable fidelity.

Discussion

In general, several key process outcomes are assessed for progression to a main trial; a composite approach would require appraising the rules of progression across all these outcomes. This methodology provides a formal framework for hypothesis testing and sample size indication around process outcome evaluation for pilot RCTs.

Supplementary Information

The online version contains supplementary material available at 10.1186/s40814-021-00770-x.