Maximizing the Implementation Quality of Evidence-Based Preventive Interventions in Schools: A Conceptual Framework

"I'm not sure whether I will implement it": exploring barriers and facilitators to implementing a digital "healthy eating" resource in early education and care settings - teachers' perspectives

BACKGROUND

Scaling up effective interventions to promote healthy eating habits in children in real-world settings is a pressing need. The success of implementation hinges crucially on engaging end-users and tailoring interventions to meet their specific needs. Building on our prior evaluation of a digital "healthy eating" resource for early childhood education and care (ECEC) staff; this qualitative study aims to pinpoint the barriers and facilitators that influence the successful implementation of such interventions.

METHODS

We conducted twelve semi-structured interviews with ECEC teachers in a Norwegian municipality. Interview participants were later invited to participate in focus groups where two discussions were conducted with five of the participants to reflect on the initial interview findings. Thematic analysis, facilitated by NVivo software, was employed to analyse the data, aiming to identify and summarize teachers' subjective experiences and perspectives.

RESULTS

Teachers' perceptions of barriers to the implementation of an upcoming digital "healthy eating" resource included: (1) No established tradition of using digital resources at work; (2) Uncertainty regarding the achievable outcomes of implementation; (3) Perception of the new "healthy eating" resource as cooking-focused and unnecessary; and (4) Hectic everyday life serving as a barrier to the long-term use of a digital resource. Facilitators for implementation included: (1) A user-friendly format; (2) Newsletters featuring seasonal tips inspire and serve as effective reminders; (3) Emphasis on research and legislation; and (4) Structuring the resource as a series and an idea bank.

CONCLUSIONS

The findings underscore the essential need for tailored strategies and comprehensive support structures to successfully implement a culturally appropriate digital "healthy eating" resource for ECEC staff, ensuring effectiveness and feasibility.

TRIAL REGISTRATION

This study was not registered in a trial registry as it is not a clinical trial or intervention study but serves as a pilot for the Nutrition Now study, trial identifier ISRCTN10694967 ( https://doi.org/10.1186/ISRCTN10694967 ), registration date: 19/06/2022.

Empowering School Staff to Support Pupil Mental Health Through a Brief, Interactive Web-Based Training Program: Mixed Methods Study

BACKGROUND

Schools in the United Kingdom and elsewhere are expected to protect and promote pupil mental health. However, many school staff members do not feel confident in identifying and responding to pupil mental health difficulties and report wanting additional training in this area.

OBJECTIVE

We aimed to explore the feasibility of Kognito's At-Risk for Elementary School Educators, a brief, interactive web-based training program that uses a simulation-based approach to improve school staff's knowledge and skills in supporting pupil mental health.

METHODS

We conducted a mixed methods, nonrandomized feasibility study of At-Risk for Elementary School Educators in 6 UK primary schools. Our outcomes were (1) school staff's self-efficacy and preparedness to identify and respond to pupil mental health difficulties, (2) school staff's identification of mental health difficulties and increased risk of mental health difficulties, (3) mental health support for identified pupils (including conversations about concerns, documentation of concerns, in-class and in-school support, and referral and access to specialist mental health services), and (4) the acceptability and practicality of the training. We assessed these outcomes using a series of questionnaires completed at baseline (T1), 1 week after the training (T2), and 3 months after the training (T3), as well as semistructured qualitative interviews. Following guidance for feasibility studies, we assessed quantitative outcomes across time points by comparing medians and IQRs and analyzed qualitative data using reflexive thematic analysis.

RESULTS

A total of 108 teachers and teaching assistants (TAs) completed T1 questionnaires, 89 (82.4%) completed T2 questionnaires, and 70 (64.8%) completed T3 questionnaires; 54 (50%) completed all 3. Eight school staff members, including teachers, TAs, mental health leads, and senior leaders, participated in the interviews. School staff reported greater confidence and preparedness in identifying and responding to mental health difficulties after completing the training. The proportion of pupils whom they identified as having mental health difficulties or increased risk declined slightly over time (medianT1=10%; medianT2=10%; medianT3=7.4%), but findings suggested a slight increase in accuracy compared with a validated screening measure (the Strengths and Difficulties Questionnaire). In-school mental health support outcomes for identified pupils improved after the training, with increases in formal documentation and communication of concerns as well as provision of in-class and in-school support. Referrals and access to external mental health services remained constant. The qualitative findings indicated that school staff perceived the training as useful, practical, and acceptable.

CONCLUSIONS

The findings suggest that brief, interactive web-based training programs such as At-Risk for Elementary School Educators are a feasible means to improve the identification of and response to mental health difficulties in UK primary schools. Such training may help address the high prevalence of mental health difficulties in this age group by helping facilitate access to care and support.

A scoping review of the implementation and cultural adaptation of school-based mental health promotion and prevention interventions in low-and middle-income countries

Effective school-based mental health promotion and prevention interventions in low-and middle-income countries (LMICs) can positively impact the mental health and well-being of large numbers of young people. This scoping review aimed to investigate the implementation of effective mental health promotion and prevention interventions in LMIC schools. A scoping review of the international literature was conducted and followed the Preferred Reporting Items for Systematic reviews and Meta-Analysis extension for Scoping Reviews guidelines. Medline, PsycInfo, Scopus, Embase, CINAHL and Cochrane were searched for peer-reviewed literature published from 2014 to 2022. PsycExtra, Google Scholar and the websites of key organisations were searched for relevant grey literature. Study selection focussed on mental health promotion interventions, including the development of social and emotional skills and mental health literacy, and prevention interventions, including anti-bullying and skill-based interventions for "at-risk" students. Twenty-seven studies evaluating 25 school-based interventions in 17 LMICs were included in the review. Fifteen interventions were developed in the implementing country and 10 were adapted from high-income countries (HICs) or other settings. Findings from the studies reviewed were generally positive, especially when interventions were implemented to a high quality. Universal life-skills interventions were found to increase social and emotional skills, decrease problem behaviours and positively impact students' mental health and well-being. Mental health literacy interventions increased mental health knowledge and decreased stigma among students and school staff. Outcomes for externally facilitated anti-bullying interventions were less positive. All 19 effective studies reported on some aspects of programme implementation, and 15 monitored implementation fidelity. Eleven studies outlined the programme's underpinning theoretical model. Only four studies reported on the cultural adaptation of programmes in detail. Including young people in the adaptation process was reported to facilitate natural cultural adaptation of programmes, while input from programme developers was considered key to ensuring that the core components of interventions were retained. The review findings indicate increasing evidence of effective mental health interventions in LMIC schools. To facilitate the sustainability, replication and scaling-up of these interventions, greater attention is needed to reporting on intervention core components, and the processes of implementation and cultural adaptation in the local setting.

Preliminary evaluation of a novel group-based motivational interviewing intervention with adolescents: a feasibility study

Drinking is closely intertwined with social life among many adolescents, particularly in Europe. Group-based interventions, such as group-based motivational interviewing (group MI), have shown the capacity to prevent and reduce hazardous drinking and related problems among adolescents, but few examinations have been conducted in a European high school setting. This study examines the preliminary outcomes of a pilot group MI intervention among Danish adolescents. High school students (ages 15-18 years) were randomly allocated to two 1-h group MI sessions delivered in a school setting (N = 65) or an assessment only control condition (N = 67). Data were collected in August and November 2020 using online self-administrated questionnaires regarding the acceptability of the intervention and past month alcohol use. The pilot group MI intervention showed high feasibility and acceptability in this setting and with this age group. Group MI adolescents significantly reduced peak drinks per drinking day compared to assessment only adolescents (-2.7 drinks, p < 0.05). Results are discussed in relation to the metrics being evaluated during COVID-19 lockdown, including increased social restrictions at follow-up compared to baseline. Group MI shows promise for reducing hazardous alcohol use among Danish adolescents. In addition, the findings indicate the importance of building on and extending this work in future larger, better-powered randomized controlled trials.

Cultural adaptation, content, and protocol of a feasibility study of school-based "Let's learn about emotions" intervention for Finnish primary school children

Introduction

Emotional awareness and emotion regulation are crucial for cognitive and socio-emotional development in children. School-based interventions on socio-emotional skills have the potential to prevent these problems and promote well-being of children. The Japanese school-based program, Universal Unified Prevention Program for Diverse Disorders (Up2-D2), has shown preventive effects on mental health of children in Japan. The aims of this protocol paper are to describe the unique process of adapting the Up2-D2 from Eastern to Western context, and to present a feasibility study of the intervention, conducted in Finland.

Methods

The cultural adaptation process started with the linguistic translation of materials, followed by the modification of language to fit the Finnish context. While the Japanese ideology was saved, some content was adapted to fit Finnish school children. Further modifications were made based on feedback from pupils and teachers. The Finnish version of the program was named "Let's learn about emotions" and consisted of 12 sessions and targeted 8- to 12-year-old pupils. A teacher education plan was established to assist Finnish teachers with the intervention, including a workshop, teachers' manual, brief introductory videos, and online support sessions. A feasibility study involving 512 4th graders in the City of Hyvinkää, South of Finland, was conducted. It assessed emotional and behavioral problems, classroom climate, bullying, loneliness, perception of school environment, knowledge of emotional awareness, and program acceptability.

Discussion

The originality of this study underlies in the East-West adaptation of a cognitive behavioral therapy-based program. If promising feasibility findings are replicated in Finland, it could pave the way for further research on implementing such programs in diverse contexts and cultures, promoting coping skills, awareness, social skills and early prevention of child mental health problems.

Ethics

The ethical board of the University of Turku gave ethics approval for this research. The educational board of the City of Hyvinkää accepted this study.

Policy, Systems, and Environmental Changes in Child Nutrition Programs: A Systematic Literature Review

The National School Lunch Program (NSLP) provides healthy food to millions of children annually. To promote increased lunch consumption, policy, systems, and environmental (PSE) change strategies are being implemented in child nutrition programs. An evaluation of the current evidence supporting PSE interventions in school nutrition programs is needed to facilitate evidence-based practices across the nation for programs. This systematic review aims to determine the quality and breadth of available evidence of the effectiveness of PSE strategies on the consumption and waste of fruits, vegetables, milk, and water in the NSLP. The inclusion criteria required studies to occur in a United States K-12 school setting, data collection after 2012, report consumption and waste findings for fruit, vegetable, milk, or water, and be an original research article. Articles included in the review are restricted to positive or neutral quality. Thirty studies are included, policy level (n = 4), systems level (n = 8), environmental level (n = 10), and multi-category (n = 8). Results from positively rated policy-level studies suggest that recess before lunch may increase milk consumption, whereas removing flavored milk may decrease consumption. System-level studies of offering vegetables first in isolation of other meal components and offering spiced vegetables compared with traditional preparations may increase vegetable consumption, and locally procuring produce may increase fruit and vegetable consumption. Environmental-level studies such as water promotion strategies such as placing cups near drinking fountains may increase water consumption. Improving the convenience, attractiveness, and palatability of fruits and vegetables may increase consumption. Future PSE research in child nutrition programs should incorporate implementation aides and metrics into their study designs to allow a better understanding of how to sustain interventions from the perspective of school nutrition professionals.

Promoting Peer Connectedness Through Social-Emotional Learning: Evaluating the Intervention Effect Mechanisms and Implementation Factors of a Social-Emotional Learning Programme for 9 to 12-Year-Olds

There is little evidence regarding the effect mechanisms of social-emotional learning programs on children's peer relationships. The current study evaluated a novel school-based social-emotional learning program for the first year of secondary school assessing effects on social-emotional skills, peer connectedness, happiness, student and teacher classroom climate. The sample included 19 intervention classrooms (n = 399) and 16 waitlist-control classrooms (n = 281), with a mean age of 10.34 (SD = 0.76) and 48.8% girls. The main intervention effect analysis followed a per-protocol approach and was thus conducted with eight classes that finished all sessions (n = 195) and the control group classes (n = 281). It was further hypothesized that increases in social-emotional skills would predict peer connectedness and class climate increases, which would predict happiness. Results indicated significant intervention effects for social skills, peer connectedness and happiness. Classroom climate declined for both groups, seemingly affected by the school transition. Hypothesized relationships between target variables were partly supported with significant effects of social-emotional skills on connectedness and significant effects of peer connectedness on happiness for children reporting connectedness decreases. Additional analyses were conducted including all classrooms to compare the intervention's effectiveness across different implementation progress groups. Significant group differences were found, indicating that implementation aspects impact intervention outcomes. The findings indicate that universal, school-based social-emotional leaning programs are effective approaches to support peer relationships in the context of the school transition. However, more implementation support seems to be needed to ensure best-practice delivery and achieve maximal intervention effectiveness.

Facilitators and Barriers to the Implementation of a School-Based Intervention for Anxiety

School-based cognitive behavioural interventions for anxiety are found to be effective, but there is a lack of research on their implementation in real world settings. The current study aims to explore the facilitators and barriers to the implementation of a school-based intervention for anxiety through a qualitative process evaluation. Evaluation of the implementation of Let's Introduce Anxiety Management (LIAM), a six-session school-based cognitive behavioural intervention, was conducted. LIAM was implemented by non-mental health professionals trained and coached on the model. Semi-structured interviews with stakeholders (N = 15) were analysed with grounded theory and framework analysis. Forty-one practitioners were trained and coached on LIAM, with thirty-five children and young people receiving the intervention. Facilitators (e.g. systemic collaboration, self-efficacy and an enabling context) and barriers (e.g. the exclusivity of the intervention and a lack of systemic understanding) to implementation emerged as themes. Implementing school-based interventions is complex and requires the involvement of multiple stakeholders.

Examining the impact of a universal social and emotional learning intervention (Passport) on internalising symptoms and other outcomes among children, compared to the usual school curriculum: study protocol for a school-based cluster randomised trial

BACKGROUND

School-based universal social and emotional learning (SEL) interventions implemented during the transition to adolescence may be efficacious in preventing the development of mental health difficulties. This protocol describes a two-arm parallel cluster randomised controlled trial to investigate the impact of a universal SEL intervention (Passport, compared to usual provision) on internalising symptoms (primary outcome), emotion regulation, well-being, loneliness, social support, bullying, academic attainment, and health-related quality of life in English primary school pupils aged 9-11 years. A developer-led trial demonstrated the feasibility, acceptability, and utility of Passport; this will be the first independent trial.

METHODS

Sixty primary schools will be recruited across the Greater Manchester city region and surrounding areas, involving 2400 pupils aged 8-9 at baseline. Schools will be allocated to the intervention arm to implement Passport over 18 weekly sessions or to the control arm to implement the usual school curriculum. Random allocation will be at school level following completion of baseline measures, with minimisation to ensure balance across trial arms in school size and free school meal eligibility. Measures will be collected at baseline, post-intervention (12 months post-baseline), and at 12 months follow-up (24 months post-baseline). The primary outcome analysis (intervention effects on internalising symptoms at post-intervention) will comprise a two-level (school, child) hierarchical linear model, following the intention-to-treat principle. Additional analyses will be undertaken to assess intervention effects on secondary outcomes, maintenance effects for all outcomes, intervention compliance moderator effects, subgroup moderator effects, and mechanisms underpinning intervention effects on the primary outcome. A mixed-methods implementation and process evaluation will examine factors that influence implementation, and a health economic evaluation will assess the cost-effectiveness of the intervention.

DISCUSSION

Findings will provide educators with crucial knowledge of whether and how increasing emotion regulation through a universal intervention impacts internalising symptoms and a range of related outcomes. Findings will also inform policy related to the promotion of mental health among children and young people. If the intervention is found to be efficacious in reducing internalising symptoms and is also cost-effective, it may offer high potential as a preventative intervention for widespread implementation.

TRIAL REGISTRATION

ISRCTN12875599; registered on 24 November 2022.

Generating Actionable Evidence for School-Based Mental Health Service Delivery: Public-Academic Partnership Based Evaluations

Public-academic partnership-based program evaluations can generate actionable evidence for policymaking, program design and implementation in improving school-based mental health service delivery. The University of Pennsylvania Center for Mental Health and public behavioral health care agencies in Philadelphia in the United States have evaluated Philadelphia's school mental health programs reimbursable through Medicaid billing since 2008. The variety of evaluations include (1) examining acute mental health service use of children receiving school-based mental health care and Medicaid expenditure, (2) examining children's externalizing and internalizing behaviors to measure school mental health providers' performance, and (3) examining effects of different types of school mental health programs on children's behavioral health functioning, school outcomes, and other out-of-school service use. This paper reports key findings of these evaluations, discusses how programs have been refined based on evaluation results, and shares lessons learned for successful public-academic partnership-based evaluations to promote use of actionable evidence.

Good behavior game - study protocol for a randomized controlled trial of a preventive behavior management program in a Swedish school context

Background

Early conduct problems and school failure are prominent risk factors for several adverse outcomes in later life. With the potential of reaching many children at early stages of their life, school-based interventions constitute a valuable approach to universal prevention. Good behavior game (GBG) is a promising school-based behavior management program, having shown immediate reductions in conduct problems along with several long-term positive effects. Adapting interventions to new contexts may however affect their effectiveness. The current study aims to evaluate the effectiveness of a Swedish adaption of GBG under pragmatic conditions. The intervention is hypothesized to reduce conduct problems in the classroom (primary outcome). Secondary analyses will investigate changes in conduct problems in common school areas, classroom climate, teacher collective efficacy, on-task behavior, as well as investigating behavioral management practices, implementation, and barriers to implementation.

Methods

This is a cluster-randomized trial with two parallel groups. Schools will be randomized (1,1, stratified by their areas sociodemographic index score) to be provided training in GBG or perform business-as-usual. The intervention and data collection lasts for a school year. Data will be collected at three time points: at baseline in the beginning of the school year (prior to training in GBG), after three months, and after nine months (at the end of the school year; primary endpoint). Data consists of teacher-rated measures of conduct problems, classroom climate, teacher collective efficacy, behavior management practices, and implementation factors, along with demographic factors. In addition, data will be collected by independent and blinded observers using corresponding measures in a subset of randomly chosen classrooms. Procedural fidelity will be rated and collected by GBG-trainers during nine observations throughout the school year. Statistical analysis will include frequentist intention-to-treat analysis, and comparisons of estimates with a corresponding Bayesian model using weakly informative priors. The study has currently completed data collection.

Discussion

This study will provide knowledge in universal prevention and school-based interventions with high reach, as well as specific knowledge concerning the effectiveness of an adapted version of GBG under real-world conditions, along with factors affecting its implementation and effects.

Clinical trial registration

ClinicalTrials.gov, identifier NCT05794893.

Implementation Science in School-Based, Universal-Level Intervention Research: A Scoping Review

PURPOSE

The aim of this study was to map the use of implementation science frameworks, models, and theories in intervention research targeting learning needs in the classroom.

METHOD

A scoping review was conducted. Electronic database and manual searches were conducted. Two reviewers independently completed screening, data extraction, and quality appraisal. Qualitative content analysis was undertaken using Nilsen's taxonomy and the domains of the Consolidated Framework for Implementation Research (CFIR). The data were further analyzed using the CFIR valence and strength rating scales.

RESULTS

Included papers (n = 22) used a diverse sample of implementation science frameworks and models. Most studies used determinant frameworks to guide data collection and analysis. Few studies used implementation science theory. Most studies were mixed methods (n = 11), published since 2019 (n = 20), and conducted in North America (n = 15). Over half of the interventions targeted social, emotional, and mental health (n = 13). A complex interplay of inner setting factors was identified as having a strong influence on implementation. Teachers' knowledge and beliefs, adaptability, and complexity of interventions were also identified as important considerations when conducting research in this context.

CONCLUSIONS

Early engagement with stakeholders in education is recommended when designing universal level speech, language, and communication interventions for use in the ordinary classroom. Adaptive, hybrid designs that test both implementation strategies and effectiveness of interventions may be warranted given the influence of inner setting factors on implementation outcomes.

"On an island by myself": implications for the inclusion of autistic students in self-contained classrooms in public elementary schools

Introduction

Autistic students have limited access to inclusive classes and activities in their schools. Principals and special education teachers who directly teach and administer programs for autistic elementary students can offer critical insight into factors, such as educators' attitudes, that may impact inclusive opportunities in schools. These attitudes may serve as barriers to or facilitators of promoting an inclusive school setting.

Methods

Semi-structured interviews with 26 elementary school principals and 26 special education teachers explored their experiences implementing evidence-based practices for autistic students (pivotal response training, discrete trial training, and visual schedules) in 26 self-contained classrooms in the United States. Autism-specific culture and inclusion emerged as a theme, which was analyzed for this paper.

Results

An inductive approach to thematic analysis revealed principals' and special education teachers' perspectives regarding the "autism-specific culture" in the school, including attitudes towards and inclusion of autistic students in self-contained classrooms in the broader school environment. Analysis of text related to "autism-specific culture" detailed aspects of inclusion, factors (i.e., barriers and facilitators) affecting inclusion, principals' and special education teachers' attitudes towards autistic students placed in self-contained classrooms, attitudes of other school staff towards teachers in self-contained classrooms, and recommendations to support an inclusive school environment for autistic students.

Discussion

Results suggest that valuing "equal" access to classes and activities for autistic students in self-contained classrooms may not be sufficient for promoting an inclusive school environment, Educators may benefit from targeted strategies to facilitate inclusion. Strategies range from supporting educators' attitudes and knowledge of autism to shifting physical aspects of the school environment (e.g., location of classrooms). Additional implications for supporting the true inclusion (i.e., inclusion that goes beyond physical inclusion) involves of autistic students in self-contained classrooms schools are discussed.

Integrity over fidelity: transformational lessons from youth participatory action research to nurture SEL with adolescents

Much has been written about social and emotional learning (SEL) and its positive impact on young people's academic and life outcomes, yet most of this research is based in early childhood and elementary settings. SEL programming for adolescents has shown mixed results, with many programs proving to be largely ineffective or even showing slightly negative impacts for some youth. Adherence to scripted SEL curricula, or "fidelity" to the program components, is often seen by young people to be "lame", inauthentic, and condescending, failing to connect to the topics and issues that feel most critical to them in this stage of their development. For all students, and especially for those whose identities have been systematically marginalized or oppressed by the dominant culture, SEL programming that fails to explicitly address these experiences of injustice often feels inauthentic and out of touch for youth. Therefore, effective implementation of SEL for adolescents is likely to require skillful adaptation and responsiveness to the identities, interests, and motivations of students by educators. In this case, effective SEL may look less like fidelity to a specific set of scripts, sessions, or activities, but rather a commitment to the wholeness of a set of core principles, relationships, and opportunities for adolescent exploration and leadership/empowerment, or what we will call integrity of implementation. In this paper, we present one promising approach to adolescent social and emotional development - youth participatory action research (YPAR) - and the ways in which studying the YPAR process (in addition to the research topics selected by youth) can provide key insights into the social and emotional learning and development of youth.

Exploring the acceptability, feasibility, and appropriateness of a communication-friendly classroom tool for use in Irish schools: A qualitative inquiry

BACKGROUND

Ten percent of the school-aged population have speech, language, and communication needs (SLCN) that impact access to the curriculum. Successful implementation of classroom-based SLCN interventions can reduce barriers to learning, thereby improving educational outcomes for this vulnerable population. The challenges of implementing innovations in educational settings are well-documented, yet limited studies have addressed such considerations when developing, and piloting universal level SLCN interventions for use in Irish schools.

METHODS

A qualitative exploratory study was undertaken to establish the acceptability, feasibility, and appropriateness of a universal level SLCN intervention. An advisory panel of teachers (n = 8) and children with SLCN (n = 2) were engaged as co-researchers in the study. The Communication Supporting Classrooms Observation Tool, developed as part of the Better Communication Project in the UK, was trialled across a diverse sample of school settings (n = 5). Semi-structured interviews were conducted with school practitioners and school leaders, and a deductive content analysis was undertaken using the domains of the Consolidation Framework for Implementation Research.

DISCUSSION

The observation tool was viewed as acceptable with suggested additions. Integrating use of the tool within existing data-informed, school self-evaluation processes aimed at supporting school improvement was noted as a potential means of supporting implementation. A knowledge gap in relation to school-based models of support for SLCN was identified which may negatively impact implementation. An implementation strategy targeting coherence, cognitive engagement and contextual integration is indicated if the tool is to be normalised into routine practice in Irish classrooms. Implementation needs appeared to vary at the school level.

CONCLUSIONS

The importance of early-stage exploration to guide implementation planning with regards to developing and testing universal level interventions for SLCN in schools is highlighted. Engaging an advisory panel provides important insights to guide implementation decisions. Findings suggest an adaptive design is required when planning implementation studies targeting classroom setting.

Aligning Organizational Priorities and System Policies to Support Implementation Scale-Up of a Tailored Classroom-Based Physical Activity Intervention in Low-Resource Schools

BACKGROUND

A mismatch between organizational priorities and system-level policies can negatively impact implementation and sustainment of classroom-based physical activity (PA) interventions. The purpose of this study was twofold: (1) present methods to systematically identify organization- and system-level implementation barriers, and (2) align organizational priorities and system policies by designing multi-level implementation strategies. This alignment will support implementation scale-up of a tailored PA intervention in one low-resource intermediate school district (ISD; 16 districts, 32 schools) in central Michigan.

METHODS

Multi-level assessments of organizational readiness were conducted using the Hexagon Discussion and Analysis Tool to assess intervention-context fit, the Wellness School Assessment Tool 3.0 to evaluate district PA policy strength and comprehensiveness, and semi-structured interviews were conducted to assess administrative support and priorities related to PA programming.

RESULTS

Our assessments revealed three implementation barriers: limited structural capacity to sustain teacher training, limited resources across districts and school buildings to support teachers, and misalignment of ISD and district PA policies and priorities.

CONCLUSIONS

Greater attention to organizational capacity and existing infrastructure should be considered a priori to support effective implementation and sustainment of PA interventions in low-resource schools.

Individual Responses versus Aggregate Group-Level Results: Examining the Strength of Evidence for Growth Mindset Interventions on Academic Performance

Mindset theory assumes that students' beliefs about their intelligence-whether these are fixed or can grow-affects students' academic performance. Based on this assumption, mindset theorists have developed growth mindset interventions to teach students that their intelligence or another attribute can be developed, with the goal of improving academic outcomes. Though many papers have reported benefits from growth mindset interventions, others have reported no effects or even detrimental effects. Recently, proponents of mindset theory have called for a "heterogeneity revolution" to understand when growth mindset interventions are effective and when-and for whom-they are not. We sought to examine the whole picture of heterogeneity of treatment effects, including benefits, lack of impacts, and potential detriments of growth mindset interventions on academic performance. We used a recently proposed approach that considers persons as effect sizes; this approach can reveal individual-level heterogeneity often lost in aggregate data analyses. Across three papers, we find that this approach reveals substantial individual-level heterogeneity unobservable at the group level, with many students and teachers exhibiting mindset and performance outcomes that run counter to the authors' claims. Understanding and reporting heterogeneity, including benefits, null effects, and detriments, will lead to better guidance for educators and policymakers considering the role of growth mindset interventions in schools.

Positive Attitude Upper Middle School social and emotional learning program: influences of implementation quality on program outcome

Introduction

There is an increased call for studies analyzing how implementation quality influences Social and Emotional Learning (SEL) program effectiveness.

Methods

The current dissemination study analyzed the effectiveness of the Positive Attitude Upper Middle School SEL program on a Portuguese nationwide sample composed of 813 middle school students (7th and 8th grade; 51.7% boys; M_age = 12.41, S.D. = 1.06), from 36 classrooms (M_classroom = 22.58; S.D. = 2.86), distributed between the control group (179 students), and three intervention groups (643 students) that reflected low, middle, and high implementer experience (respectively, Gulbenkian Academies of Knowledge, Positive Attitude Cadaval and Positive Attitude Torres Vedras). Dosage and fidelity (as implementation quality dimensions), gender, and classroom size (as individual and classroom-level variables) were also analyzed. Self-report questionnaires were administered pre- and post-intervention and at a 6-month follow-up.

Results

Multilevel models were employed, and results showed that participating in the PAUMS SEL program led to more positive trajectories in self-control, social awareness, relationship skills, and responsible decision-making when compared with control groups. Regarding implementation quality, only the implementer's experience impacted the effectiveness of the PAUMS SEL program; students in the Gulbenkian Academies of Knowledge intervention group displayed a less positive trajectory in self-control than students in the Positive Attitude Torres Vedras intervention group.

Discussion

Altogether, results showed that the PAUMS SEL program is ready for dissemination in Portugal, although a higher level of implementer experience is needed to achieve the best effectiveness, and they support the importance of analyzing implementer experience in SEL programs' effectiveness studies.

From a teacher and school leader perspective: What happened with smoking rules and practices during a three-year smoking preventive intervention? - Findings from the X:IT II study

BACKGROUND

School-based smoking preventive interventions are most effective if they consist of multiple components; one of them being strict anti-smoking rules, i.e., no smoking by anyone at any location during school hours. However, there is a lack of in-depth knowledge about how smoking practices and rules about smoking actually change over time. Therefore, the purpose of this study was to examine how smoking practices changed during a three-year period in which a smoking preventive intervention with strict anti-smoking rules was implemented at schools.

METHODS

We used data from 46 Danish schools that were enrolled in the evaluation of the X:IT II study - a smoking preventive intervention with three main components: one of them being strict anti-smoking rules. We collected questionnaire data among school employees at four timepoints from the autumn of 2017 until April 2020. The questionnaire covered topics regarding students' and employees' tobacco patterns at the school, control with smoking, and aspects of the intervention delivery.

RESULTS

Over time, student smoking rules got stricter; from three out of four at baseline to almost all schools having rules about no student smoking during school hours three years later. Employee smoking rules also changed, although not as much as student rules. Overall, smoking at school grounds seemed to decline - however, student smoking at other locations increased, hence, violating the rule about no smoking during school hours. Enforcement of smoking rules also increased over time.

CONCLUSION

Although implementing and enforcing new rules in a school setting may be difficult, it seemed that most schools in the X:IT II study changed their rules and smoking practices for both students and employees over the three-year intervention period. It seemed, however, that students relocated their smoking to other places than the school or just outside school grounds. Implications of these findings are important to consider in future interventions, i.e., students leaving school during school hours to smoke and the physical separation between those who smoke and those who do not.

Predicting implementation of the PAX Good Behavior Game + MyTeachingPartner interventions

Introduction

Effective classroom management is critical to creating a classroom environment in which social, emotional, and academic learning can take place. The present study investigated the association between early career, early elementary teachers’ occupational health (job stress, burnout, and perceived teaching ability) and perceptions of program feasibility in relation to their implementation dosage and quality of two evidence-based classroom management programs implemented together: the PAX Good Behavior Game (GBG) and MyTeachingPartner (MTP) intervention.

Methods

Teachers provided information on their occupational health at the start of the school year and were then randomized to the PAX GBG + MTP condition or control condition. Teachers’ perceptions of the feasibility of the program, implementation dosage, and implementation quality of the intervention were measured at the end of the school year for the 94 intervention teachers.

Results

Teachers participated in more MTP coaching cycles when they reported that the combined PAX GBG + MTP program was feasible. Although there were no main effects of occupational health on implementation, the associations between job stress and implementation quality were moderated by perceptions of feasibility.

Discussion

Findings highlight the complexity of factors influencing the implementation of evidence-based programs in school settings.

The Implementation Playbook: study protocol for the development and feasibility evaluation of a digital tool for effective implementation of evidence-based innovations

BACKGROUND

Evidence-based innovations can improve health outcomes, but only if successfully implemented. Implementation can be complex, highly susceptible to failure, costly and resource intensive. Internationally, there is an urgent need to improve the implementation of effective innovations. Successful implementation is best guided by implementation science, but organizations lack implementation know-how and have difficulty applying it. Implementation support is typically shared in static, non-interactive, overly academic guides and is rarely evaluated. In-person implementation facilitation is often soft-funded, costly, and scarce. This study seeks to improve effective implementation by (1) developing a first-in-kind digital tool to guide pragmatic, empirically based and self-directed implementation planning in real-time; and (2) exploring the tool's feasibility in six health organizations implementing different innovations.

METHODS

Ideation emerged from a paper-based resource, The Implementation Game©, and a revision called The Implementation Roadmap©; both integrate core implementation components from evidence, models and frameworks to guide structured, explicit, and pragmatic planning. Prior funding also generated user personas and high-level product requirements. This study will design, develop, and evaluate the feasibility of a digital tool called The Implementation Playbook©. In Phase 1, user-centred design and usability testing will inform tool content, visual interface, and functions to produce a minimum viable product. Phase 2 will explore the Playbook's feasibility in six purposefully selected health organizations sampled for maximum variation. Organizations will use the Playbook for up to 24 months to implement an innovation of their choosing. Mixed methods will gather: (i) field notes from implementation team check-in meetings; (ii) interviews with implementation teams about their experience using the tool; (iii) user free-form content entered into the tool as teams work through implementation planning; (iv) Organizational Readiness for Implementing Change questionnaire; (v) System Usability Scale; and (vi) tool metrics on how users progressed through activities and the time required to do so.

DISCUSSION

Effective implementation of evidence-based innovations is essential for optimal health. We seek to develop a prototype digital tool and demonstrate its feasibility and usefulness across organizations implementing different innovations. This technology could fill a significant need globally, be highly scalable, and potentially valid for diverse organizations implementing various innovations.

Exploring Teachers' Intentions to Use Behavioral Classroom Interventions

Multi-tiered systems of behavioral supports offer teachers tools to implement positive, antecedent- or consequence-based interventions for all students (i.e., Tier 1), and for those who need additional support (i.e., Tier 2), such as students with ADHD. Because these interventions may be challenging to use, targeted, theory-driven implementation strategies may assist teachers in implementing them with fidelity. This exploratory study examined teachers' intended and self-reported use of specific Tier 1 and Tier 2 behavioral classroom interventions. Sixty-five K-8 teachers from five urban public schools completed an online survey about their intentions to use and self-reported use of four Tier 1 and Tier 2 behavioral classroom interventions. Teachers' intentions varied by intervention, with the weakest intentions for using a daily behavior report (Tier 2), and weaker intentions for using high rates of specific praise than for other Tier 1 interventions. Teacher's self-reported use was significantly lower than intended use for Tier 1 interventions, but not Tier 2 interventions. Results were generally similar whether the referent group was students with ADHD symptoms or the entire class. These results suggest specific factors to target to support teachers' use of behavioral classroom interventions.

Health-Promoting School Culture: How Do We Measure it and Does it Vary by School Neighborhood Deprivation?

BACKGROUND

The context in which school-based health-promoting interventions are implemented is key for the delivery and success of these interventions. However, little is known about whether school culture differs by school deprivation.

METHODS

Using data from PromeSS, a cross-sectional study of 161 elementary schools in Québec, Canada, we drew from the Health Promoting Schools theoretical framework to develop four measures of health-promoting school culture (i.e., school physical environment, school/teacher commitment to student health, parent/community engagement with the school, ease of principal leadership) using exploratory factor analysis. One-way ANOVA with post-hoc Tukey-Kramer analyses was used to examine associations between each measure and social and material deprivation in the school neighborhood.

RESULTS

Factor loadings supported the content of the school culture measures and Cronbach's alpha indicated good reliability (range: 0.68-0.77). As social deprivation in the school neighborhood increased, scores for both school/teacher commitment to student health and parent/community engagement with the school decreased.

IMPLICATIONS FOR SCHOOL HEALTH POLICY, PRACTICE, AND EQUITY

Implementation of health-promoting interventions in schools located in socially deprived neighborhoods may require adapted strategies to address challenges related to staff commitment and parental and community involvement.

CONCLUSION

The measures developed herein can be used to investigate school culture and interventions for health equity.

Strategies for Enhancing the Implementation of Universal Mental Health Prevention Programs in Schools: A Systematic Review

A number of school-based mental health prevention programs have been found to be effective in research trials, but little is known about how to support implementation in real-life settings. To address this translational problem, this systematic review aims to identify effective strategies for enhancing the implementation of mental health prevention programs for children in schools. Four electronic databases were searched for empirical, peer-reviewed articles in English from January 2000 to October 2021 reporting the effects of implementation strategies for school-based universal mental health programs. Twenty-one articles were included in the narrative synthesis and assessed for quality using the Mixed Methods Appraisal Tool. Twenty-two strategies were found to be effective at improving program fidelity or adoption. The strategies with the strongest positive evidence base were those that involved monitoring and provision of feedback, engaging principals as program leaders, improving teachers' buy-in and organising school personnel implementation meetings. We recommend school-based practitioners trial strategies with positive findings from this review as part of their continuous quality improvement. This review highlights the pressing need for large-scale, randomised controlled trials to develop and trial more robust strategies to enhance adoption, as the five implementation studies found to measure adoption used qualitative methods limited by small samples sizes and case study designs.

Mixed methods implementation evaluation of virtual interview training for transition-age autistic youth in pre-employment transition services

BACKGROUND: Autistic transition-age youth are employed at rates far lower than their non-disabled peers as well as youth with other disabilities. Meanwhile, very few studies have evaluated the implementation of job interviewing practices within pre-employment transition services. OBJECTIVE: We conducted an initial implementation evaluation as part of a Hybrid Type 1 randomized controlled effectiveness-implementation trial where we trained teachers to deliver Virtual Interview Training for Transition-Age Youth (VIT-TAY) within five pre-employment transition services programs. METHOD: We used mixed methods to evaluate leader (n = 5), teacher (n = 15) and autistic transition age youth (n = 48) perceptions of VIT-TAY. We used descriptive statistics and thematic network analysis to evaluate survey data. Mixed methods integration was then performed to make comparisons between quantitative and qualitative results. RESULTS: Quantitative survey data revealed that leaders and teachers found VIT-TAY to be highly acceptable and appropriate for pre-employment transition services; findings which were confirmed via thematic network analysis of qualitative interview data. Autistic students reported via quantitative surveys that VIT-TAY was acceptable and usable, which was confirmed via thematic network analysis of open-ended survey data. CONCLUSION: This initial implementation evaluation can be used to inform a larger scale implementation evaluation of VIT-TAY in schools.

A Systematic Review of School-Based Suicide Prevention Interventions for Adolescents, and Intervention and Contextual Factors in Prevention

Suicide is the fourth leading cause of death among adolescents, globally. Though post-primary, school-based suicide prevention (PSSP) has the potential to be a key strategy for preventing adolescent suicidal thoughts and behaviours (STBs), there are persisting challenges to translating PSSP research to practice. Intervention and contextual factors relevant to PSSP are likely key to both PSSP effectiveness and implementation. As such, this systematic review aimed to summarise the effectiveness of PSSP for adolescent STBs and highlight important intervention and contextual factors with respect to PSSP. PsycINFO, Medline, Education Source, ERIC, Web of Science, and the Cochrane Central Register of Controlled Trials were searched to identify randomised and non-randomised studies evaluating the effectiveness of interventions located in post-primary, school-based settings targeting adolescent STBs. PSSP effectiveness and intervention and contextual factors were synthesised narratively. Twenty-eight studies were retained, containing nearly 47,000 participants. Twelve out of twenty-nine trials comparing intervention and independent control comparators reported statistically significant reductions in STBs postintervention, and 5/7 trials comparing preintervention and postintervention scores demonstrated significant reductions in STBs over time. Reporting and analysis of intervention and contextual factors were lacking across studies, but PSSP effectiveness and intervention acceptability varied across type of school. Although school personnel commonly delivered PSSP interventions, their input and perspectives on PSSP interventions were lacking. Notably, adolescents had little involvement in designing, inputting on, delivering and sharing their perspectives on PSSP interventions. Twenty out of twenty-eight studies were rated as moderate/high risk of bias, with non-randomised trials demonstrating greater risks of bias and trial effectiveness, in comparison to cluster randomised trials. Future research should prioritise complete reporting and analysis of intervention and contextual factors with respect to PSSP, involving key stakeholders (including adolescents and school personnel) in PSSP, and investigating key stakeholders' perspectives on PSSP. Given the inverse associations between both study quality and study design with PSSP effectiveness, particular consideration to study quality and design in PSSP research is needed. Future practice should consider PSSP interventions with universal components and PSSP which supports and involves key stakeholders in engaging with PSSP.

Therapist and supervisor perspectives about two train-the-trainer implementation strategies in schools: A qualitative study

Background

Train-the-trainer (TT) implementation strategies (in which designated clinicians are trained to then train others in an intervention) are promising approaches to support mental health clinician use of evidence-based interventions in school contexts. However, there is little evidence to date examining clinicians' perceptions of the acceptability and feasibility of TT strategies, or comparing clinicians' perceptions of different types of TT strategies.

Methods

The current study was conducted as part of a larger hybrid effectiveness-implementation trial, in which school-based therapists and supervisors received one of two different types of implementation support to implement cognitive behavioral therapy (CBT) groups for anxiety: TT (i.e., initial training for therapists and supervisors) or enhanced TT (TT+; i.e., initial training for therapists and supervisors, and ongoing external consultation for supervisors). We used applied thematic analysis to compare qualitative interview transcripts from 28 therapist interviews and 33 supervisor interviews from therapists and supervisors who received TT or TT+ support and report themes that were similar and different across the two groups.

Results

Most themes were similar across the TT and TT+ conditions: therapists and supervisors in both conditions perceived the group anxiety intervention as acceptable and viewed supervision as acceptable, helpful, and feasible. Therapists and supervisors in both conditions had mixed impressions of the contextual appropriateness of the group anxiety intervention, and some reported logistical challenges with weekly supervision. Some unique themes were identified among the TT+ condition, including supervisors experiencing professional growth, and therapists and supervisors perceiving supervision as critically important and enjoyable.

Conclusions

These results suggest that TT implementation support, using a model in which an internal supervisor receives initial training and then provides ongoing supervision, is acceptable and feasible to support a group CBT intervention in schools. The results also highlight additional benefits that therapists and supervisors perceived when supervisors received ongoing consultation.

Clinical Trial Registration Information

The clinical trial from which these data were derived was registered at ClinicalTrials.gov (https://clinicaltrials.gov/) prior to the time of first patient enrollment. The registration number is: NCT02651402.

Barriers and facilitators to the adoption of physical activity policies in elementary schools from the perspective of principals: An application of the consolidated framework for implementation research-A cross-sectional study

Background

Studies have shown that policies to promote physical activity in schools can have a positive impact on children's physical activity behavior. However, a large research gap exists as to what determinants may influence the adoption of such policies. Applying the Consolidated Framework for Implementation Research (CFIR), we investigated barriers and facilitators to the adoption of physical activity policies in elementary schools in Baden-Wuerttemberg, Germany, from the perspective of school principals.

Methods

A cross-sectional study was conducted between May and June 2021. School principals from elementary and special needs schools (n = 2,838) were invited to participate in the study. The online questionnaire used was developed based on the CFIR and included questions on school characteristics and constructs of the CFIR domains inner setting, characteristics of individuals, and process. Logistic regression analyses were performed to examine associations between policy adoption and school characteristics as well as CFIR determinants.

Results

In total, 121 schools (4%) participated in the survey, of which 49 (40.5%) reported having adopted a policy to promote physical activity. Positive associations with policy adoption were found for general willingness among teaching staff [odds ratio (OR): 5.37, 95% confidence interval (CI): 1.92-15.05], available resources (OR: 2.15, 95% CI: 1.18-3.91), access to knowledge and information (OR: 2.11, 95% CI: 1.09-4.09), and stakeholder engagement (OR: 3.47, 95% CI: 1.24-9.75).

Conclusions

This study provides a first insight into potential barriers and facilitators at the organizational level of schools that may be relevant to the adoption of physical activity policies, from the perspective of school principals. However, due to a low response rate, the results must be interpreted with caution. A strength of this study includes theoretical foundation through the use of the CFIR. The CFIR could be well-adapted to the school setting and provided valuable support for developing the questionnaire and interpreting the study results.

Adapting CBT for youth anxiety: Flexibility, within fidelity, in different settings

Anxiety disorders are common in youth, associated with impairments in daily functioning, and often persist into adulthood when untreated. Cognitive behavioral therapy (CBT) for youth anxiety is a well-established intervention and has been modified to fit several treatment settings. Despite decades of results supporting the efficacy of CBT, there is a large gap in access to this treatment and a need to consider how it can best be administered flexibly to increase uptake and personalization. We first discuss the core components of treatment for CBT through the lens of the Coping Cat treatment. Next, we review the empirical findings regarding adjustments made for CBT for youth anxiety delivered (a) in schools, (b) in community settings, (c) through telehealth, (d) through online computer programs, and (e) by caregivers at home. In each setting, we provide specific suggestions for how to implement CBT with flexibility while maintaining fidelity.

High quality implementation of 4Rs + MTP increases classroom emotional support and reduces absenteeism

School-based social and emotional learning (SEL) programs are associated with improvements in children's SEL and academic outcomes, and the quality of classroom interactions. The magnitude of these effects increases at high levels of program implementation quality. This study aimed to (1) identify teachers' profiles of quality of implementation, (2) explore teachers and classroom characteristics contributing to their propensity to comply with high quality of implementation, and (3) examine the relations between school assignment to an SEL program, quality of classroom interactions, and child SEL and academic outcomes at different levels of teachers' compliance propensity. This study drew upon data from a cluster-randomized controlled trial evaluating the efficacy of 4Rs + MTP, a literacy-based SEL program, on third and fourth grade teachers (n = 330) and their students (n = 5,081) across 60 New York City public elementary schools. Latent profile analysis indicated that measures of teacher responsiveness and amount of exposure to implementation supports contributed to the differentiation of profiles of high and low quality of implementation. Random forest analysis showed that more experienced teachers with low levels of professional burnout had high propensity to comply with high quality of implementation. Multilevel moderated mediation analysis indicated that 4Rs + MTP teachers with high compliance propensity were associated with higher classroom emotional support and lower children's school absences than their counterparts in the control group. These findings may inform debates in policy research about the importance of providing the supports teachers need to implement SEL school programs with high quality.

Identifying Teacher Beliefs and Experiences Associated with Curriculum Implementation Fidelity in Early Childhood Education

Implementation science research points to the importance of improving implementation fidelity to improve outcomes and sustainability of interventions. Despite our growing understanding of factors related to implementation in K-12 settings, much less is known about factors influencing implementation in early childhood education contexts. Understanding factors related to how well early childhood educators implement an intervention is critical to developing ways to improve implementation fidelity and ultimately education quality. The current study explored how teacher beliefs and experiences were related to initial uptake and later implementation in a sample of 87 early childhood educators implementing a novel comprehensive curriculum, STREAMin³. Across teacher dosage, classroom dosage, and teacher responsiveness, teachers with more positive initial perceptions of the curriculum had higher implementation. Teacher stress and perception of center climate were inconsistently related to implementation. Public preschool teachers and teachers with fewer years of teaching experience also reported higher levels of implementation. Implications for supporting teachers to improve implementation fidelity are discussed.

Professional Development Targeting Classroom Management and Behavioral Support Skills in Early Childhood Settings: A Systematic Review

Early childhood educators are expected to provide the children in their centers high-quality care and preparation for later school success. At the same time, nearly a third of children enter early care and education settings displaying challenging behaviors, which in turn impacts educators' stress levels and wellbeing. It is then unsurprising that classroom management and providing behavioral supports are consistently identified as areas where educators require further training upon entering the workforce. The purpose of this study is to conduct a systematic review of the empirical literature on professional development (PD) approaches targeting these areas for early childhood professionals. Forty-two studies were identified as meeting inclusion criteria and were coded for strategies targeted, the context and characteristics of the PD series, and the research design and outcomes utilized. Findings revealed that studies were highly variable in terms of targeted strategies, format of administration, training dose, research design, educator and child samples, and reporting practices. The majority of studies were conducted with educators from Head Start and public preschools and utilized research staff in the administration of the PD approaches. This indicates a need for more high-quality empirical evidence on PD approaches that cater to the larger early childhood workforce and centers' needs. Implications for researchers and early childhood mental health professionals and administrators are discussed.

Supplementary Information

The online version contains supplementary material available at 10.1007/s12310-022-09562-x.

Sustaining BEST in CLASS: Teacher-Reported Evidence-Based Practice Use with Students at Risk for Emotional and Behavioral Disorders Amidst the COVID-19 Pandemic

There is growing evidence of the efficacy of evidence-based interventions in improving the academic and social outcomes of children who exhibit challenging behaviors during program implementation periods. However, less is known about the extent to which practices learned as part of these interventions are sustained after these projects end, when funding is paused temporarily, and in less-than-ideal conditions. This study used qualitative methods to investigate whether teachers previously trained in the BEST in CLASS-Elementary intervention continued to use the program's evidence-based practices with students 1-2 years after completing the program and in the context of the COVID-19 pandemic. It also examined teachers' perceptions of the impact of practice use on students' academic and social outcomes. Thirteen BEST in CLASS-Elementary teachers from elementary schools in two southeastern states in the USA where the program was implemented completed semi-structured interviews on the topic. Data were coded thematically, and the results indicated that over 50% of teachers reported using "rules," "supportive relationships," and "praise" frequently with their students. However, "precorrection" and "opportunities to respond" were reportedly used less often. Teachers also perceived that their use of these evidence-based practices was linked to increases in their students' academic engagement and academic performance and knowledge, improvements in students' behaviors, their relationships with teachers, and general comfort and self-confidence. The discussion highlights modality-specific patterns noted in the results that might influence sustainment and the implication of these findings for interventions and programs aimed at promoting positive behavioral outcomes for early elementary school students.

Predicting school readiness program implementation in community-based childcare centers

Introduction

Targeted curricular interventions can increase preschool program quality and boost children's academic and social-emotional readiness skills, but variable funding and weak organizational infrastructure in many community-based childcare centers may reduce the effective implementation of these programs.

Method

This study examined individual teacher and workplace predictors of the REDI program implementation, a targeted school readiness program that was adapted to support delivery in childcare centers. REDI was delivered by 63 teachers in 37 community-based childcare centers with center directors serving as local implementation coaches.

Results

Results showed that individual teacher factors (e.g., teaching skills and receptivity to intervention consultation) predicted the quality with which REDI activities and teaching strategies were delivered, and workplace factors were important predictors across multiple implementation indicators.

Discussion

Practice and policy implications for improving intervention implementation and corresponding program quality in childcare centers are highlighted.

Teacher and school staff perspectives on their role in school-based vision programs

OBJECTIVE

School-based vision programs (SBVPs) are one approach to increase access to vision care by providing vision screenings, eye examinations, and eyeglasses directly in schools. Few studies report on the perspectives of teachers and staff, who are important stakeholders, on SBVPs. We examined teacher and staff perspectives on their involvement in SBVPs.

DESIGN

Qualitative study using focus groups.

PARTICIPANTS

Teachers and staff at Baltimore and Chicago public schools served by SBVPs between 2016 and 2018.

METHODS

We conducted 21 semistructured focus groups with 117 teachers and staff in 10 Baltimore and 11 Chicago public preK-12 schools that participated in SBVPs. Sessions were recorded, transcribed, and coded using inductive thematic analysis.

RESULTS

Participants identified 2 main themes regarding teacher and staff involvement in SBVPs: (i) program outreach, including using multiple communication modalities to engage parents, explaining program details to families, and helping with program consent form return and (ii) promoting vision health, including identifying vision problems in the classroom, encouraging eyeglasses wear, and supporting eyeglasses maintenance. Participants also discussed limitations in capacity to partake in these activities.

CONCLUSION

Teachers interact with parents and students throughout the SBVP process, undertaking important roles in outreach and health promotion to ensure uptake of SBVP services. SBVPs and other school-based health programs should explore strategies to support teachers in the roles they fill to optimize program impact.

Reaching consensus on definitions for food and physical activity policies: experience from the Policy Evaluation Network

BACKGROUND

An upsurge in policy evaluation research within public health sciences has led to multi-disciplinary research networks like the 'Policy Evaluation Network' (PEN). This multi-disciplinary collaboration highlighted the need for consensus on clear, common terminology and definitions to facilitate the multi-disciplinary research. This article outlines the development process of the PEN definitions glossary tool, with a focus on the key domains of policy design, implementation and outcomes as they apply to physical activity, sedentary behaviour and dietary behaviours.

METHODS

A project specific participatory process was undertaken, involving PEN researchers (n = 48) from seven European countries across various disciplinary backgrounds. All involved researchers were invited to identify and collate definitions that were commonly used in their research field. Terms and definitions were discussed and debated during three online workshops. Subsequently, the definitions were discussed and refined until consensus was reached.

RESULTS

Consensus definitions for 93 terms related to the evaluation of policy design, implementation and outcomes are provided. Consensus was reached on a range of terms where the terms were understood and used differently across represented disciplines (e.g. 'Outcome' and 'Impact'). A conceptual 'Inter-relations in policy-related concepts' diagram was developed to enable navigation through an online database with key terms.

CONCLUSIONS

The definitions resulting from this participatory process has supported PEN researchers and practitioners across disciplines to reach a shared understanding of different terms related to policy evaluation. Thus, providing a platform for avoiding conflicting use of the same terms in differing contexts over the course of the PEN work programme, facilitating clear and consistent communication, and allowing for clarity within collaborative multi-disciplinary projects and in public-facing messages.

Monitoring system of implementation of the Promoting Mental Health at Schools (PROMEHS) program

Effective school-based mental health programs are a research field with growing interest and great social value. At the stage of development and initial testing of the program, as well as during dissemination, and adaptation in other cultures, it is important that the implementation is carried out in the way that was originally intended. Fidelity or adherence is the most often used concept relating to the extent to which the implemented intervention corresponds to the originally intended program. Therefore, monitoring of the implementation is an essential element necessary to integrate into contemporary evidence-based program. The current paper describes the monitoring system developed for the Promoting Mental Health at Schools (PROMEHS) project. The monitoring was done on both the structural and procedural aspects of the program's implementation, involving the evaluation of five core aspects: fidelity, dosage, quality, responsiveness, and adaptation. This methods article aims to describe the development of the monitoring system and to analyze the strengths of the qualitative-quantitative multi-informant approach in the monitoring of the intervention's implementation. In the future, this would support further research on effectiveness of the PROMEHS program.

Sustainment of an Evidence-Based, Behavioral Health Curriculum in Schools

The development of evidence-based interventions (EBIs) for the prevention of behavioral health problems is well supported. However, limited data exist on the sustainability of EBIs once initial support has ceased. The current study assessed, at 2 years after initial start-up support: (1) What percent of schools sustained EBI implementation? (2) To what degree did sustaining schools implement the EBI with fidelity? (3) What were the primary reasons for not sustaining the EBI? (4) What theoretical and contextual factors during initial start-up support predicted sustainment of the EBI? The study used process evaluation data from the dissemination of the Botvin LifeSkills Training (LST) middle school program in 158 school districts (including 419 schools). Fifty-one percent of districts sustained the EBI and most of these sustaining districts reported following key fidelity guidelines. Primary reasons for discontinuing centered around low teacher or administrative support and turnover. Significant predictors of sustaining LST were higher ratings of LST's complexity, benefit, and compatibility by teachers; more positive perceptions of organizational support from administrators; and smaller proportions of Black students. Results show that EBI sustainment and fidelity of implementation post-initial startup support are feasible, though likely not for all schools. Findings also suggest that cultivating support for the EBI among staff during start-up support may be helpful for sustainment and that social determinants of a school have a complex relationship with EBI sustainment. Future research should explore true causes of differences due to race/ethnicity as well as COVID-19 effects.

Adolescent Substance Use Behavior Change Through School Intervention Is Improved by Teacher and School Implementation Support Together, Especially for Girls

An earlier trial of a school-based, preventative intervention, HealthWise South Africa, demonstrated some efficacy in preventing adolescent drinking and smoking in the Western Cape, South Africa. This was followed by the current implementation trial, which examined how implementation support conditions influenced adolescent drinking and smoking behavior change in the HealthWise intervention. The hybridized implementation-effectiveness trial included 34 schools (n = 2175 students) that provided student data at four waves through Grades 8 to 10. Implementation support conditions included a combination of two components (enhanced school environment and teacher consultation), resulting in four conditions: enhanced school environment, teacher consultation, both components, and standard delivery. Using configural frequency analysis, we examined how drinking and smoking behavior change differed across four configurations of intervention support longitudinally and by gender. For baseline non-smokers, results indicated sustained differences in abstention from smoking and smoking initiation between schools that received both support components and schools in the standard delivery condition. These results were primarily driven by girls. For baseline drinkers and smokers in the both components condition, changes in drinking were delayed until Grade 9 and smoking decayed by Grade 10. Results suggest that providing both school and teacher implementation support synergistically facilitates improved intervention outcomes both immediately following intervention and 2 years later across schools with diverse resource levels. Future iterations of HealthWise, and other interventions, may benefit from a multilevel implementation support strategy to improve student outcomes. More research is needed to understand how to improve HealthWise effectiveness among boys.

Developing a core outcome set for physical activity interventions in primary schools: a modified-Delphi study

OBJECTIVES

To develop a core outcome set (COS) for physical activity interventions in primary schools.

DESIGN

Modified-Delphi study.

SETTING

The UK and international.

PARTICIPANTS

104 participants from four stakeholder groups (educators, public health professionals, health researchers, parents); 16 children (aged 8-9 years) from 1 London primary school.

INTERVENTIONS

Physical activity interventions.

METHODS

Four-stage process: (1) outcomes extracted from relevant studies identified from an umbrella review and a focus group; (2) list of outcomes produced and domains established; (3) stakeholders completed a two-round Delphi survey by rating (Round 1) and re-rating (Round 2) each outcome on a nine-point Likert Scale from 'not important' to 'critical': a>70% participant threshold identified the outcomes rated 'critical' to measure, and outcomes important to children were identified through a workshop; and (4) a stakeholder meeting to achieve consensus of the outcomes to include in the COS.

RESULTS

In total, 74 studies were extracted from 53 reviews. A list of 50 outcomes was produced and three domains were established: 'physical activity and health' (16 outcomes), 'social and emotional health' (22 outcomes) and 'educational performance' (12 outcomes). 104 participants completed survey Round 1; 65 participants completed both rounds. In total, 13 outcomes met the threshold; children identified 8 outcomes. Fourteen outcomes achieved consensus to produce the COS: five outcomes for physical activity and health (diet (varied and balanced), energy, fitness, intensity of physical activity, sleep (number of hours)); seven outcomes for social and emotional health (anxiety, depression, enjoyment, happiness, self-esteem, stress, well-being); and two outcomes for educational performance (concentration, focus).

CONCLUSIONS

We have developed the first COS for physical activity interventions in primary schools in consultation with those interested in the development and application of an agreed standardised set of outcomes. Future studies including these outcomes will reduce heterogeneity across studies.

TRIAL REGISTRATION NUMBER

Core Outcome Measures in Effectiveness Trials Initiative registration number 1322; Results.

Is level of implementation linked with intervention outcomes? Process evaluation of the TransformUs intervention to increase children’s physical activity and reduce sedentary behaviour

Background

TransformUs was a four-arm school-based intervention to increase physical activity and reduce sedentary behaviour among primary school children. Pedagogical and environmental strategies targeted the classroom, school grounds and family setting. The aims of this study were to evaluate program fidelity, dose, appropriateness, satisfaction and sustainability, and associations between implementation level and outcomes among the three intervention arms.

Methods

At baseline, 18-months (mid-intervention) and 30-months (post-intervention), teachers, parents and children completed surveys, and children wore GT3X ActiGraph accelerometers for 8 days at each time point to determine physical activity and sedentary time. Implementation data were pooled across the three intervention groups and teachers were categorised by level of implementation: (i) ‘Low’ (< 33% delivered); (ii) ‘Moderate’ (33–67% delivered); and (iii) ‘High’ (> 67% delivered). Linear and logistic mixed models examined between group differences in implementation, and the association with children’s physical activity and sedentary time outcomes. Qualitative survey data were analysed thematically.

Results

Among intervention recipients, 52% (n = 85) of teachers, 29% (n = 331) of parents and 92% (n = 407) of children completed baseline evaluation surveys. At 18-months, teachers delivered on average 70% of the key messages, 65% set active/standing homework, 30% reported delivering > 1 standing lesson/day, and 56% delivered active breaks per day. The majority of teachers (96%) made activity/sports equipment available during recess and lunch, and also used this equipment in class (81%). Fidelity and dose of key messages and active homework reduced over time, whilst fidelity of standing lessons, active breaks and equipment use increased. TransformUs was deemed appropriate for the school setting and positively received. Implementation level and child behavioural outcomes were not associated. Integration of TransformUs into existing practices, children’s enjoyment, and teachers’ awareness of program benefits all facilitated delivery and sustainability.

Conclusions

This study demonstrated that intervention dose and fidelity increased over time, and that children’s enjoyment, senior school leadership and effective integration of interventions into school practices facilitated improved intervention delivery and sustainability. Teacher implementation level and child behavioural outcomes were unrelated, suggesting intervention efficacy was achieved irrespective of implementation variability. The potential translatability of TransformUs into practice contexts may therefore be increased. Findings have informed scale-up of TransformUs across Victoria, Australia.

Trial registration

International Standard Randomized Controlled Trial Number ISRCTN83725066; Australian New Zealand Clinical Trials Registry Number ACTRN12609000715279. Registered 19 August 2009. Available at: https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=308387&isReview=true

Supplementary Information

The online version contains supplementary material available at 10.1186/s12966-022-01354-5.

Understanding the Needs of Primary School Teachers in Supporting Their Students' Emotion Regulation

BACKGROUND

Accumulating evidence has underscored the importance of fostering children's emotion regulation (ER) within primary school settings and the role of teachers in such efforts. This study sought to assess the needs of teachers in supporting students' ER, through a better understanding of teachers' perceptions and use of healthy versus unhealthy ER strategies in the classroom.

METHODS

Primary school teachers (n = 212; 91% female) completed an online, researcher-developed needs assessment survey assessing their perceptions regarding the importance of ER instruction and challenges surrounding children's ER, as well as the perceived effectiveness and reported use of healthy and unhealthy ER strategies in the classroom.

RESULTS

Cochran's Q and chi-square analyses revealed misperceptions regarding the effectiveness of healthy and unhealthy strategies, as well as discrepancies between teachers' perceptions regarding the effectiveness of specific healthy strategies (eg, meditation) and their reported use of them.

CONCLUSIONS

While teachers recognize the growing importance of fostering ER in the classroom, the present findings suggest that there is a need for more professional development regarding the effectiveness and implementation of ER strategies in the primary school context. Efforts should be made to provide teachers with concrete recommendations for the implementation of ER strategies in the classroom.

Using readiness to understand implementation challenges in school mental health research

Schools and research partners are increasingly implementing complex, multicomponent interventions and school-wide frameworks to better meet students' social, emotional, behavioral, and academic needs; however, in the research and real-world contexts, implementation is often fraught with many challenges and barriers to success. This study explores implementation barriers encountered during a randomized controlled trial testing effects of one complex intervention strategy-the Interconnected Systems Framework-from the lens of a practical model for conceptualizing organizational readiness-the Interactive Systems Framework for Dissemination and Implementation. Implementation of the Interconnected Systems Framework was explored via focus group and key informant interviews with school and mental health professionals, and research team members responsible for implementing the intervention in randomly assigned study schools. Results from inductive thematic analysis of verbatim transcripts identified three primary implementation challenges: staff turnover, inadequate leadership buy-in, and insufficient time for training/planning. Each challenge is explored from interview participants' perspectives and the extant literature, then connected to recommendations from implementation science to help others avoid similar challenges in their well-intentioned efforts to address the mounting concern for students' wellbeing.

A Mixed-Method Study Examining Solutions to Common Barriers to Teachers' Adoption of Evidence-Based Classroom Practices

Objectives

We conducted a mixed-method focus group study to (a) assess the appropriateness and likely effectiveness of strategies that target individual behavior change mechanisms associated with perceived barriers of lack of time and unsupportive leadership and (b) identify recommendations regarding strategies for overcoming the barriers.

Method

Sample included 39 school-based staff (80% female, 77% White) across two districts in the Midwest. Mixed methods included a simultaneous approach.

Results

Lack of time and supportive leadership continue to pervade school-based implementation efforts. Recommendations centered around the need for school leaders to give teachers the power to re-prioritize how they spend their time as well as providing protected, facilitated time for teachers to collaborate and learn practical skills targeting self-advocacy.

Conclusion

Our findings provide compelling evidence for the use of implementation methodology to strategically target mechanisms of individual behavior change during the process of incorporating new and innovative practices in schools.

Correlates of perceived success of health-promoting interventions in elementary school

INTRODUCTION

School-based health-promoting interventions (HPIs) aim to support youth development and positively influence modifiable lifestyle behaviours. Identifying factors that contribute to or hinder the perceived success of HPIs could facilitate their adaptation, improve implementation and contribute to HPI sustainability. The objective of this study was to identify factors in three domains (school characteristics, characteristics of the HPI and factors related to planning and implementing the HPI) associated with perceived success of HPIs among school principals in elementary schools.

METHODS

Data were drawn from Project PromeSS, a cross-sectional survey of school principals and/or nominated staff members in a convenience sample of 171 public elementary schools in Quebec, Canada. School board and school recruitment spanned three academic school years (2016-2019). Data on school and participant characteristics, HPI characteristics, variables related to HPI planning and implementation and perceived success of the HPI were collected in two-part, structured telephone interviews. Descriptive statistics were used to characterize schools and study participants. Twenty-eight potential correlates of perceived HPI success were investigated separately in multivariable linear regression modelling.

RESULTS

Participants generally perceived HPIs as highly successful. After controlling for number of students, language of instruction, school neighbourhood and school deprivation, we identified five correlates of perceived success, including lower teacher turnover, higher scores for school physical environment, school/teacher commitment to student health, principal leadership and school being a developer (vs. adopter) of the HPI.

CONCLUSION

If replicated, these factors should be considered by HPI developers and school personnel when planning and implementing HPIs in elementary schools.