Understanding the supports needed for policy implementation: a comparative analysis of the placement of intermediaries across three mental health systems

Tackling the non-communicable disease epidemic: a framework for policy action in low- and middle-income countries

Health policy frameworks for the prevention and control of non-communicable diseases have largely been developed for application in high-income countries. Limited attention has been given to the policy exigencies in lower- and middle-income countries where the impacts of these conditions have been most severe, and further clarification of the policy requirements for effective prevention is needed. This paper presents a policy approach to prevention that, although relevant to high-income countries, recognizes the peculiar situation of low-and middle-income countries. Rather than a narrow emphasis on the implementation of piecemeal interventions, this paper encourages policymakers to utilize a framework of four embedded policy levels, namely health services, risk factors, environmental, and global policies. For a better understanding of the non-communicable disease challenge from a policy standpoint, it is proposed that a policy framework that recognizes responsible health services, addresses key risk factors, tackles underlying health determinants, and implements global non-communicable disease conventions, offers the best leverage for prevention.

Pilot evaluation of the Fiscal Mapping Process for sustainable financing of evidence-based youth mental health treatments: A comparative case study analysis

Background

Sustained delivery of evidence-based treatments (EBTs) is essential to addressing the public health impacts of youth mental health problems, but is complicated by the limited and fragmented funding available to youth mental health service agencies. Supports are needed that can guide service agencies in accessing sustainable funding for EBTs. We conducted a pilot evaluation of the Fiscal Mapping Process, an Excel-based strategic planning tool that helps service agency leaders identify and coordinate financing strategies for their EBT programs.

Method

Pilot testing of the Fiscal Mapping Process was completed with 10 youth mental health service agencies over a 12-month period, using trauma-focused cognitive-behavioral therapy or parent-child interaction therapy programs. Service agency representatives received initial training and monthly coaching in using the tool. We used case study methods to synthesize all available data (surveys, focus groups, coaching notes, document review) and contrast agency experiences to identify key findings through explanation building.

Results

Key evaluation findings related to the process and outcomes of using the Fiscal Mapping Process, as well as contextual influences. Process evaluation findings helped clarify the primary use case for the tool and identified the importance-and challenges-of engaging external collaborators. Outcome evaluation findings documented the impacts of the Fiscal Mapping Process on agency-reported sustainment capacities (strategic planning, funding stability), which fully explained reported improvements in outcomes (extent and likelihood)-although these impacts were incremental. Findings on contextual factors documented the influence of environmental and organizational capacities on engagement with the tool and concerns about equitable impacts, but also the view that the process could usefully generalize to other EBTs.

Conclusions

Our pilot evaluation of the Fiscal Mapping Process was promising. In future work, we plan to integrate the tool into EBT implementation initiatives and test its impact on long-term sustainment outcomes across various EBTs, while increasing attention to equity considerations.

Mental health services implementation in Colombia-A systematic review

Colombia's mental health services have a complex history shaped by 60 years of armed conflict, a predominantly clinical approach to mental health, and social factors such as inequities and stigma. The 1990 Caracas declaration proposed a shift towards decentralised community mental health services and interventions based on the recovery approach and emphasis on social determinants of mental health in the Americas. Colombia has adopted these approaches in its legal and practical framework in recent years, but implementation has been uneven. This systematic review aims to contribute to mental health services understanding in Colombia by examining the barriers and facilitators to the implementation of mental health services in Colombia. A search was conducted to explore available peer-reviewed studies on Colombian mental health services across five databases (Medline, PubMed, Scopus, Scielo and BVS) on quantitative and qualitative research papers published in the last ten years and without language restrictions. The Consolidated Framework for Implementation Research (CFIR) was used to structure the analysis and identify barriers and facilitators during the implementation of mental health services. We adapted the CFIR to attend to gender, race and age informed by the Socio-Political Economy of Global Mental Health framework, given the importance of these factors to the Colombian health landscape. Finally, narrative synthesis was used to summarise the data. 1 530 records were identified, and 12 articles met all inclusion criteria and were included in the analysis. 8 papers described substance use disorders services, 11 involved multidisciplinary healthcare professionals, and 7 were implemented at a local scale. The primary barriers to implementation were the lack of coordination, high workloads, and low funding. Facilitators included the use of protocols, and the involvement of communities, stakeholders, users, and external champions. Findings suggest the continued importance of community and recovery approaches and efforts to improve coordination between multi-sector actors involved in the mental health spaces (e.g., public, and private organisations, users and their families).

Implementation of Medicines Pricing Policies in Ghana: The Interplay of Policy Content, Actors' Participation, and Context

BACKGROUND

Implementing medicines pricing policy effectively is important for ensuring equitable access to essential medicines and ultimately achieving universal health coverage. However, published analyses of policy implementations are scarce from low- and middleincome countries. This paper contributes to bridging this knowledge gap by reporting analysis of implementation of two medicines pricing policies in Ghana: value-added tax (VAT) exemptions and framework contracting (FC) for selected medicines. We analysed implications of actor involvements, contexts, and contents on the implementation of these policies, and the interplay between these. This paper should be of interest, and relevance, to policy designers, implementers, the private sector and policy analysts.

METHODS

Data were collected through document reviews (n=18), in-depth interviews (n=30), focus groups (n=2) and consultative meetings (n=6) with purposefully identified policy actors. Data were analysed thematically, guided by the four components of the health policy triangle framework.

RESULTS

The nature and complexity of policy contents determined duration and degree of formality of implementation processes. For instance, in the FC policy, negotiating medicines prices and standardizing the tendering processes lengthened implementation. Highly varied stakeholder participation created avenues for decision-making and promoted inclusiveness, but also raised the need to manage different agendas and interests. Key contextual enablers and constraints to implementation included high political support and currency depreciation, respectively. The interrelatedness of policy content, actors, and context influenced the timeliness of policy implementations and achievement of intended outcomes, and suggest five attributes of effective policy implementation: (1) policy nature and complexity, (2) inclusiveness, (3) organizational feasibility, (4) economic feasibility, and (5) political will and leadership.

CONCLUSION

Varied contextual factors, active participation of stakeholders, nature, and complexity of policy content, and structures have all influenced the implementation of medicines pricing policies in Ghana.

Who's "in the room where it happens"? A taxonomy and five-step methodology for identifying and characterizing policy actors

BACKGROUND

Engaging policy actors in research design and execution is critical to increasing the practical relevance and real-world impact of policy-focused dissemination and implementation science. Identifying and selecting which policy actors to engage, particularly actors involved in "Big P" public policies such as laws, is distinct from traditional engaged research methods. This current study aimed to develop a transparent, structured method for iteratively identifying policy actors involved in key policy decisions-such as adopting evidence-based interventions at systems-scale-and to guide implementation study sampling and engagement approaches. A flexible policy actor taxonomy was developed to supplement existing methods and help identify policy developers, disseminators, implementers, enforcers, and influencers.

METHODS

A five-step methodology for identifying policy actors to potentially engage in policy dissemination and implementation research was developed. Leveraging a recent federal policy as a case study-The Family First Prevention Services Act (FFPSA)-publicly available documentation (e.g., websites, reports) were searched, retrieved, and coded using content analysis to characterize the organizations and individual policy actors in the "room" during policy decisions.

RESULTS

The five steps are as follows: (1) clarify the policy implementation phase(s) of interest, (2) identify relevant proverbial or actual policymaking "rooms," (3) identify and characterize organizations in the room, (4) identify and characterize policy actors in the "room," and (5) quantify (e.g., count actors across groups), summarize, and compare "rooms" to develop or select engagement approaches aligned with the "room" and actors. The use and outcomes of each step are exemplified through the FFPSA case study.

CONCLUSIONS

The pragmatic and transparent policy actor identification steps presented here can guide researchers' methods for continuous sampling and successful policy actor engagement. Future work should explore the utility of the proposed methods for guiding selection and tailoring of engagement and implementation strategies (e.g., research-policy actor partnerships) to improve both "Big P" and "little p" (administrative guidelines, procedures) policymaking and implementation in global contexts.

A qualitative analysis of factors influencing the implementation of antiretroviral treatment adherence policy in Ghana: stakeholders perspective

BACKGROUND

The Joint United Nations Programme on HIV/AIDS launched the 90-90-90 initiative. Failure to meet the target reflects the difficulties in successfully implementing HIV treatment policy. There are research gaps in exploring personal and external factors influencing HIV treatment in Ghana. To fill this gap, we explored individual and environmental (interpersonal, community and structural) factors influencing stakeholders' HIV treatment policy implementation in Ghana.

METHODS

Fifteen qualitative semi-structured in-depth interviews were conducted among representatives in different management positions at hospitals, health directorates, the Ghana AIDS Commission, the National AIDS and STI control program, and the National Association of People Living with HIV.

RESULTS

Using thematic analysis, the findings suggest that individual and environmental factors such as attitude towards policy, awareness of HIV treatment policy, training received on policy implementation, difficulties related to patient factors, alternate sources of HIV care, inefficient policy decision-making, monitoring and evaluation of HIV treatment policy, lack of HIV treatment policy implementation training, poor availability of logistics, policy and guidelines, infrastructure, organization of training, and staff availability may hinder successful HIV treatment policy implementation.

CONCLUSION

Several individual and environmental (interpersonal, community and structural) factors seem to influence HIV treatment policy implementation. To ensure successful policy implementation stakeholders need to receive training on new policies, availability of sufficient supplies of material resources, inclusive decision-making, receive supportive monitoring of policy implementation, and oversight.

Multi-level stakeholders' perspectives on implementation and scaling up community-based health promotion in Germany

Community-based health promotion has the potential to address existing health inequities, although such approaches are scarcely scaled up. For a successful scale up, various stakeholders at different levels and sectors need to be involved. The article's aims are to assess what kind of external support communities need for implementation and to identify facilitators and barriers for scaling up community-based health promotion. Two national digital workshops were conducted in Germany with stakeholders at the community level (n = 161) and with stakeholders at the federal and state levels (n = 84). Protocols were compiled and coded using qualitative content analysis. During the first workshop, we revealed 11 themes for external support needs ('Strategic approach', 'Define & compare indicators', 'International human resource', 'Tools & Aids', 'External conduction of the assessment', 'Involvement of people in difficult life situations', 'Overview of actors', 'Moderation', 'Obtain funding', 'Quality assurance/evaluation' and 'External support'). Eleven facilitators and barriers were identified for scaling up ('Assessment and evaluation', 'Intersectoral collaboration and partnerships', 'Communication', 'Characteristics of the program', 'Political and legal conditions', 'Political support', 'Local coordinator', 'Resources', 'Participation', 'Strategic planning/methods' and 'Intermediary organization'). The identified results provide practice-based evidence on support needed for scaling up, facilitators that promote scaling up and barriers that might hinder scaling up community-based health promotion in Germany. In a next step, this practice-based evidence needs to be systematically integrated with scientific-based evidence on key components for scaling up such approaches for the development of an effective scaling-up concept.

A landscape assessment of the activities and capacities of evidence-to-policy intermediaries (EPI) in behavioral health

BACKGROUND

A significant gap exists between the production of research evidence and its use in behavioral health policymaking. Organizations providing consulting and support activities for improving policy represent a promising source for strengthening the infrastructure to address this gap. Understanding the characteristics and activities of these evidence-to-policy intermediary (EPI) organizations can inform the development of capacity-building activities, leading to strengthened evidence-to-policy infrastructure and more widespread evidence-based policymaking.

METHODS

Online surveys were sent to 51 organizations from English-speaking countries involved in evidence-to-policy activities in behavioral health. The survey was grounded in a rapid evidence review of the academic literature regarding strategies used to influence research use in policymaking. The review identified 17 strategies, which were classified into four activity categories. We administered the surveys via Qualtrics and calculated the descriptive statistics, scales, and internal consistency statistics using R.

RESULTS

A total of 31 individuals completed the surveys from 27 organizations (53% response rate) in four English-speaking countries. EPIs were evenly split between university (49%) and non-university (51%) settings. Nearly all EPIs conducted direct program support (mean = 4.19/5 [sd = 1.25]) and knowledge-building (4.03 [1.17]) activities. However, engagement with traditionally marginalized and non-traditional partners (2.84 [1.39]) and development of evidence reviews using formal critical appraisal methods (2.81 [1.70]) were uncommon. EPIs tend to be specialized, focusing on a group of highly related strategies rather than incorporating multiple evidence-to-policy strategies in their portfolios. Inter-item consistency was moderate to high, with scale α's ranging from 0.67 to 0.85. Ratings of respondents' willingness to pay for training in one of three evidence dissemination strategies revealed high interest in program and policy design.

CONCLUSIONS

Our results suggest that evidence-to-policy strategies are frequently used by existing EPIs; however, organizations tend to specialize rather than engage in a breadth of strategies. Furthermore, few organizations reported consistently engaging with non-traditional or community partners. Focusing on building capacity for a network of new and existing EPIs could be a promising strategy for growing the infrastructure needed for evidence-informed behavioral health policymaking.

Towards a successful implementation of environmental sanitation policy in Ghana: An assessment of key impeding factors

Environmental sanitation policy implementation is a vital factor that ensures that citizens are kept healthy and productive. The study sought to assess key factors impeding the implementation of environmental sanitation policy in Ghana. Using an explanatory design, a sample of 384 respondents was selected from the population of Accra through a simple random sampling technique. The questionnaire was the main instrument used to collect the data. The Partial Least Square-Structural Equation Modeling (PLS-SEM) technique was used to analyse the hypothesised path models. Based on the results, government approach, community representation, and lack of citizens' commitment were found to be statistically significant. The study also revealed that the government approach partially mediated the relationship between community representation and environmental sanitation policy implementation, and lack of citizens' commitment and environmental sanitation policy implementation. The study has contributed to knowledge in the research endeavour by demonstrating that the implementation of public policy can easily be realised when the government uses the right approach in involving its citizens in policy decision-making and enhance their commitment to policy implementation.

What we know about the actual implementation process of public physical activity policies: results from a scoping review

BACKGROUND

Physical inactivity rates have remained high worldwide since 2001. Public policies are an essential upstream lever to target individual physical activity (PA) behaviour. However, implementers have different strategies and face implementation challenges that are poorly understood. The present study analyzes the implementation processes of public policies to promote PA in terms of: (i) the policies covered and their legal quality, (ii) the actors and stakeholders involved in the implementation process and (iii) the used implementation strategies (vertical, horizontal or a mix).

METHODS

A scoping review was systematically conducted (registered Open Science Framework: osf.io/7w84q/), searching 10 databases and grey literature until March 2022. Of the 7741 titles and abstracts identified initially, 10 studies were included.

RESULTS

The current evidence includes high-income countries (USA, n = 7; UK, New Zealand and Oman, n = 1 each). Policy areas covered are education (school sector) and PA promotion in general (national PA plans or city-wide approaches). The legal classification ranges from laws (school sector) to coordination and budgeting to non-legally binding recommendations. The jurisdictions covered were federal (n = 4), state (n = 1), county (n = 1), school district (n = 1) and city (n = 3). Implementation strategies for city-wide approaches are characterized by a coordinated approach with vertical and horizontal integration; federal PA policies by a mix of implementation strategies; and the school sector by a strict horizontal top-down integration without the involvement of other actors.

CONCLUSION

Implementation strategies differ by policy field. Therefore, continuous evaluation of the implementation process is necessary to align policy implementation with policy goals to promote individual PA behaviour.

Using implementation strategies in community settings: an introduction to the Expert Recommendations for Implementing Change (ERIC) compilation and future directions

In this commentary, we introduce public health practitioners and researchers to implementation science through an established compilation of implementation strategies. We provide terminology and examples for community settings so public health practitioners and researchers can use implementation strategies and document efforts using standard terminology. We also discuss the need for future work to determine the extent to which these implementation strategies work and are most useful in community settings, and ultimately, how health behaviors are impacted. We intend this commentary to serve as a dissemination strategy for implementation strategies and to contribute to knowledge in the growing field of implementation science in community settings.

Mental Health Reform: Design and Implementation of a System to Optimize Outcomes for Veterans and Their Families

The social, health, and economic burden of mental health problems in the veteran community is heavy. Internationally, the array of services and support available to veterans and their families are extensive but vary in quality, are often disconnected, complex to navigate, and lack clear coordination. This paper describes a conceptual framework to guide the design and implementation of a system of services and supports to optimize the mental health and wellbeing of all veterans and their families. The framework recognizes the diversity of veterans across intersecting identities that uniquely shape experiences of posttraumatic mental health and wellbeing. It brings together several strands of research: the values and principles that should underpin the system; the needs of diverse veterans and their families; challenges in the current services and supports; evidence-based interventions; and principles of effective implementation. Central to the future system design is a next generation stepped model of care that organizes best and next practice interventions in a coherent system, matches service provision to level of need and addresses access and navigation. Practical guidance on implementation provides an aspirational and flexible structure for system evolution, and a template for all stakeholders-individuals, groups, agencies and organizations-to effect system change.

The expression of 'policy' in palliative care: A critical review

The importance of 'policy' within palliative care has steadily increased over the past 25 years. Whilst this has been welcomed within the palliative care field and seen as a route to greater recognition, we focus here on a more critical perspective that challenge the effectiveness of a 'policy turn' in palliative care. Applying Bacchi's "What's the Problem Represented to Be?" (WPR) framework to data from a systematic search, we address the research question, "in what ways has 'policy' been articulated in palliative care literature?". The paper describes the construction of 'the problem' context and reflects critically on the robustness and pragmatic utility of such representations. In particular, we identify five elements as prominent and problematic: (1) a lack of empirical evidence that connects policy to practice; (2) the dominance of 'Global North' approaches; (3) the use of a policy narrative based on 'catastrophe' in justifying the need for palliative care; (4) the use of idealistic and aspirational 'calls to action'; and (5) a disengaged and antagonistic orientation to existing health systems. We conclude by suggesting that the efficacy of palliative care policy could be enhanced via greater emphases on 'Global South' perspectives, 'assets-based' approaches and attention to pragmatic implementation.

Knowledge and attitudes of Implementation Support Practitioners-Findings from a systematic integrative review

BACKGROUND

It requires thoughtful planning and work to successfully apply and sustain research-supported interventions like healthcare treatments, social support, or preventive programs in practice. Implementation support practitioners (ISPs) such as facilitators, technical assistance providers, knowledge brokers, coaches or consultants may be involved to actively support the implementation process. This article presents knowledge and attitudes ISPs bring to their work.

METHODS

Building on a previously developed program logic, a systematic integrative review was conducted. Literature was sourced by searching nine electronic data bases, organizational websites, and by launching a call for publications among selected experts and social media. Article screening was performed independently by two researchers, and data from included studies were extracted by members of the research team and quality-assured by the lead researcher. The quality of included RCTs was assessed based on a framework by Hodder and colleagues. Thematic Analysis was used to capture information on knowledge and attitudes of ISPs across the included studies. Euler diagrams and heatmaps were used to present the results.

RESULTS

Results are based on 79 included studies. ISPs reportedly displayed knowledge about the clinical practice they work with, implementation / improvement practice, the local context, supporting change processes, and facilitating evidence-based practice in general. In particular, knowledge about the intervention to be implemented and its target population, specific improvement / implementation methods and approaches, organizational structures and sensitivities, training, and characteristics of (good) research was described in the literature. Seven themes describing ISPs' attitudes were identified: 1) professional, 2) motivated / motivating / encouraging / empowering, 3) empathetic / respectful / sensitive, 4) collaborative / inclusive, 5) authentic, 6) creative / flexible / innovative / adaptive, and 7) frank / direct / honest. Pertaining to a professional attitude, being responsive and focused were the most prevalent indicators across included publications.

CONCLUSION

The wide range and complexity of knowledge and attitudes found in the literature calls for a comprehensive and systematic approach to collaboratively develop a professional role for ISPs across disciplines. Embedding the ISP role in different health and social welfare settings will enhance implementation capacities considerably.

Revisiting concepts of evidence in implementation science

Background

Evidence, in multiple forms, is a foundation of implementation science. For public health and clinical practice, evidence includes the following: type 1 evidence on etiology and burden; type 2 evidence on effectiveness of interventions; and type 3: evidence on dissemination and implementation (D&I) within context. To support a vision for development and use of evidence in D&I science that is more comprehensive and equitable (particularly for type 3 evidence), this article aims to clarify concepts of evidence, summarize ongoing debates about evidence, and provide a set of recommendations and tools/resources for addressing the “how-to” in filling evidence gaps most critical to advancing implementation science.

Main text

Because current conceptualizations of evidence have been relatively narrow and insufficiently characterized in our opinion, we identify and discuss challenges and debates about the uses, usefulness, and gaps in evidence for implementation science. A set of questions is proposed to assist in determining when evidence is sufficient for dissemination and implementation. Intersecting gaps include the need to (1) reconsider how the evidence base is determined, (2) improve understanding of contextual effects on implementation, (3) sharpen the focus on health equity in how we approach and build the evidence-base, (4) conduct more policy implementation research and evaluation, and (5) learn from audience and stakeholder perspectives. We offer 15 recommendations to assist in filling these gaps and describe a set of tools for enhancing the evidence most needed in implementation science.

Conclusions

To address our recommendations, we see capacity as a necessary ingredient to shift the field’s approach to evidence. Capacity includes the “push” for implementation science where researchers are trained to develop and evaluate evidence which should be useful and feasible for implementers and reflect community or stakeholder priorities. Equally important, there has been inadequate training and too little emphasis on the “pull” for implementation science (e.g., training implementers, practice-based research). We suggest that funders and reviewers of research should adopt and support a more robust definition of evidence. By critically examining the evolving nature of evidence, implementation science can better fulfill its vision of facilitating widespread and equitable adoption, delivery, and sustainment of scientific advances.

Scaling Up Public Health Interventions: Engaging Partners Across Multiple Levels

Advancing the science of intervention scale-up is essential to increasing the impact of effective interventions at the regional and national levels. In contrast with work in high-income countries (HICs), where scale-up research has been limited, researchers in low- and middle-income countries (LMICs) have conducted numerous studies on the regional and national scale-up of interventions. In this article, we review the state of the science on intervention scale-up in both HICs and LMICs. We provide an introduction to the elements of scale-up followed by a description of the scale-up process, with an illustrative case study from our own research. We then present findings from a scoping review comparing scale-up studies in LMIC and HIC settings. We conclude with lessons learned and recommendations for improving scale-up research. Expected final online publication date for the Annual Review of Public Health, Volume 43 is April 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Challenges in implementing emergency obstetric care (EmOC) policies: perspectives and behaviours of frontline health workers in Uganda

Uganda is among the sub-Saharan African Countries which continue to experience high preventable maternal mortality due to obstetric emergencies. Several Emergency Obstetric Care (EmOC) policies rolled out have never achieved their intended targets to date. To explore why upstream policy expectations were not achieved at the frontline during the MDG period, we examined the implementation of EmOC policies in Uganda by; exploring the barriers frontline implementers of EmOC policies faced, their coping behaviours and the consequences for maternal health. We conducted a retrospective exploratory qualitative study between March and June 2019 in Luwero, Iganga and Masindi districts selected based on differences in maternal mortality. Data were collected using 8 in-depth interviews with doctors and 17 midwives who provided EmOC services in Uganda's public health facilities during the MDG period. We reviewed two national maternal health policy documents and interviewed two Ministry of Health Officials on referral by participants. Data analysis was guided by the theory of Street-Level Bureaucracy (SLB). Implementation of EmOC was affected by the incompatibility of policies with implementation systems. Street-level bureaucrats were expected to offer to their continuously increasing clients, sometimes presenting late, ideal EmOC services using an incomplete and unreliable package of inputs, supplies, inadequate workforce size and skills mix. To continue performing their duties and prevent services from total collapse, frontline implementers' coping behaviours oftentimes involved improvization leading to delivery of incomplete and inconsistent EmOC service packages. This resulted in unresponsive EmOC services with mothers receiving inadequate interventions sometimes after major delays across different levels of care. We suggest that SLB theory can be enriched by reflecting on the consequences of the coping behaviours of street-level bureaucrats. Future reforms should align policies to implementation contexts and resources for optimal results.

A systematic review of school health policy measurement tools: implementation determinants and outcomes

BACKGROUND

Governments in some countries or states/provinces mandate school-based policies intended to improve the health and well-being of primary and secondary students and in some cases the health of school staff. Examples include mandating a minimum time spent per week in programmed physical activity, mandating provision of healthy foods and limiting fat content of school meals, and banning tobacco products or use on school campuses. Although school health researchers have studied whether schools, districts, or states/provinces are meeting requirements, it is unclear to what extent implementation processes and determinants are assessed. The purposes of the present systematic review of quantitative measures of school policy implementation were to (1) identify quantitative school health policy measurement tools developed to measure implementation at the school, district, or state/provincial levels; (2) describe the policy implementation outcomes and determinants assessed and identify the trends in measurement; and (3) assess pragmatic and psychometric properties of identified implementation measures to understand their quality and suitability for broader application.

METHODS

Peer-reviewed journal articles published 1995-2020 were included if they (1) had multiple-item quantitative measures of school policy implementation and (2) addressed overall wellness, tobacco, physical activity, nutrition, obesity prevention, or mental health/bullying/social-emotional learning. The final sample comprised 86 measurement tools from 67 peer-review articles. We extracted study characteristics, such as psychometric and pragmatic measure properties, from included articles based on three frameworks: (1) Implementation Outcomes Framework, (2) Consolidated Framework for Implementation Research, and (3) Policy Implementation Determinants Framework.

RESULTS

Most implementation tools were developed to measure overall wellness policies which combined multiple policy topics (n = 35, 40%) and were in survey form (n = 75, 87%). Fidelity was the most frequently prevalent implementation outcome (n = 70, 81%), followed by adoption (n = 32, 81%). The implementation determinants most assessed were readiness for implementation, including resources (n = 43, 50%), leadership (n = 42, 49%), and policy communication (n = 41, 48%). Overall, measures were low-cost and had easy readability. However, lengthy tools and lack of reported validity/reliability data indicate low transferability.

CONCLUSIONS

Implementation science can contribute to more complete and rigorous assessment of school health policy implementation processes, which can improve implementation strategies and ultimately the intended health benefits. Several high-quality measures of implementation determinants and implementation outcomes can be applied to school health policy implementation assessment. Dissemination and implementation science researchers can also benefit from measurement experiences of school health researchers.

Determinants of using children's mental health research in policymaking: variation by type of research use and phase of policy process

Background

Research use in policymaking is multi-faceted and has been the focus of extensive study. However, virtually no quantitative studies have examined whether the determinants of research use vary according to the type of research use or phase of policy process. Understanding such variation is important for selecting the targets of implementation strategies that aim to increase the frequency of research use in policymaking.

Methods

A web-based survey of US state agency officials involved with children’s mental health policymaking was conducted between December 2019 and February 2020 (n = 224, response rate = 33.7%, 49 states responding (98%), median respondents per state = 4). The dependent variables were composite scores of the frequency of using children’s mental health research in general, specific types of research use (i.e., conceptual, instrumental, tactical, imposed), and during different phases of the policy process (i.e., agenda setting, policy development, policy implementation). The independent variables were four composite scores of determinants of research use: agency leadership for research use, agency barriers to research use, research use skills, and dissemination barriers (e.g., lack of actionable messages/recommendations in research summaries, lack of interaction/collaboration with researchers). Separate multiple linear regression models estimated associations between determinant and frequency of research use scores.

Results

Determinants of research use varied significantly by type of research use and phase of policy process. For example, agency leadership for research use was the only determinant significantly associated with imposed research use (β = 0.31, p < 0.001). Skills for research use were the only determinant associated with tactical research use (β = 0.17, p = 0.03) and were only associated with research use in the agenda-setting phase (β = 0.16, p = 0.04). Dissemination barriers were the most universal determinants of research use, as they were significantly and inversely associated with frequency of conceptual (β = −0.21, p = 0.01) and instrumental (β = −0.22, p = 0.01) research use and during all three phases of policy process.

Conclusions

Decisions about the determinants to target with policy-focused implementation strategies—and the strategies that are selected to affect these targets—should reflect the specific types of research use that these strategies aim to influence.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-021-01081-8.

Dissemination Strategies to Accelerate the Policy Impact of Children's Mental Health Services Research

The United States is in the midst of a children's mental health crisis, with rates of depression, anxiety, and suicide increasing precipitously. Evidence produced by children's mental health services research can help address this crisis by informing public policy decisions about service delivery, system design, and investments in the social determinants of mental health. Unfortunately, the policy impact of children's mental health services research is limited because evidence often fails to reach policy makers, be responsive to their needs, resonate with their worldview, or reflect the contexts in which they make decisions. Dissemination strategies-defined as the development and targeted distribution of messages and materials about research evidence pertaining to a specific issue or intervention-can help address these challenges. Yet, limited integrated guidance exists to inform the design of such strategies. This article addresses this need by synthesizing the results of empirical studies to provide guidance about how to enhance the dissemination of children's mental health services research to policy makers. The article provides four recommendations about the content of policy maker-focused dissemination materials, discusses how strategic framing and message tailoring can increase the chances that evidence is persuasive to policy makers, and highlights strategies to ensure that evidence reaches policy makers.