State behavioral health agency website references to evidence-based program registers

Open Science Standards at Journals that Inform Evidence-Based Policy

Evidence-based policy uses intervention research to inform consequential decisions about resource allocation. Research findings are often published in peer-reviewed journals. Because detrimental research practices associated with closed science are common, journal articles report more false-positives and exaggerated effect sizes than would be desirable. Journal implementation of standards that promote open science-such as the transparency and openness promotion (TOP) guidelines-could reduce detrimental research practices and improve the trustworthiness of research evidence on intervention effectiveness. We evaluated TOP implementation at 339 peer-reviewed journals that have been used to identify evidence-based interventions for policymaking and programmatic decisions. Each of ten open science standards in TOP was not implemented in most journals' policies (instructions to authors), procedures (manuscript submission systems), or practices (published articles). Journals implementing at least one standard typically encouraged, but did not require, an open science practice. We discuss why and how journals could improve implementation of open science standards to safeguard evidence-based policy.

A Scoping Review of the Influence of Evidence-Based Program Resources (EBPR) Websites for Behavioral Health

Evidence-based program resources (EBPR) websites evaluate behavioral health programs, practices or policies (i.e., interventions) according to a predetermined set of research criteria and standards, usually resulting in a summary rating of the strength of an intervention's evidence base. This study is a mixed-methods analysis of the peer-reviewed academic literature relating to the influence of EBPRs on clinical practice and policy in the behavioral health field. Using an existing framework for a scoping review, we searched for research articles in PubMed, Web of Science, SCOPUS, and ProQuest that were published between January 2002 and March 2022, referenced an EBPR or multiple EBPRs, and presented data showing the influence of one or more EBPRs on behavioral health. A total of 210 articles met the inclusion criteria and were classified into five distinct categories of influence, the most important of which was showing the direct impact of one or more EBPRs on behavioral health (8.1% of articles), defined as documenting observable changes in interventions or organizations that are at least partly due to information obtained from EBPR(s). These included impacts at the state legislative and policy-making level, at the community intervention level, provider agency level, and individual practitioner level. The majority of influences identified in the study were indirect demonstrations of how EBPRs are used in various ways. However, more studies are needed to learn about the direct impact of information from EBPRs on the behavioral health field, including impact on clinician practice and treatment outcomes for consumers.

The influence of evidence-based program registry websites for dissemination of evidence-based interventions in behavioral healthcare

PURPOSE

Evidence-based program registries (EBPRs) are web-based databases of evaluation studies that summarize the available evidence for the effectiveness of behavioral healthcare programs, including programs addressing substance misuse, mental health, child welfare, or offender rehabilitation. The study determined the extent to which visitors to selected EBPRs accomplished the objectives of their visits and how often those visits resulted in the adoption of new or improved evidence-based interventions (EBIs).

METHOD

A follow-up telephone survey was conducted with 216 visitors to a convenience sample of six EBPRs an average of six months after the visitors' incident visit to the EBPR.

RESULTS

The most frequent objective was to identify evidence-based programs/services, curricula or assessments, followed by finding resources to implement or improve the preceding and writing a grant proposal including to comply with funding requirements; 71% of such objectives were achieved across the full set of objectives. Implementation of an EBI was completely achieved for 31% of relevant objectives and some progress on EBI implementation occurred for 19% of relevant objectives.

CONCLUSIONS

This is the first study to document the usage of EBPRs as a modality to increase the utilization of EBIs in the actual practice of behavioral healthcare. The results support the continued use of web-based EBPRs for disseminating information on evidence-based interventions for behavioral healthcare.

State Department and Provider Agency Utilization of Evidence-Based Program Registries in Behavioral Healthcare and Child Welfare

Evidence-based program registries (EBPRs) are web-based compilations of behavioral healthcare programs/interventions that rely on research-based criteria to rate program efficacy or effectiveness for support of programmatic decision-making. The objective was to determine the extent to which behavioral health decision-makers access EBPRs and to understand whether and exactly how they use the information obtained from EPBRs. Single State Authorities (SSAs) and service provider agencies in the areas of behavioral health and child welfare were recruited nationally. Senior staff (n = 375) responsible for the selection and implementation of programs and/or policies were interviewed by telephone concerning their visits (if any) to 28 relevant EBPRs, the types of information they were seeking, whether they found it, and how they may have used that information to effect changes in their organizations. At least one EBPR was visited by 80% of the respondents, with a median of three different registers being visited. Most visitors (55%) found all the information they were seeking; those who did not desired more guidance or tools for individual program implementation or were unable to locate the program or practice that they were seeking. Most visitors (65%) related using the information obtained to make changes in their organizations, in particular to select, start or change a program, or to support the adoption or improvement of evidence-based clinical practices. EBPRs were shown to be important resources for dissemination of research-based program effectiveness data, leading to increased use of evidence-based practices in the field, but the study also identified needs for greater awareness of EBPRs generally and for more attention to implementation of specific recommended programs and practices.

Visitors' Assessment and Utilization of Evidence-Based Program Resources (EBPR) Websites

Background: Evidence-based program resources (EBPR) websites for behavioral health are a potentially useful tool to assist decision-makers and practitioners in deciding which behavioral health interventions to implement. EBPR websites apply rigorous research standards to assess the effectiveness of behavioral healthcare programs, models, and clinical practices. Method: Visitors to a convenience sample of six EBPR websites (N=369, excluding students) were recruited for telephone interviews primarily by means of a pop-up invitation on the sites. Results: The visitors view the EBPR sites as important sources of information to support the identification and adoption of evidence-based programs/practices (EBPs) in behavioral healthcare, which aligns with the primary mission of EBPRs. For repeat visitors, there was some indication that the information obtained helped effect certain changes in their agencies' programs and policies. However, increased or improved guidance on EBP implementation was also requested. Conclusion: EBPR websites should be better publicized to the behavioral healthcare field.

Variations in Definitions of Evidence-Based Interventions for Behavioral Health in Eight Selected U.S. States

BACKGROUND

U.S. state legislatures fill a vital role in supporting the use of evidence-based interventions (EBIs) through statutes and regulations (mandates).

OBJECTIVE

The study determined the terms used by selected states to describe EBIs and how those terms are defined in mandates.

RESEARCH METHODS

The mandates of eight purposely selected states were accessed and coded using the Westlaw Legal Research Database.

RESULTS

Considerable variation was found in the terms used by states to describe EBIs. Although "evidence-based" was the most frequently utilized term (60% of mandates), an additional 29 alternative terms appeared with varying frequencies. Most terms were simply mentioned, with no further definition or elaboration. When terms were further defined or elaborated, the majority were defined using numerous and different types of external sources or references. Three approaches were found in the mandates defining EBIs: "single definition," "hierarchies of evidence levels," and "best available evidence"; the states differed considerably in the approaches used in their mandates.

CONCLUSIONS

The variations in EBI-related terminology across states and within states, coupled with a lack of elaboration on the meaning of important terms and the predominant use of external rather than internal guidelines, may be a source of confusion for behavioral health provider agencies that seek direction about what constitutes an EBI. Prior studies indicate that many agencies may lack staff with the technical ability to adequately evaluate what constitutes an EBI. Thus, lack of clear guidance from official state government mandates may impede the implementation of EBIs within states.

References to Evidence-based Program Registry (EBPR) websites for behavioral health in U.S. state government statutes and regulations

BACKGROUND AND AIM

U.S. state governments have the responsibility to regulate and license behavioral healthcare interventions, such as for addiction and mental illness, with increasing emphasis on implementing evidence-based programs (EBPs). A serious obstacle to this is lack of clarity or agreement about what constitutes "evidence-based." The study's purpose was to determine the extent to which and in what contexts web-based Evidence-based Program Registries (EBPRs) are referenced in state government statutes and regulations ("mandates") concerning behavioral healthcare. Examples are: What Works Clearinghouse; National Register of Evidence-based Programs and Practices; Cochrane Database of Systematic Reviews.

METHODS

The study employed the Westlaw Legal Research Database to search for 30 known EBPR websites relevant to behavioral healthcare within the statutes and regulations of all 50 states.

RESULTS

There was low prevalence of EBPR references in state statutes and regulations pertaining to behavioral healthcare; 20 states had a total of 33 mandates that referenced an EBPR. These mandates usually do not rely on an EBPR as the sole acceptable source for classifying a program or practice as "evidence-based." Instead, EBPRs were named in conjunction with internal state or external sources of information about putative program effectiveness, which may be less valid than EBPRs, to determine what is "evidence-based."

CONCLUSION

Greater awareness of scientifically - based EBPRs and greater understanding of their advantages need to be fostered among state legislators and regulators charged with making policy to increase or improve the use of evidence-based programs and practices in behavioral healthcare in the U.S.

Utilization of Evidence-based Intervention Criteria in U.S. Federal Grant Funding Announcements for Behavioral Healthcare

Recent U.S. federal government policy has required or recommended the use of evidence-based interventions (EBIs), so that it is important to determine the extent to which this priority is reflected in actual federal solicitations for intervention funding, particularly for behavioral healthcare interventions. Understanding how well such policies are incorporated in federal opportunity announcements (FOAs) for grant funding could improve compliance with policy and increase the societal use of evidence-based interventions for behavioral healthcare. FOAs for discretionary grants (n = 243) in fiscal year 2021 were obtained from the Grants.gov website for 44 federal departments, agencies and sub-agencies that were likely to fund interventions in behavioral health-related areas. FOAs for block/formula grants to states that included behavioral healthcare (n = 17) were obtained from the SAM.gov website. Across both discretionary and block grants, EBIs were required in 60% and recommended in 21% of these FOAs for funding. Numerous different terms were used to signify EBIs by the FOAs, with the greatest variation occurring among the block grants. Lack of adequate elaboration or definition of alternative EBI terms prominently characterized FOAs issued by the Department of Health and Human Services, although less so for those issued by the Departments of Justice and Education. Overall, 43% of FOAs referenced evidence-based program registers on the web, which are scientifically credible sources of EBIs. Otherwise, most of the remaining elaborations of EBI terms in these FOAs were quite brief, often idiosyncratic, and not scientifically vetted. The FOAs generally adhered to federal policy requiring or encouraging the use of EBIs for funding requests. However, an overall pattern showing lack or inadequate elaboration of terms signifying EBIs makes it difficult for applicants to comply with federal policies regarding use of EBIs for behavioral healthcare.