1
|
STADNICK NICOLEA, GEREMIA CARRIE, MAURI AMANDAI, SWANSON KERA, WYNECOOP MEGAN, PURTLE JONATHAN. A Mixed-Methods Exploration of the Implementation of Policies That Earmarked Taxes for Behavioral Health. Milbank Q 2024; 102:913-943. [PMID: 39240049 PMCID: PMC11654764 DOI: 10.1111/1468-0009.12715] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2024] [Revised: 07/10/2024] [Accepted: 08/19/2024] [Indexed: 09/07/2024] Open
Abstract
Policy Points Earmarked tax policies for behavioral health are perceived as having positive impacts related to increasing flexible funding, suggesting benefits to expand this financing approach. Implementation challenges related to these earmarked taxes included tax base volatility that impedes long-term service delivery planning and inequities in the distribution of tax revenue. Recommendations for designing or revising earmarked tax policies include developing clear guidelines and support systems to manage the administrative aspects of earmarked tax programs, cocreating reporting and oversight structures with system and service delivery agents, and selecting revenue streams that are relatively stable across years. CONTEXT Over 200 cities and counties in the United States have implemented policies earmarking tax revenue for behavioral health services. This mixed-methods study was conducted with the aim of characterizing perceptions of the impacts of these earmarked tax policies, strengths and weaknesses of tax policy designs, and factors that influence decision making about how tax revenue is allocated for services. METHODS Study data came from surveys completed by 274 officials involved in behavioral health earmarked tax policy implementation and 37 interviews with officials in a sample of jurisdictions with these taxes-California (n = 16), Washington (n = 12), Colorado (n = 6), and Iowa (n = 3). Interviews primarily explored perceptions of the advantages and drawbacks of the earmarked tax, perceptions of tax policy design, and factors influencing decisions about revenue allocation. FINDINGS A total of 83% of respondents strongly agreed that it was better to have the tax than not, 73.2% strongly agreed that the tax increased flexibility to address complex behavioral health needs, and 65.1% strongly agreed that the tax increased the number of people served by evidence-based practices. Only 43.3%, however, strongly agreed that it was easy to satisfy tax-reporting requirements. Interviews revealed that the taxes enabled funding for services and implementation supports, such as training in the delivery of evidence-based practices, and supplemented mainstream funding sources (e.g., Medicaid). However, some interviewees also reported challenges related to volatility of funding, inequities in the distribution of tax revenue, and, in some cases, administratively burdensome tax reporting. Decisions about tax revenue allocation were influenced by goals such as reducing behavioral health care inequities, being responsive to community needs, addressing constraints of mainstream funding sources, and, to a lesser degree, supporting services considered to be evidence based. CONCLUSIONS Earmarked taxes are a promising financing strategy to improve access to, and quality of, behavioral health services by supplementing mainstream state and federal financing.
Collapse
Affiliation(s)
- NICOLE A. STADNICK
- University of CaliforniaSan Diego
- Altman Clinical and Translational Research Institute, Dissemination and Implementation Science CenterUniversity of CaliforniaSan Diego
- Child and Adolescent Services Research Center
| | | | - AMANDA I. MAURI
- Public Health Policy and Management, School of Global Public HealthNew York University
| | - KERA SWANSON
- Altman Clinical and Translational Research Institute, Dissemination and Implementation Science CenterUniversity of CaliforniaSan Diego
| | - MEGAN WYNECOOP
- Public Health Policy and Management, School of Global Public HealthNew York University
| | - JONATHAN PURTLE
- Public Health Policy and Management, School of Global Public HealthNew York University
| |
Collapse
|
2
|
Hoagwood KE, Richards-Rachlin S, Baier M, Vilgorin B, Horwitz SM, Narcisse I, Diedrich N, Cleek A. Implementation Feasibility and Hidden Costs of Statewide Scaling of Evidence-Based Therapies for Children and Adolescents. Psychiatr Serv 2024; 75:461-469. [PMID: 38268465 PMCID: PMC11099614 DOI: 10.1176/appi.ps.20230183] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/26/2024]
Abstract
OBJECTIVE State mental health systems are retraining their workforces to deliver services supported by research. Knowledge about evidence-based therapies (EBTs) for child and adolescent disorders is robust, but the feasibility of their statewide scaling has not been examined. The authors reviewed implementation feasibility for 12 commonly used EBTs, defining feasibility for statewide scaling as an EBT having at least one study documenting acceptability, facilitators and barriers, or fidelity; at least one study with a racially and ethnically diverse sample; an entity for training, certification, or licensing; and fiscal data reflecting the costs of implementation. METHODS The authors reviewed materials for 12 EBTs being scaled in New York State and conducted a literature review with search terms relevant to their implementation. Costs and certification information were supplemented by discussions with treatment developers and implementers. RESULTS All 12 EBTs had been examined for implementation feasibility, but only three had been examined for statewide scaling. Eleven had been studied in populations reflecting racial-ethnic diversity, but few had sufficient power for subgroup analyses to demonstrate effectiveness with these samples. All had certifying or licensing entities. The per-clinician costs of implementation ranged from $500 to $3,500, with overall ongoing costs ranging from $100 to $6,000. A fiscal analysis of three EBTs revealed hidden costs ranging from $5,000 to $24,000 per clinician, potentially limiting sustainability. CONCLUSIONS The evidence necessary for embedding EBTs in state systems has notable gaps that may hinder sustainability. Research-funding agencies should prioritize studies that focus on the practical aspects of scaling to assist states as they retrain their workforces.
Collapse
Affiliation(s)
- Kimberly Eaton Hoagwood
- Department of Child and Adolescent Psychiatry, New York University (NYU) Grossman School of Medicine, New York City (Hoagwood, Richards-Rachlin, Horwitz, Narcisse); Department of Psychology, St. John's University, New York City (Richards-Rachlin); McSilver Institute for Poverty Policy and Research, NYU Silver School of Social Work, New York City (Baier, Vilgorin, Diedrich, Cleek)
| | - Shira Richards-Rachlin
- Department of Child and Adolescent Psychiatry, New York University (NYU) Grossman School of Medicine, New York City (Hoagwood, Richards-Rachlin, Horwitz, Narcisse); Department of Psychology, St. John's University, New York City (Richards-Rachlin); McSilver Institute for Poverty Policy and Research, NYU Silver School of Social Work, New York City (Baier, Vilgorin, Diedrich, Cleek)
| | - Meaghan Baier
- Department of Child and Adolescent Psychiatry, New York University (NYU) Grossman School of Medicine, New York City (Hoagwood, Richards-Rachlin, Horwitz, Narcisse); Department of Psychology, St. John's University, New York City (Richards-Rachlin); McSilver Institute for Poverty Policy and Research, NYU Silver School of Social Work, New York City (Baier, Vilgorin, Diedrich, Cleek)
| | - Boris Vilgorin
- Department of Child and Adolescent Psychiatry, New York University (NYU) Grossman School of Medicine, New York City (Hoagwood, Richards-Rachlin, Horwitz, Narcisse); Department of Psychology, St. John's University, New York City (Richards-Rachlin); McSilver Institute for Poverty Policy and Research, NYU Silver School of Social Work, New York City (Baier, Vilgorin, Diedrich, Cleek)
| | - Sarah McCue Horwitz
- Department of Child and Adolescent Psychiatry, New York University (NYU) Grossman School of Medicine, New York City (Hoagwood, Richards-Rachlin, Horwitz, Narcisse); Department of Psychology, St. John's University, New York City (Richards-Rachlin); McSilver Institute for Poverty Policy and Research, NYU Silver School of Social Work, New York City (Baier, Vilgorin, Diedrich, Cleek)
| | - Iriane Narcisse
- Department of Child and Adolescent Psychiatry, New York University (NYU) Grossman School of Medicine, New York City (Hoagwood, Richards-Rachlin, Horwitz, Narcisse); Department of Psychology, St. John's University, New York City (Richards-Rachlin); McSilver Institute for Poverty Policy and Research, NYU Silver School of Social Work, New York City (Baier, Vilgorin, Diedrich, Cleek)
| | - Nadege Diedrich
- Department of Child and Adolescent Psychiatry, New York University (NYU) Grossman School of Medicine, New York City (Hoagwood, Richards-Rachlin, Horwitz, Narcisse); Department of Psychology, St. John's University, New York City (Richards-Rachlin); McSilver Institute for Poverty Policy and Research, NYU Silver School of Social Work, New York City (Baier, Vilgorin, Diedrich, Cleek)
| | - Andrew Cleek
- Department of Child and Adolescent Psychiatry, New York University (NYU) Grossman School of Medicine, New York City (Hoagwood, Richards-Rachlin, Horwitz, Narcisse); Department of Psychology, St. John's University, New York City (Richards-Rachlin); McSilver Institute for Poverty Policy and Research, NYU Silver School of Social Work, New York City (Baier, Vilgorin, Diedrich, Cleek)
| |
Collapse
|
3
|
Purtle J, Stadnick NA, Wynecoop M, Walker SC, Bruns EJ, Aarons GA. A Tale of Two Taxes: Implementation of Earmarked Taxes for Behavioral Health Services in California and Washington State. Psychiatr Serv 2024; 75:410-418. [PMID: 37933132 PMCID: PMC11139541 DOI: 10.1176/appi.ps.20230257] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/08/2023]
Abstract
OBJECTIVE The authors sought to characterize perceptions of the impacts, attributes, and support for taxes earmarked for behavioral health services and to compare perceptions of the taxes among professionals in California and Washington, two states differing in earmarked tax designs. METHODS Surveys were completed by 155 public agency and community organization professionals involved in tax implementation in California (N=87) and Washington State (N=68) during 2022-2023 (29% response rate). Respondents indicated their perceptions of the taxes' impacts, attributes, and support. Responses were summed as aggregate scores and were also analyzed as individual items. Bivariate analyses were used to compare responses of professionals in California versus Washington State. RESULTS Earmarked taxes were generally regarded positively. Of the respondents, >80% strongly agreed that the taxes increased funding for services and were helpful, and only 10% strongly agreed that the taxes decreased behavioral health funding from other sources. Substantially more respondents in California than in Washington State strongly agreed that taxes' reporting requirements were complicated (45% vs. 5%, p<0.001) and that the taxes increased unjustified scrutiny of services or systems (33% vs. 2%, p<0.001). However, more respondents in California than in Washington State also strongly agreed that the taxes increased public awareness about behavioral health (56% vs. 15%, p<0.001) and decreased behavioral health stigma (47% vs. 14%, p<0.001). CONCLUSIONS Perceptions of the strengths and weaknesses of taxes earmarked for behavioral health services may vary by design features of the tax. Such features include stigma-reduction initiatives and tax spending and reporting requirements.
Collapse
Affiliation(s)
- Jonathan Purtle
- Global Center for Implementation Science, Department of Public Health Policy and Management, New York University School of Global Public Health, New York City (Purtle, Wynecoop); Dissemination and Implementation Science Center, Department of Psychiatry, Altman Clinical and Translational Research Institute, University of California, San Diego, La Jolla (Stadnick, Aarons); Department of Psychiatry and Behavioral Sciences, University of Washington School of Medicine, Seattle (Walker, Bruns)
| | - Nicole A Stadnick
- Global Center for Implementation Science, Department of Public Health Policy and Management, New York University School of Global Public Health, New York City (Purtle, Wynecoop); Dissemination and Implementation Science Center, Department of Psychiatry, Altman Clinical and Translational Research Institute, University of California, San Diego, La Jolla (Stadnick, Aarons); Department of Psychiatry and Behavioral Sciences, University of Washington School of Medicine, Seattle (Walker, Bruns)
| | - Megan Wynecoop
- Global Center for Implementation Science, Department of Public Health Policy and Management, New York University School of Global Public Health, New York City (Purtle, Wynecoop); Dissemination and Implementation Science Center, Department of Psychiatry, Altman Clinical and Translational Research Institute, University of California, San Diego, La Jolla (Stadnick, Aarons); Department of Psychiatry and Behavioral Sciences, University of Washington School of Medicine, Seattle (Walker, Bruns)
| | - Sarah C Walker
- Global Center for Implementation Science, Department of Public Health Policy and Management, New York University School of Global Public Health, New York City (Purtle, Wynecoop); Dissemination and Implementation Science Center, Department of Psychiatry, Altman Clinical and Translational Research Institute, University of California, San Diego, La Jolla (Stadnick, Aarons); Department of Psychiatry and Behavioral Sciences, University of Washington School of Medicine, Seattle (Walker, Bruns)
| | - Eric J Bruns
- Global Center for Implementation Science, Department of Public Health Policy and Management, New York University School of Global Public Health, New York City (Purtle, Wynecoop); Dissemination and Implementation Science Center, Department of Psychiatry, Altman Clinical and Translational Research Institute, University of California, San Diego, La Jolla (Stadnick, Aarons); Department of Psychiatry and Behavioral Sciences, University of Washington School of Medicine, Seattle (Walker, Bruns)
| | - Gregory A Aarons
- Global Center for Implementation Science, Department of Public Health Policy and Management, New York University School of Global Public Health, New York City (Purtle, Wynecoop); Dissemination and Implementation Science Center, Department of Psychiatry, Altman Clinical and Translational Research Institute, University of California, San Diego, La Jolla (Stadnick, Aarons); Department of Psychiatry and Behavioral Sciences, University of Washington School of Medicine, Seattle (Walker, Bruns)
| |
Collapse
|
4
|
Wortham WK, Rodwin AH, Purtle J, Munson MR, Raghavan R. Revisiting the policy ecology framework for implementation of evidence-based practices in mental health settings. Implement Sci 2023; 18:58. [PMID: 37936123 PMCID: PMC10629012 DOI: 10.1186/s13012-023-01309-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2023] [Accepted: 10/01/2023] [Indexed: 11/09/2023] Open
Abstract
BACKGROUND Over the past three decades, policy actors and actions have been highly influential in supporting the implementation of evidence-based practices (EBPs) in mental health settings. An early examination of these actions resulted in the Policy Ecology Framework (PEF), which was originally developed as a tactical primer for state and local mental health regulators in the field of child mental health. However, the policy landscape for implementation has evolved significantly since the original PEF was published. An interrogation of the strategies originally proposed in the PEF is necessary to provide an updated menu of strategies to improve our understanding of the mechanisms of policy action and promote system improvement. OBJECTIVES This paper builds upon the original PEF to address changes in the policy landscape for the implementation of mental health EBPs between 2009 and 2022. We review the current state of policy strategies that support the implementation of EBPs in mental health care and outline key areas for policy-oriented implementation research. Our review identifies policy strategies at federal, state, agency, and organizational levels, and highlights developments in the social context in which EBPs are implemented. Furthermore, our review is organized around some key changes that occurred across each PEF domain that span organizational, agency, political, and social contexts along with subdomains within each area. DISCUSSION We present an updated menu of policy strategies to support the implementation of EBPs in mental health settings. This updated menu of strategies considers the broad range of conceptual developments and changes in the policy landscape. These developments have occurred across the organizational, agency, political, and social contexts and are important for policymakers to consider in the context of supporting the implementation of EBPs. The updated PEF expands and enhances the specification of policy levers currently available, and identifies policy targets that are underdeveloped (e.g., de-implementation and sustainment) but are becoming visible opportunities for policy to support system improvement. The updated PEF clarifies current policy efforts within the field of implementation science in health to conceptualize and better operationalize the role of policy in the implementation of EBPs.
Collapse
Affiliation(s)
- Whitney K Wortham
- Silver School of Social Work, New York University, 1 Washington Square North, New York, NY, 10003, USA.
| | - Aaron H Rodwin
- Silver School of Social Work, New York University, 1 Washington Square North, New York, NY, 10003, USA
| | - Jonathan Purtle
- Department of Public Health Policy & Management, Global Center for Implementation Science, School of Global Public Health, New York University, 708 Broadway, New York, NY, 10003, USA
| | - Michelle R Munson
- Silver School of Social Work, New York University, 1 Washington Square North, New York, NY, 10003, USA
| | - Ramesh Raghavan
- Silver School of Social Work, New York University, 1 Washington Square North, New York, NY, 10003, USA
| |
Collapse
|
5
|
Purtle J, Stadnick NA, Wynecoop M, Bruns EJ, Crane ME, Aarons G. A policy implementation study of earmarked taxes for mental health services: study protocol. Implement Sci Commun 2023; 4:37. [PMID: 37004117 PMCID: PMC10067193 DOI: 10.1186/s43058-023-00408-4] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Accepted: 03/05/2023] [Indexed: 04/03/2023] Open
Abstract
BACKGROUND Insufficient funding is frequently identified as a critical barrier to the implementation and sustainment of evidence-based practices (EBPs). Thus, increasing access to funding is recognized as an implementation strategy. Policies that create earmarked taxes-defined as taxes for which revenue can only be spent on specific activities-are an increasingly common mental health financing strategy that could improve the reach of EBPs. This project's specific aims are to (1) identify all jurisdictions in the USA that have implemented earmarked taxes for mental health and catalogue information about tax design; (2) characterize experiences implementing earmarked taxes among local (e.g., county, city) mental health agency leaders and other government and community organization officials and assess their perceptions of the acceptability and feasibility of different types of policy implementation strategies; and (3) develop a framework to guide effect earmarked tax designs, inform the selection of implementation strategies, and disseminate the framework to policy audiences. METHODS The project uses the Exploration, Preparation, Implementation, Sustainment (EPIS) framework to inform data collection about the determinants and processes of tax implementation and Leeman's typology of implementation strategies to examine the acceptability and feasibility strategies which could support earmarked tax policy implementation. A legal mapping will be conducted to achieve aim 1. To achieve aim 2, a survey will be conducted of 300 local mental health agency leaders and other government and community organization officials involved with the implementation of earmarked taxes for mental health. The survey will be followed by approximately 50 interviews with these officials. To achieve aim 3, quantitative and qualitative data will be integrated through a systematic framework development and dissemination process. DISCUSSION This exploratory policy implementation process study will build the evidence base for outer-context implementation determinants and strategies by focusing on policies that earmarked taxes for mental health services.
Collapse
Affiliation(s)
- Jonathan Purtle
- Department of Public Health Policy & Management, Global Center for Implementation Science, New York University School of Global Public Health, 708, Broadway, New York, NY, 10003, USA.
| | - Nicole A Stadnick
- Department of Psychiatry, Dissemination and Implementation Science Center, Altman Clinical and Translational Research Institute, University of California San Diego, 9500 Gilman Drive, La Jolla, CA, 92093, USA
| | - Megan Wynecoop
- Department of Public Health Policy & Management, Global Center for Implementation Science, New York University School of Global Public Health, 708, Broadway, New York, NY, 10003, USA
| | - Eric J Bruns
- Department of Psychiatry and Behavioral Sciences, University of Washington School of Medicine, 6200 NE 74Th St, Building 29, Suite 110, Seattle, WA, 98115, USA
| | - Margaret E Crane
- Department of Psychology, Temple University, Weiss Hall, 1701 N 13Th St, Philadelphia, PA, 19122, USA
- Department of Psychiatry, New York Presbyterian-Weill Cornell Medicine, 425 E 61St St, New York, NY, 10065, USA
| | - Gregory Aarons
- Department of Psychiatry, Dissemination and Implementation Science Center, Altman Clinical and Translational Research Institute, University of California San Diego, 9500 Gilman Drive, La Jolla, CA, 92093, USA
| |
Collapse
|
6
|
Frank HE, Milgram L, Freeman JB, Benito KG. Expanding the reach of evidence-based mental health interventions to private practice: Qualitative assessment using a policy ecology framework. FRONTIERS IN HEALTH SERVICES 2022; 2:892294. [PMID: 36925863 PMCID: PMC10012822 DOI: 10.3389/frhs.2022.892294] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/08/2022] [Accepted: 06/29/2022] [Indexed: 11/13/2022]
Abstract
Background Evidence-based interventions (EBIs) for mental health disorders are underutilized in routine clinical practice. Exposure therapy for anxiety disorders is one particularly difficult-to-implement EBI that has robust empirical support. Previous research has examined EBI implementation determinants in publicly funded mental health settings, but few studies have examined EBI implementation determinants in private practice settings. Private practice clinicians likely face unique barriers to implementation, including setting-specific contextual barriers to EBI use. The policy ecology framework considers broad systemic determinants, including organizational, regulatory, social, and political contexts, which are likely relevant to EBI implementation in private practice settings but have not been examined in prior research. Methods Qualitative interviews were conducted to assess private practice clinicians' perceptions of EBI implementation determinants using the policy ecology framework. Clinicians were asked about implementing mental health EBIs broadly and exposure therapy specifically. Mixed methods analyses compared responses from clinicians working in solo vs. group private practice and clinicians who reported high vs. low organizational support for exposure therapy. Results Responses highlight several barriers and facilitators to EBI implementation in private practice. Examples include determinants related to organizational support (e.g., colleagues using EBIs), payer restrictions (e.g., lack of reimbursement for longer sessions), fiscal incentives (e.g., payment for attending training), and consumer demand for EBIs. There were notable differences in barriers faced by clinicians who work in group private practices compared to those working in solo practices. Solo private practice clinicians described ways in which their practice setting limits their degree of colleague support (e.g., for consultation or exposure therapy planning), while also allowing for flexibility (e.g., in their schedules and practice location) that may not be available to clinicians in group practice. Conclusions Using the policy ecology framework provides a broad understanding of contextual factors that impact private practice clinicians' use of EBIs, including exposure therapy. Findings point to potential implementation strategies that may address barriers that are unique to clinicians working in private practice.
Collapse
Affiliation(s)
- Hannah E Frank
- Warren Alpert Medical School of Brown University, Providence, RI, United States.,Bradley Hospital, East Providence, RI, United States
| | - Lauren Milgram
- Warren Alpert Medical School of Brown University, Providence, RI, United States.,Bradley Hospital, East Providence, RI, United States
| | - Jennifer B Freeman
- Warren Alpert Medical School of Brown University, Providence, RI, United States.,Bradley Hospital, East Providence, RI, United States
| | - Kristen G Benito
- Warren Alpert Medical School of Brown University, Providence, RI, United States.,Bradley Hospital, East Providence, RI, United States
| |
Collapse
|
7
|
Miranda AAV, Murta SG. Difusão de Programas Preventivos em Saúde Mental: Notas Teóricas. PSICOLOGIA: TEORIA E PESQUISA 2022. [DOI: 10.1590/0102.3772e38421.pt] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022] Open
Abstract
Resumo Transtornos mentais causam incapacidades e mortes prematuras, além de representarem custos psicológico, social e econômico elevados. Embora programas de prevenção e promoção em saúde mental com evidências de efetividade estejam disponíveis, são pouco difundidos mundo afora e, em particular, no Brasil. Este artigo tem o propósito de discutir a difusão de programas de prevenção e promoção em saúde mental, segundo as lentes da Teoria da Difusão de Inovações e os desdobramentos contemporâneos. Ênfase será dada às etapas de disseminação, adoção, implementação e sustentabilidade. Implicações para o ensino, a pesquisa e a gestão são discutidas.
Collapse
|
8
|
Byeon YV, Lau AS, Lind T, Hamilton AB, Brookman-Frazee L. Organizational factors associated with community therapists' self-efficacy in EBP delivery: The interplay between sustainment leadership, sustainment climate, and psychological safety. IMPLEMENTATION RESEARCH AND PRACTICE 2022; 3:26334895221110263. [PMID: 37091086 PMCID: PMC9978605 DOI: 10.1177/26334895221110263] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022] Open
Abstract
Background Inner context organizational factors proximally shape therapist experiences with evidence-based practice (EBP) implementation and may influence therapist self-efficacy, which has been linked to sustained use of EBPs in community mental health settings. Research has primarily focused on constructs such as implementation leadership and climate. However, the effects of such factors may depend upon other inner context dimensions, such as psychological safety. Psychologically safe environments are conducive to taking risks, speaking up about problems, and requesting feedback and may promote therapist self-efficacy during implementation. This study examines whether organizational sustainment leadership and sustainment climate relate to therapist EBP self-efficacy only under conditions of psychological safety. Methods Data were collected from 410 clinicians in 85 programs during the sustainment phase of a system-driven implementation of multiple EBPs in children's mental health services. Therapists reported on their organization's sustainment leadership, sustainment climate, psychological safety, and their own self-efficacy in delivering specific EBPs. Multilevel regression analyses were conducted to account for nested data structure. Results Among program-level variables, sustainment leadership and psychological safety both significantly predicted therapist self-efficacy. However, there were no significant interactions between program-level sustainment climate and psychological safety. Exploratory post-hoc analyses revealed a significant interaction between program-level sustainment leadership and therapist-level perceptions of psychological safety such that that the conditional effect of psychological safety on EBP self-efficacy was significant at high levels of sustainment leadership, but not at low or average levels. Conclusion We noted independent links between sustainment leadership, organizational psychological safety and therapists feelings of confidence and mastery with EBPs. Therapists' individual perceptions of psychological safety were linked to self-efficacy only in programs with high sustainment leadership. Thus, sustainment leadership and psychological safety may both represent implementation intervention targets, but it may not be critical to assess for perceptions of psychological safety before deploying organizational leadership strategies.Plain language abstract Therapist self-efficacy is a therapist's belief that they are capable, knowledgeable, and skilled enough to deliver evidence-based practices (EBPs), and is thought to promote improved clinical and implementation outcomes, such as therapists' sustained use of EBPs. Conditions within community mental health organizations may influence therapists' sense of EBP self-efficacy. Leaders' support and expectations for EBP implementation, and collective staff perceptions about the organization's climate to support EBPs are linked to positive therapist attitudes and EBP adoption. However, less is known about how these implementation-specific organizational factors associated with therapist EBP self-efficacy in the long-term, and how this may depend on general workplace conditions. Specifically, psychologically safe environments - where therapists feel safe taking risks such as asking questions, admitting mistakes, and trying new skills - may be needed to promote self-efficacy when therapists are tasked with learning and using complex multi-component EBP innovations. The current study tested the prediction that leader-driven and program-wide focus on EBP sustainment may promote therapist EBP self-efficacy only in organizations where conditions for learning are psychologically safe. Our findings confirmed that fostering strong sustainment focused leadership and psychologically safe environments may each be important for increasing therapists' EBP self-efficacy. The model results suggested that individual therapist perceptions of psychological safety were more strongly related to EBP self-efficacy in programs with greater implementation leadership. Findings suggest the importance of increasing EBP leadership behavior to fully potentiate other facilitating conditions for therapist learning in the sustainment phase of EBP implementation initiatives.
Collapse
Affiliation(s)
- Y. Vivian Byeon
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Anna S. Lau
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Teresa Lind
- Department of Child & Family Development, San Diego State
University, San Diego, CA, USA
- Child and Adolescent Services Research
Center, San Diego, CA, USA
| | - Alison B. Hamilton
- Department of Psychiatry and Biobehavioral Sciences, University of California, Los Angeles, Los Angeles, CA, USA
- Center for the Study of Healthcare Innovation, Implementation, &
Policy, VA Greater Los Angeles Healthcare System, Los Angeles, CA, USA
| | - Lauren Brookman-Frazee
- Child and Adolescent Services Research
Center, San Diego, CA, USA
- Department of Psychiatry, University of California, San Diego, La
Jolla, CA, USA
| |
Collapse
|
9
|
Stewart RE, Mandell DS, Beidas RS. Lessons From Maslow: Prioritizing Funding to Improve the Quality of Community Mental Health and Substance Use Services. Psychiatr Serv 2021; 72:1219-1221. [PMID: 33853379 PMCID: PMC8928566 DOI: 10.1176/appi.ps.202000209] [Citation(s) in RCA: 22] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
Evidence-based practices (EBPs) are frequently proposed as an approach to close the quality chasm in behavioral health treatment, and many U.S. municipalities are investing in EBPs as a primary way to improve the quality of care delivered to individuals most in need. In this Open Forum, the authors argue that EBPs often cannot be successfully implemented because basic organizational needs are not met in the current fiscal environment. The authors summarize research that supports why EBPs, along with other approaches to improve quality, are likely to fail until there is adequate financing. They also propose a policy and research agenda to ameliorate and address the fiscal challenges inherent in community mental health and substance use services.
Collapse
Affiliation(s)
- Rebecca E Stewart
- Department of Psychiatry, Perelman School of Medicine and Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia (all authors); Penn Implementation Science Center at the Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia (Beidas)
| | - David S Mandell
- Department of Psychiatry, Perelman School of Medicine and Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia (all authors); Penn Implementation Science Center at the Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia (Beidas)
| | - Rinad S Beidas
- Department of Psychiatry, Perelman School of Medicine and Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia (all authors); Penn Implementation Science Center at the Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia (Beidas)
| |
Collapse
|
10
|
Lui JHL, Brookman-Frazee L, Lind T, Le K, Roesch S, Aarons GA, Innes-Gomberg D, Pesanti K, Lau AS. Outer-context determinants in the sustainment phase of a reimbursement-driven implementation of evidence-based practices in children's mental health services. Implement Sci 2021; 16:82. [PMID: 34412666 PMCID: PMC8375074 DOI: 10.1186/s13012-021-01149-5] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/05/2021] [Accepted: 08/04/2021] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Although there is increasing investment to implement evidence-based practices (EBPs) in public systems across the USA, continued or sustained use of EBPs after initial implementation remains a challenge. The low integration of EBPs in routine practice severely limits their public health impact, highlighting the need to understand factors that affect the return on costly investments in EBP implementation. This study aims to (1) characterize trajectories of EBP delivery volume through a reimbursement-driven implementation and (2) examine impacts of system-level policy regulatory activity and state-level mental health services funding on the implementation reimbursement strategy. METHODS This study involved secondary data analyses. Psychotherapy administrative claims and regulatory site visit data from the Los Angeles County Department of Mental Health and California state mental health expenditures were extracted from 2010 to 2017. Multilevel regression examined EBP claims volume over time with state expenditures and regulatory compliance as predictors. RESULTS EBP claims volume trajectories demonstrated a rapid initial increase, followed by a period of decrease, and a small increase in the final year. State mental health expenditures increased across time reflecting increased funding availability. State mental health expenditures and system regulatory compliance were inversely related to EBP claims volume. CONCLUSIONS The impact of reimbursement-driven EBP implementation strategy is sensitive to multiple outer-context determinants. At the system level, commitment to fidelity of implementation regulations resulted in reduced use of the reimbursement strategy. Alternative reimbursement streams not tied to EBPs coupled with an expanded array of reimbursable services also impacted the use of the reimbursement strategy to implement EBPs.
Collapse
Affiliation(s)
- Joyce H L Lui
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA.
- Department of Psychology, University of Maryland, College Park, College Park, MD, USA.
| | - Lauren Brookman-Frazee
- Department of Psychiatry, University of California, San Diego, San Diego, CA, USA
- Child and Adolescent Services Research Center, San Diego, CA, USA
| | - Teresa Lind
- Department of Psychiatry, University of California, San Diego, San Diego, CA, USA
- Child and Adolescent Services Research Center, San Diego, CA, USA
| | - Kenny Le
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
| | - Scott Roesch
- Child and Adolescent Services Research Center, San Diego, CA, USA
- Department of Psychology, San Diego State University, San Diego, CA, USA
| | - Gregory A Aarons
- Department of Psychiatry, University of California, San Diego, San Diego, CA, USA
- Child and Adolescent Services Research Center, San Diego, CA, USA
| | | | - Keri Pesanti
- Los Angeles County Department of Mental Health, Los Angeles, CA, USA
| | - Anna S Lau
- Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA
| |
Collapse
|
11
|
Stewart RE, Wolk CB, Neimark G, Vyas R, Young J, Tjoa C, Kampman K, Jones DT, Mandell DS. It's not just the money: The role of treatment ideology in publicly funded substance use disorder treatment. J Subst Abuse Treat 2021; 120:108176. [PMID: 33298303 PMCID: PMC7808060 DOI: 10.1016/j.jsat.2020.108176] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2020] [Revised: 08/04/2020] [Accepted: 10/15/2020] [Indexed: 11/19/2022]
Abstract
Medications for opioid use disorder (MOUD) are a first-line treatment for opioid use disorder, yet national surveys indicate that most substance use treatment facilities do not offer MOUD. This article presents the results of a qualitative analysis of interviews with leaders from 25 treatment organizations in Philadelphia, Pennsylvania, that investigated attitudes and barriers toward MOUD. Most treatment organizations that we interviewed are adopting at least one MOUD, suggesting that Philadelphia exceeds the national average of organizations with MOUD capacity. Leaders indicated that both practical resources and ideological barriers thwart adoption and implementation of MOUD in publicly funded substance use disorder treatment agencies. Organizations that had recently adopted MOUDs revealed facilitators to MOUD adoption, such as strong leadership that champions the implementation to staff and redefining recovery from substance use disorders throughout the organization. This study's findings highlight that clients, clinicians, and leadership need to address both practical and ideological barriers to expanding MOUD use.
Collapse
Affiliation(s)
- Rebecca E Stewart
- Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA.
| | - Courtney Benjamin Wolk
- Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
| | | | - Ridhi Vyas
- Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
| | - Jordyn Young
- Department of Psychology, University of Pennsylvania, Philadelphia, PA, USA
| | - Chris Tjoa
- Community Behavioral Health, Philadelphia, PA, USA
| | - Kyle Kampman
- Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
| | - David T Jones
- Department of Behavioral Health and Intellectual disAbility Services, Philadelphia, PA, USA
| | - David S Mandell
- Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
| |
Collapse
|
12
|
Hoagwood KE, Purtle J, Spandorfer J, Peth-Pierce R, Horwitz SM. Aligning dissemination and implementation science with health policies to improve children's mental health. AMERICAN PSYCHOLOGIST 2020; 75:1130-1145. [PMID: 33252950 PMCID: PMC8034490 DOI: 10.1037/amp0000706] [Citation(s) in RCA: 43] [Impact Index Per Article: 8.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
The prevalence of mental health problems among children (ages 0-21) in the United States remains unacceptably high and, post-COVID-19, is expected to increase dramatically. Decades of psychological knowledge about effective treatments should inform the delivery of better services. Dissemination and implementation (D&I) science has been heralded as a solution to the persistent problem of poor quality services and has, to some extent, improved our understanding of the contexts of delivery systems that implement effective practices. However, there are few studies demonstrating clear, population-level impacts of psychological interventions on children. Momentum is growing among communities, cities, states, and some federal agencies to build "health in all policies" to address broad familial, social, and economic factors known to affect children's healthy development and mental health. These health policy initiatives offer a rare opportunity to repurpose D&I science, shifting it from a primary focus on evidence-based practice implementation, to a focus on policy development and implementation to support child and family health and well-being. This shift is critical as states develop policy responses to address the health and mental health impacts of the COVID-19 pandemic on already-vulnerable families. We provide a typology for building research on D&I and children's mental health policy. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
Collapse
Affiliation(s)
| | - Jonathan Purtle
- Department of Health Management and Policy, Dornsife School of Public Health, Drexel University
| | - Julia Spandorfer
- Department of Child and Adolescent Psychiatry, New York University Langone Health
| | | | - Sarah McCue Horwitz
- Department of Child and Adolescent Psychiatry, New York University Langone Health
| |
Collapse
|
13
|
Graaf G, Snowden L. State Strategies for Enhancing Access and Quality in Systems of Care for Youth with Complex Behavioral Health Needs. ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2020; 48:185-200. [PMID: 32638137 DOI: 10.1007/s10488-020-01061-y] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/21/2022]
Abstract
The investigators conducted a qualitative study with state administrators and policymakers about the financing and policies that structure public systems of care for children with complex behavioral healthcare needs. The objective was to characterize diverse strategies states employed to enhance funding for, access to, and quality and effectiveness of home and community-based services (HCBS) for these youth. States report using a wide variety of creative solutions while navigating the unique needs and constraints of their political and economic environments. Findings can benefit public officials, researchers, and advocates by advancing knowledge-sharing of public policy-making and resourceful problem-solving.
Collapse
Affiliation(s)
- Genevieve Graaf
- School of Social Work, University of Texas At Arlington, Social Work Complex - A, 112D, 211 South Cooper Street, Box 19129, Arlington, TX, 76019, USA.
| | - Lonnie Snowden
- School of Public Health, University of California, Berkeley, USA
| |
Collapse
|
14
|
Stone EM, Daumit GL, Kennedy-Hendricks A, McGinty EE. The Policy Ecology of Behavioral Health Homes: Case Study of Maryland's Medicaid Health Home Program. ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2020; 47:60-72. [PMID: 31506860 PMCID: PMC7040852 DOI: 10.1007/s10488-019-00973-8] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/02/2023]
Abstract
Behavioral health homes, shown to improve receipt of evidence-based medical services among people with serious mental illness in randomized clinical trials, have had limited results in real-world settings; nonetheless, these programs are spreading rapidly. To date, no studies have considered what set of policies is needed to support effective implementation of these programs. As a first step toward identifying an optimal set of policies to support behavioral health home implementation, we use the policy ecology framework to map the policies surrounding Maryland's Medicaid behavioral health home program. Results suggest that existing policies fail to address important implementation barriers.
Collapse
Affiliation(s)
- Elizabeth M Stone
- Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA.
| | - Gail L Daumit
- Division of General Internal Medicine, Johns Hopkins School of Medicine, Baltimore, MD, USA
| | - Alene Kennedy-Hendricks
- Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
| | - Emma E McGinty
- Department of Health Policy and Management, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
| |
Collapse
|
15
|
Purtle J, Lê-Scherban F, Nelson KL, Shattuck PT, Proctor EK, Brownson RC. State mental health agency officials' preferences for and sources of behavioral health research. Psychol Serv 2020; 17:93-97. [PMID: 31192673 PMCID: PMC6908783 DOI: 10.1037/ser0000364] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/05/2023]
Abstract
State mental health agencies (SMHAs) are integral to public behavioral health service systems. As such, senior-level officials within SMHAs are important targets for advocacy and dissemination of behavioral health research findings. Evidence-informed decision making in SMHAs can potentially be enhanced by developing summaries of behavioral health research (e.g., policy briefs) that reflect SMHA officials' information preferences, but knowledge about these preferences is lacking. An exploratory study was conducted with the aims of characterizing senior-level SMHA officials' preferences for behavioral health research and describing where they turn for this research when making policy decisions. A cross-sectional, web-based survey of senior-level SMHA officials (1 per state) was conducted in March-May 2017 (n = 43, response rate = 84%). The features of behavioral health research that SMHA officials identified as "very important" most frequently were research being relevant to state residents (93.0%), providing data on cost-effectiveness (86.0%) and budget impact (81.4%), and being presented concisely (81.0%). The primary sources that SMHA officials turned to for behavioral research when making policy decisions were professional organizations (79.1%), SMHA agency staff (60.5%), and university researchers (55.8%). Compared with state legislators' responses to the same survey questions, results suggest that senior-level SMHA officials and legislators have similar preferences for behavioral health research but turn to different sources for this research. Advocates and researchers who seek to promote evidence-informed decision making in SMHAs should consider developing policy briefs that are concise, provide state-level prevalence data about behavioral conditions, and contain economic evaluation data, and they should disseminate these materials to multiple sources. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
Collapse
|
16
|
Beidas RS, Williams NJ, Becker-Haimes EM, Aarons GA, Barg FK, Evans AC, Jackson K, Jones D, Hadley T, Hoagwood K, Marcus SC, Neimark G, Rubin RM, Schoenwald SK, Adams DR, Walsh LM, Zentgraf K, Mandell DS. A repeated cross-sectional study of clinicians' use of psychotherapy techniques during 5 years of a system-wide effort to implement evidence-based practices in Philadelphia. Implement Sci 2019; 14:67. [PMID: 31226992 PMCID: PMC6588873 DOI: 10.1186/s13012-019-0912-4] [Citation(s) in RCA: 33] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2018] [Accepted: 06/03/2019] [Indexed: 02/06/2023] Open
Abstract
BACKGROUND Little work investigates the effect of behavioral health system efforts to increase use of evidence-based practices or how organizational characteristics moderate the effect of these efforts. The objective of this study was to investigate clinician practice change in a system encouraging implementation of evidence-based practices over 5 years and how organizational characteristics moderate this effect. We hypothesized that evidence-based techniques would increase over time, whereas use of non-evidence-based techniques would remain static. METHOD Using a repeated cross-sectional design, data were collected three times from 2013 to 2017 in Philadelphia's public behavioral health system. Clinicians from 20 behavioral health outpatient clinics serving youth were surveyed three times over 5 years (n = 340; overall response rate = 60%). All organizations and clinicians were exposed to system-level support provided by the Evidence-based Practice Innovation Center from 2013 to 2017. Additionally, approximately half of the clinicians participated in city-funded evidence-based practice training initiatives. The main outcome included clinician self-reported use of cognitive-behavioral and psychodynamic techniques measured by the Therapy Procedures Checklist-Family Revised. RESULTS Clinicians were 80% female and averaged 37.52 years of age (SD = 11.40); there were no significant differences in clinician characteristics across waves (all ps > .05). Controlling for organizational and clinician covariates, average use of CBT techniques increased by 6% from wave 1 (M = 3.18) to wave 3 (M = 3.37, p = .021, d = .29), compared to no change in psychodynamic techniques (p = .570). Each evidence-based practice training initiative in which clinicians participated predicted a 3% increase in CBT use (p = .019) but no change in psychodynamic technique use (p = .709). In organizations with more proficient cultures at baseline, clinicians exhibited greater increases in CBT use compared to organizations with less proficient cultures (8% increase vs. 2% decrease, p = .048). CONCLUSIONS System implementation of evidence-based practices is associated with modest changes in clinician practice; these effects are moderated by organizational characteristics. Findings identify preliminary targets to improve implementation.
Collapse
Affiliation(s)
- Rinad S. Beidas
- Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA USA
- Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA USA
- Penn Implementation Science Center at the Leonard Davis Institute of Health Economics (PISCE@LDI), University of Pennsylvania, Hall- Mercer Community Mental Health Center, Philadelphia, PA USA
| | | | - Emily M. Becker-Haimes
- Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA USA
- Hall- Mercer Community Mental Health Center, Philadelphia, FL USA
| | - Gregory A. Aarons
- Department of Psychiatry, University of California San Diego, San Diego, CA USA
| | - Frances K. Barg
- Department of Family Medicine and Community Health, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA USA
| | | | - Kamilah Jackson
- Community Behavioral Health, Impact Reach, LLC, Philadelphia, PA USA
| | - David Jones
- Department of Behavioral Health, Philadelphia, PA USA
| | - Trevor Hadley
- Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA USA
| | - Kimberly Hoagwood
- Department of Child and Adolescent Psychiatry, New York University Langone Health, New York, NY USA
| | - Steven C. Marcus
- School of Social Policy and Practice, University of Pennsylvania, Philadelphia, PA USA
| | - Geoffrey Neimark
- Community Behavioral Health, Impact Reach, LLC, Philadelphia, PA USA
| | - Ronnie M. Rubin
- Community Behavioral Health, Impact Reach, LLC, Philadelphia, PA USA
- Impact Reach, LLC, Philadelphia, PA USA
| | | | - Danielle R. Adams
- School of Social Service Administration, University of Chicago, Chicago, IL USA
| | - Lucia M. Walsh
- Department of Psychology, University of Miami, Miami, FL USA
| | - Kelly Zentgraf
- Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA USA
| | - David S. Mandell
- Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA USA
- Penn Implementation Science Center at the Leonard Davis Institute of Health Economics (PISCE@LDI), University of Pennsylvania, Hall- Mercer Community Mental Health Center, Philadelphia, PA USA
| |
Collapse
|