1
|
Dopp AR, Hunter SB, Godley MD, González I, Bongard M, Han B, Cantor J, Hindmarch G, Lindquist K, Wright B, Schlang D, Passetti LL, Wright KL, Kilmer B, Aarons GA, Purtle J. Comparing organization-focused and state-focused financing strategies on provider-level reach of a youth substance use treatment model: a mixed-method study. Implement Sci 2023; 18:50. [PMID: 37828518 PMCID: PMC10571404 DOI: 10.1186/s13012-023-01305-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2023] [Accepted: 09/18/2023] [Indexed: 10/14/2023] Open
Abstract
BACKGROUND Financial barriers in substance use disorder service systems have limited the widespread adoption-i.e., provider-level reach-of evidence-based practices (EBPs) for youth substance use disorders. Reach is essential to maximizing the population-level impact of EBPs. One promising, but rarely studied, type of implementation strategy for overcoming barriers to EBP reach is financing strategies, which direct financial resources in various ways to support implementation. We evaluated financing strategies for the Adolescent Community Reinforcement Approach (A-CRA) EBP by comparing two US federal grant mechanisms, organization-focused and state-focused grants, on organization-level A-CRA reach outcomes. METHOD A-CRA implementation took place through organization-focused and state-focused grantee cohorts from 2006 to 2021. We used a quasi-experimental, mixed-method design to compare reach between treatment organizations funded by organization-focused versus state-focused grants (164 organizations, 35 states). Using administrative training records, we calculated reach as the per-organization proportion of trained individuals who received certification in A-CRA clinical delivery and/or supervision by the end of grant funding. We tested differences in certification rate by grant type using multivariable linear regression models that controlled for key covariates (e.g., time), and tested threats to internal validity from our quasi-experimental design through a series of sensitivity analyses. We also drew on interviews and surveys collected from the treatment organizations and (when relevant) interviews with state administrators to identify factors that influenced reach. RESULTS The overall certification rates were 27 percentage points lower in state-focused versus organization-focused grants (p = .01). Sensitivity analyses suggested these findings were not explained by confounding temporal trends nor by organizational or state characteristics. We did not identify significant quantitative moderators of reach outcomes, but qualitative findings suggested certain facilitating factors were more influential for organization-focused grants (e.g., strategic planning) and certain barrier factors were more impactful for state-focused grants (e.g., states finding it difficult to execute grant activities). DISCUSSION As the first published comparison of EBP reach outcomes between financing strategies, our findings can help guide state and federal policy related to financing strategies for implementing EBPs that reduce youth substance use. Future work should explore contextual conditions under which different financing strategies can support the widespread implementation of EBPs for substance use disorder treatment.
Collapse
Affiliation(s)
- Alex R Dopp
- RAND Corporation, 1776 Main Street, Santa Monica, CA, 90401, USA.
| | - Sarah B Hunter
- RAND Corporation, 1776 Main Street, Santa Monica, CA, 90401, USA
| | - Mark D Godley
- Chestnut Health Systems, 448 Wylie Drive, Normal, IL, 61761, USA
| | | | - Michelle Bongard
- RAND Corporation, 1776 Main Street, Santa Monica, CA, 90401, USA
| | - Bing Han
- Department of Research and Evaluation, Division of Biostatistics Research, Kaiser Permanente Southern California, 100 South Los Robles Avenue 2nd Floor, Pasadena, CA, 91101, USA
| | - Jonathan Cantor
- RAND Corporation, 1776 Main Street, Santa Monica, CA, 90401, USA
| | - Grace Hindmarch
- RAND Corporation, 1776 Main Street, Santa Monica, CA, 90401, USA
| | - Kerry Lindquist
- RAND Corporation, 1776 Main Street, Santa Monica, CA, 90401, USA
| | - Blanche Wright
- RAND Corporation, 1776 Main Street, Santa Monica, CA, 90401, USA
- Department of Health Policy and Management, University of California Los Angeles, 650 Charles Young Dr. S., 31-269 CHS Box 951772, Los Angeles, CA, 90095, USA
| | - Danielle Schlang
- RAND Corporation, 1776 Main Street, Santa Monica, CA, 90401, USA
| | - Lora L Passetti
- Chestnut Health Systems, 448 Wylie Drive, Normal, IL, 61761, USA
| | - Kelli L Wright
- Chestnut Health Systems, 448 Wylie Drive, Normal, IL, 61761, USA
| | - Beau Kilmer
- RAND Corporation, 1776 Main Street, Santa Monica, CA, 90401, USA
| | - Gregory A Aarons
- Department of Psychiatry and Altman Clinical and Translational Research Institute Dissemination and Implementation Science Center, University of California San Diego, 9500 Gilman Dr. (0812), La Jolla, San Diego, CA, 92093, USA
| | - Jonathan Purtle
- Department of Public Health Policy & Management and Global Center for Implementation Science, New York University School of Global Public Health, 708 Broadway, New York, NY, 10003, USA
| |
Collapse
|
2
|
Hanbury A, Mafirakureva N, Chicken N, Bailey L. Evaluating change in a pressured healthcare system: a cross-sectional study of implementation outcomes using routine data indicators and proxies. Implement Sci Commun 2023; 4:96. [PMID: 37587530 PMCID: PMC10428631 DOI: 10.1186/s43058-023-00471-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2023] [Accepted: 07/15/2023] [Indexed: 08/18/2023] Open
Abstract
BACKGROUND Implementation evaluation should focus on implementation success, guided by theories and frameworks. With high staff vacancies in the health services, it is important to consider pragmatic methods of data collection for implementation evaluation. This paper presents a cross-sectional rapid evaluation of a handheld medical device designed for remote examinations, piloted in Northern England. By using downloaded device data and administrative records mapped to domains from the implementation outcomes framework, this evaluation offers a pragmatic example of assessing implementation success. METHODS The pilot design was pragmatic: sites volunteered, decided which services to use the device in, and launched when ready. The pilot and evaluation together lasted 1 year. Data was downloaded from the devices, and administrative records for the pilot accessed. Variables were mapped to five of the implementation outcomes, after reviewing with the device manufacturer and pilot team to assess robustness. RESULTS N=352 care episodes were recorded using the device with 223 patients. Out of 19 sites 'signed up' to the pilot, 5 launched and delivered 10 of 35 proposed projects: a site and project adoption rate of 26 and 29%, respectively. Six sites signed up to an extension period; three had launched and three had not during the original timelines, indicating some sustainability. Feasibility was high, with only one in seven care episodes needing to be repeated due to poor device quality or error (sound/audio/internet). Fidelity of device usage was low for two of the eight available device examinations. Device and staffing costs were high but potential cost savings were attributable to fewer in-person appointments. CONCLUSIONS Through using device and administrative data, this evaluation minimised burden on busy healthcare staff yet was still guided by an evaluation framework. Five out of the eight implementation outcomes were measured, including sustainability and costs. The findings give insight into implementation challenges, particularly around adoption. For future research, it is recommended to engage with staff to prioritise outcome measurements and to focus on meaningful interpretation of indicators.
Collapse
Affiliation(s)
- Andria Hanbury
- Yorkshire and Humber Applied Research Collaboration, Bradford Institute for Health Research, Bradford, UK.
| | - Nyasha Mafirakureva
- School of Health and Related Research, University of Sheffield, Sheffield, UK
| | - Nicola Chicken
- Yorkshire and Humber Academic Health Science Network, Wakefield, UK
| | - Liam Bailey
- Yorkshire Quality and Safety Research Group, Bradford Institute for Health Research, Bradford, UK
| |
Collapse
|
3
|
Program Evaluation of Implementation Science Outcomes From an Intervention to Improve Compliance With Chlorhexidine Gluconate Bathing: A Qualitative Study. Dimens Crit Care Nurs 2022; 41:200-208. [PMID: 35617584 DOI: 10.1097/dcc.0000000000000530] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022] Open
Abstract
BACKGROUND AND OBJECTIVES Evaluation of implementation science research is warranted to better understand and determine the success of translating evidence-based infection prevention practices at the bedside. The purpose of this program evaluation was to evaluate implementation outcomes from the perspectives of nurses and nursing leaders regarding a previously conducted chlorhexidine gluconate (CHG) bathing implementation science study among 14 critical care units. METHODS Focus groups and interviews, using semistructured interview questions, were conducted to examine the perceptions of nurses who participated in a CHG bathing implementation science study. A deductive qualitative analysis using Proctor and colleagues' implementation outcomes framework was used. Transcripts were analyzed and categorized using the framework as a predetermined code list to structure the implementation outcomes of acceptability, appropriateness, adoption, feasibility, and sustainability. FINDINGS A total of 19 nurses and nurse leaders participated in a focus group or interview. Participants noted that both implementation strategies used in the initial study (educational outreach and audit and feedback) were acceptable and appropriate and expressed that the evidence-based CHG bathing practice was feasible to integrate into practice and was being adopted. DISCUSSION The program evaluation identified strengths and opportunities for improvement related to the implementation strategies and evidence-based CHG bathing protocol. Findings can inform future studies that seek to implement CHG bathing protocols in the critical care setting using audit and feedback and educational outreach strategies.
Collapse
|
4
|
Dopp AR, Hunter SB, Godley MD, Pham C, Han B, Smart R, Cantor J, Kilmer B, Hindmarch G, González I, Passetti LL, Wright KL, Aarons GA, Purtle J. Comparing two federal financing strategies on penetration and sustainment of the adolescent community reinforcement approach for substance use disorders: protocol for a mixed-method study. Implement Sci Commun 2022; 3:51. [PMID: 35562836 PMCID: PMC9099033 DOI: 10.1186/s43058-022-00298-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2022] [Accepted: 04/21/2022] [Indexed: 01/09/2023] Open
Abstract
BACKGROUND Sustained, widespread availability of evidence-based practices (EBPs) is essential to address the public health and societal impacts of adolescent substance use disorders (SUD). There remains a particularly significant need to identify effective financing strategies, which secure and direct financial resources to support the costs associated with EBP implementation and sustainment. This protocol describes a new project comparing two types of U.S. federal grant mechanisms (i.e., a type of financing strategy), which supported the implementation of the Adolescent Community Reinforcement Approach (A-CRA) EBP for SUD, through either organization-focused or state-focused granting of funds. The Exploration-Preparation-Implementation-Sustainment (EPIS) framework will guide our study aims, hypotheses, and selection of measures. METHOD We will employ a longitudinal, mixed-method (i.e., web surveys, semi-structured interviews, document review, focus groups, administrative data), quasi-experimental design to compare the grant types' outcomes and examine theoretically informed mediators and moderators. Aim 1 will examine the proportion of eligible clinicians certified in A-CRA with adequate fidelity levels (i.e., penetration outcomes) at the end of grant funding. Aim 2 will examine the sustainment of A-CRA up to 5 years post-funding, using a 10-element composite measure of treatment delivery and supervision activities. We will integrate the new data collected from state-focused grant recipients (~85 organizations in 19 states) with previously collected data from organization-focused grant recipients (Hunter et al., Implement Sci 9:104, 2014) (82 organizations in 26 states) for analysis. We will also use sensitivity analyses to characterize the effects of observed and unobserved secular trends in our quasi-experimental design. Finally, aim 3 will use comparative case study methods (integrating diverse quantitative and qualitative measures) to identify and disseminate policy implications about the roles of state- and organization-focused federal grants in efforts to promote adolescent SUD EBP implementation and sustainment. DISCUSSION The proposed research will have direct, practical implications for behavioral health administrators, policymakers, implementation experts, and the public. It will offer new knowledge that can directly inform financing strategies to support large-scale, sustained EBP delivery in behavioral health-while advancing implementation science through the use of novel methods to study financing strategies and sustainment.
Collapse
Affiliation(s)
- Alex R Dopp
- RAND Corporation, 1776 Main Street, Santa Monica, CA, 90401, USA.
| | - Sarah B Hunter
- RAND Corporation, 1776 Main Street, Santa Monica, CA, 90401, USA
| | - Mark D Godley
- Chestnut Health Systems, 448 Wylie Drive, Normal, IL, 61761, USA
| | - Chau Pham
- RAND Corporation, 1776 Main Street, Santa Monica, CA, 90401, USA
| | - Bing Han
- Department of Research and Evaluation, Division of Biostatistics Research, Kaiser Permanente Southern California, 100 South Los Robles Avenue 2nd Floor, Pasadena, CA, 91101, USA
| | - Rosanna Smart
- RAND Corporation, 1776 Main Street, Santa Monica, CA, 90401, USA
| | - Jonathan Cantor
- RAND Corporation, 1776 Main Street, Santa Monica, CA, 90401, USA
| | - Beau Kilmer
- RAND Corporation, 1776 Main Street, Santa Monica, CA, 90401, USA
| | - Grace Hindmarch
- RAND Corporation, 1776 Main Street, Santa Monica, CA, 90401, USA
| | - Isabelle González
- RAND Corporation, 1200 South Hayes Street, Arlington, VA, 22202, USA
| | - Lora L Passetti
- Chestnut Health Systems, 448 Wylie Drive, Normal, IL, 61761, USA
| | - Kelli L Wright
- Chestnut Health Systems, 448 Wylie Drive, Normal, IL, 61761, USA
| | - Gregory A Aarons
- Department of Psychiatry, 9500 Gilman Dr. (0812), University of California San Diego, La Jolla, CA, 92093, USA
- UC San Diego Altman Clinical and Translational Research Institute Dissemination and Implementation Science Center, La Jolla, CA, 92093, USA
| | - Jonathan Purtle
- Department of Public Health Policy & Management and Global Center for Implementation Science, New York University School of Global Public Health, 708 Broadway, New York, NY, 10003, USA
| |
Collapse
|
5
|
Becker SJ, Murphy CM, Hartzler B, Rash CJ, Janssen T, Roosa M, Madden LM, Garner BR. Project MIMIC (Maximizing Implementation of Motivational Incentives in Clinics): A cluster-randomized type 3 hybrid effectiveness-implementation trial. Addict Sci Clin Pract 2021; 16:61. [PMID: 34635178 PMCID: PMC8505014 DOI: 10.1186/s13722-021-00268-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2021] [Accepted: 09/14/2021] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Opioid-related overdoses and harms have been declared a public health emergency in the United States, highlighting an urgent need to implement evidence-based treatments. Contingency management (CM) is one of the most effective behavioral interventions when delivered in combination with medication for opioid use disorder, but its implementation in opioid treatment programs is woefully limited. Project MIMIC (Maximizing Implementation of Motivational Incentives in Clinics) was funded by the National Institute on Drug Abuse to identify effective strategies for helping opioid treatment programs improve CM implementation as an adjunct to medication. Specific aims will test the impact of two different strategies on implementation outcomes (primary aim) and patient outcomes (secondary aims), as well as test putative mediators of implementation effectiveness (exploratory aim). METHODS A 3-cohort, cluster-randomized, type 3 hybrid design is used with the opioid treatment programs as the unit of randomization. Thirty programs are randomized to one of two conditions. The control condition is the Addiction Technology Transfer Center (ATTC) Network implementation strategy, which consists of three core approaches: didactic training, performance feedback, and on-going consultation. The experimental condition is an enhanced ATTC strategy, with the same core ATTC elements plus two additional theory-driven elements. The two additional elements are Pay-for-Performance, which aims to increase implementing staff's extrinsic motivations, and Implementation & Sustainment Facilitation, which targets staff's intrinsic motivations. Data will be collected using a novel, CM Tracker tool to document CM session delivery, session audio recordings, provider surveys, and patient surveys. Implementation outcomes include CM Exposure (number of CM sessions delivered per patient), CM Skill (ratings of CM fidelity), and CM Sustainment (number of patients receiving CM after removal of support). Patient outcomes include self-reported opioid abstinence and opioid-related problems (both assessed at 3- and 6-months post-baseline). DISCUSSION There is urgent public health need to improve the implementation of CM as an adjunct to medication for opioid use disorder. Consistent with its hybrid type 3 design, Project MIMIC is advancing implementation science by comparing impacts of these two multifaceted strategies on both implementation and patient outcomes, and by examining the extent to which the impacts of those strategies can be explained by putative mediators. TRIAL REGISTRATION This clinical trial has been registered with clinicaltrials.gov (NCT03931174). Registered April 30, 2019. https://clinicaltrials.gov/ct2/show/NCT03931174?term=project+mimic&draw=2&rank=1.
Collapse
Affiliation(s)
- Sara J. Becker
- Center for Alcohol and Addiction Studies, Brown University School of Public Health, Box G-S121-5, Providence, RI 02912 USA
| | - Cara M. Murphy
- Center for Alcohol and Addiction Studies, Brown University School of Public Health, Box G-S121-5, Providence, RI 02912 USA
| | - Bryan Hartzler
- Addictions, Drug, & Alcohol Institute, University of Washington, 1107 NE 45th Street, Suite 120, Seattle, WA 98105 USA
| | - Carla J. Rash
- Calhoun Cardiology Center - Behavioral Health, UConn Health, 263 Farmington Avenue, Farmington, CT 06030 USA
| | - Tim Janssen
- Center for Alcohol and Addiction Studies, Brown University School of Public Health, Box G-S121-5, Providence, RI 02912 USA
| | - Mat Roosa
- Roosa Consulting, LLC, 3 Bradford Drive, Syracuse, NY 13224 USA
| | - Lynn M. Madden
- APT Foundation, 1 Long Wharf Drive, Suite 321, New Haven, CT 06511 USA
| | - Bryan R. Garner
- RTI International, 3040 E. Cornwallis Rd.Research Triangle Park, P.O. Box 12194, Durham, NC 27709 USA
| |
Collapse
|
6
|
Garner BR, Patel SV, Kirk MA. Priority domains, aims, and testable hypotheses for implementation research: Protocol for a scoping review and evidence map. Syst Rev 2020; 9:277. [PMID: 33272313 PMCID: PMC7716483 DOI: 10.1186/s13643-020-01535-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/10/2020] [Accepted: 11/17/2020] [Indexed: 11/20/2022] Open
Abstract
BACKGROUND The challenge of implementing evidence-based innovations within practice settings is a significant public health issue that the field of implementation research (IR) is focused on addressing. Significant amounts of funding, time, and effort have been invested in IR to date, yet there remains significant room for advancement, especially regarding IR's development of scientific theories as defined by the National Academy of Sciences (i.e., a comprehensive explanation of the relationship between variables that is supported by a vast body of evidence). Research priority setting (i.e., promoting consensus about areas where research effort will have wide benefits to society) is a key approach to helping accelerate research advancements. Thus, building upon existing IR, general principles of data reduction, and a general framework for moderated mediation, this article identifies four priority domains, three priority aims, and four testable hypotheses for IR, which we organize in the priority aims and testable hypotheses (PATH) diagram. METHODS The objective of this scoping review is to map the extent to which IR has examined the identified PATH priorities to date. Our sample will include IR published in leading implementation-focused journals (i.e., Implementation Science, Implementation Science Communications, and Implementation Research and Practice) between their inception and December 2020. The protocol for the current scoping review and evidence map has been developed in accordance with the approach developed by Arksey and O'Malley and advanced by Levac, Colquhoun, and O'Brien. Because scoping reviews seek to provide an overview of the identified evidence base rather than synthesize findings from across studies, we plan to use our data-charting form to provide a descriptive overview of implementation research to date and summarize the research via one or more summary tables. We will use the PATH diagram to organize a map of the evidence to date. DISCUSSION This scoping review and evidence map is intended to help accelerate IR focused on suggested priority aims and testable hypotheses, which in turn will accelerate IR's development of National Academy of Sciences-defined scientific theories and, subsequently, improvements in public health. SYSTEMATIC REVIEW REGISTRATION Open Science Framework https://osf.io/3vhuj/.
Collapse
Affiliation(s)
- Bryan R Garner
- RTI International, P. O. Box 12194, Research Triangle Park, NC, 27709-2194, USA.
| | - Sheila V Patel
- RTI International, P. O. Box 12194, Research Triangle Park, NC, 27709-2194, USA
| | - M Alexis Kirk
- Centerstone Research Institute, 44 Vantage Way, Suite 400, Nashville, TN, 37228, USA
| |
Collapse
|
7
|
Swan JE, Votaw VR, Stein ER, Witkiewitz K. The Role of Affect in Psychosocial Treatments for Substance Use Disorders. CURRENT ADDICTION REPORTS 2020; 7:108-116. [PMID: 34327114 PMCID: PMC8317473 DOI: 10.1007/s40429-020-00304-0] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
PURPOSE OF REVIEW This paper provides a narrative review of studies published over the past five years that have examined the role of affect, including both affective symptoms and affective disorders, in psychosocial treatments for substance use disorder. RECENT FINDINGS A growing body of literature suggests that affective symptoms and affective disorders may moderate substance use disorder treatment efficacy, mediate the effects of treatment on substance use outcomes, and may be directly changed by substance use disorder treatment. SUMMARY Substance use disorders and affective disorders commonly co-occur, and both affect and affective disorders are associated with substance use disorder treatment outcomes. Future research should continue to examine affect as a moderator, mediator, and outcome of substance use disorder treatments. In particular, new studies that are designed to test precision medicine hypotheses would greatly expand our understanding of the role of affective symptoms and disorders in substance use disorder treatment.
Collapse
Affiliation(s)
- Julia E Swan
- Department of Psychology, University of New Mexico
| | | | | | | |
Collapse
|
8
|
Torres EM, Seijo C, Ehrhart MG, Aarons GA. Validation of a pragmatic measure of implementation citizenship behavior in substance use disorder treatment agencies. J Subst Abuse Treat 2020; 111:47-53. [PMID: 32087838 DOI: 10.1016/j.jsat.2020.01.002] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2019] [Revised: 01/07/2020] [Accepted: 01/07/2020] [Indexed: 11/15/2022]
Abstract
The organizational context in which substance use disorder treatment (SUDT) evidence-based practices (EBPs) are implemented plays a critical role in successful implementation. Employee behaviors that go above and beyond typical job requirements to support EBP implementation have been suggested to facilitate the likelihood of overall implementation success. The current study explored the psychometric properties of the Implementation Citizenship Behavior Scale (ICBS) within SUDT settings. Utilizing a sample of 322 direct service providers and 60 of their respective supervisors from three SUDT agencies, results from a confirmatory factor analysis and construct validity analysis support the use of the ICBS in the SUDT context. Validation of the ICBS provides a useful, pragmatic tool for both researchers and practitioners to assess employee citizenship behavior to support EBP implementation. The ICBS can provide critical insights into how providers respond to organizational context that may facilitate EBP implementation.
Collapse
Affiliation(s)
- Elisa M Torres
- University of California, San Diego, 9500 Gilman Dr, La Jolla, CA 92093, USA; Child and Adolescent Services Research Center, 3020 Children's Way, MC5033, San Diego, CA 92123, USA; George Mason University, 4400 University Drive, MSN 3F5, Fairfax, VA 22030, USA.
| | - Chariz Seijo
- University of California, San Diego, 9500 Gilman Dr, La Jolla, CA 92093, USA; Child and Adolescent Services Research Center, 3020 Children's Way, MC5033, San Diego, CA 92123, USA.
| | - Mark G Ehrhart
- University of Central Florida, 4000 Central Florida Blvd, Orlando, FL 32816, USA.
| | - Gregory A Aarons
- University of California, San Diego, 9500 Gilman Dr, La Jolla, CA 92093, USA; Child and Adolescent Services Research Center, 3020 Children's Way, MC5033, San Diego, CA 92123, USA.
| |
Collapse
|
9
|
Willmeroth T, Wesselborg B, Kuske S. Implementation Outcomes and Indicators as a New Challenge in Health Services Research: A Systematic Scoping Review. INQUIRY: The Journal of Health Care Organization, Provision, and Financing 2020; 56:46958019861257. [PMID: 31347418 PMCID: PMC6661793 DOI: 10.1177/0046958019861257] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
The aim of this systematic scoping review was to identify and analyze indicators
that address implementation quality or success in health care services and to
deduce recommendations for further indicator development. This review was
conducted according to the Joanna Briggs Manual and the PRISMA Statement.
CINAHL, EMBASE, MEDLINE, and PsycINFO were searched. Studies or reviews
published between August 2008 and 2018 that reported monitoring of the quality
or the implementation success in health care services by using indicators based
on continuous variables and proportion-based, ratio-based, standardized
ratio–based, or rate-based variables or indices were included. The records were
screened by title and abstract, and the full-text articles were also
independently double-screened by 3 reviewers for eligibility. In total, 4376
records were identified that resulted in 10 eligible studies, including 67
implementation indicators. There was heterogeneity regarding the theoretical
backgrounds, designs, objectives, settings, and implementation indicators among
the publications. None of the indicators addressed the implementation outcomes
of appropriateness or sustainability. Service implementation
efficiency was identified as an additional outcome. Achieving
consensus in framing implementation outcomes and indicators will be a new
challenge in health services research. Considering the new debates regarding
health care complexity, the further development of indicators based on
complementary qualitative and quantitative approaches is needed.
Collapse
Affiliation(s)
| | | | - Silke Kuske
- 1 Fliedner Fachhochschule Düsseldorf, Germany
| |
Collapse
|
10
|
Kraan AEM, Dijkstra BAG, Markus W. Treatment delivery of the community reinforcement approach in outpatient addiction treatment. EVALUATION AND PROGRAM PLANNING 2018; 70:61-66. [PMID: 30005319 DOI: 10.1016/j.evalprogplan.2018.05.004] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/24/2017] [Revised: 04/26/2018] [Accepted: 05/30/2018] [Indexed: 06/08/2023]
Abstract
OBJECTIVE Treatment model adherence is an important predictor of treatment outcome. In clinical practice evidence-based treatments are delivered in widely varying degrees. This study examines which Community Reinforcement Approach (CRA) procedures are delivered by addiction care therapists and how this is associated with therapist characteristics. METHOD The study integrated two observational designs. Firstly, using a prospective design, 24 therapists registered every CRA procedure delivered during every patient contact over a six month period. Secondly, using a cross-sectional design, personal characteristics of 69 therapists were assessed including their self-reported delivery of CRA procedures and their perceptions with regard to the meaningfulness and complexity of these procedures. RESULTS The number of CRA procedures delivered varied substantially among therapists both at session and patient level. More experienced therapists and those that had received advanced training previously, delivered more CRA procedures. Finally, the delivery of CRA procedures was positively associated with experienced meaningfulness and negatively associated with difficulty. CONCLUSIONS The results confirm the relation between treatment delivery and experienced meaningfulness and difficulty of CRA procedures and provides support for advanced training to enhance the delivery of a wider range of CRA procedures.
Collapse
Affiliation(s)
- Anneleen E M Kraan
- IrisZorg Addiction Treatment, P.O. Box 351, 6800 AJ Arnhem, The Netherlands; Nijmegen Institute for Scientist-Practitioners in Addiction (NISPA), P.O. Box 6909, 6503 GK Nijmegen, The Netherlands.
| | - Boukje A G Dijkstra
- Nijmegen Institute for Scientist-Practitioners in Addiction (NISPA), P.O. Box 6909, 6503 GK Nijmegen, The Netherlands.
| | - Wiebren Markus
- IrisZorg Addiction Treatment, P.O. Box 351, 6800 AJ Arnhem, The Netherlands; Nijmegen Institute for Scientist-Practitioners in Addiction (NISPA), P.O. Box 6909, 6503 GK Nijmegen, The Netherlands.
| |
Collapse
|
11
|
Garner BR, Lwin AK, Strickler GK, Hunter BD, Shepard DS. Pay-for-performance as a cost-effective implementation strategy: results from a cluster randomized trial. Implement Sci 2018; 13:92. [PMID: 29973280 PMCID: PMC6033288 DOI: 10.1186/s13012-018-0774-1] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2017] [Accepted: 05/31/2018] [Indexed: 01/12/2023] Open
Abstract
BACKGROUND Pay-for-performance (P4P) has been recommended as a promising strategy to improve implementation of high-quality care. This study examined the incremental cost-effectiveness of a P4P strategy found to be highly effective in improving the implementation and effectiveness of the Adolescent Community Reinforcement Approach (A-CRA), an evidence-based treatment (EBT) for adolescent substance use disorders (SUDs). METHODS Building on a $30 million national initiative to implement A-CRA in SUD treatment settings, urn randomization was used to assign 29 organizations and their 105 therapists and 1173 patients to one of two conditions (implementation-as-usual (IAU) control condition or IAU+P4P experimental condition). It was not possible to blind organizations, therapists, or all research staff to condition assignment. All treatment organizations and their therapists received a multifaceted implementation strategy. In addition to those IAU strategies, therapists in the IAU+P4P condition received US $50 for each month that they demonstrated competence in treatment delivery (A-CRA competence) and US $200 for each patient who received a specified number of treatment procedures and sessions found to be associated with significantly improved patient outcomes (target A-CRA). Incremental cost-effectiveness ratios (ICERs), which represent the difference between the two conditions in average cost per treatment organization divided by the corresponding average difference in effectiveness per organization, and quality-adjusted life years (QALYs) were the primary outcomes. RESULTS At trial completion, 15 organizations were randomized to the IAU condition and 14 organizations were randomized to the IAU+P4P condition. Data from all 29 organizations were analyzed. Cluster-level analyses suggested the P4P strategy led to significantly higher average total costs compared to the IAU control condition, yet this average increase of 5% resulted in a 116% increase in the average number of months therapists demonstrated competence in treatment delivery (ICER = $333), a 325% increase in the average number of patients who received the targeted dosage of treatment (ICER = $453), and a 325% increase in the number of days of abstinence per patient in treatment (ICER = $8.134). Further supporting P4P as a cost-effective implementation strategy, the cost per QALY was only $8681 (95% confidence interval $1191-$16,171). CONCLUSION This study provides experimental evidence supporting P4P as a cost-effective implementation strategy. TRIAL REGISTRATION NCT01016704 .
Collapse
Affiliation(s)
- Bryan R. Garner
- RTI International, P. O. Box 12194, Research Triangle Park, Raleigh, NC 27709-2194 USA
| | - Aung K. Lwin
- Schneider Institutes for Health Policy, The Heller School, MS035, Brandeis University, Waltham, MA USA
| | - Gail K. Strickler
- Schneider Institutes for Health Policy, The Heller School, MS035, Brandeis University, Waltham, MA USA
| | | | - Donald S. Shepard
- Schneider Institutes for Health Policy, The Heller School, MS035, Brandeis University, Waltham, MA USA
| |
Collapse
|
12
|
Garner BR, Hunter SB, Slaughter ME, Han B, Godley SH. Factors associated with an evidence-based measure of implementation for the Adolescent Community Reinforcement Approach. Drug Alcohol Depend 2017; 180:144-150. [PMID: 28888154 PMCID: PMC6757189 DOI: 10.1016/j.drugalcdep.2017.08.011] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/09/2017] [Revised: 08/10/2017] [Accepted: 08/12/2017] [Indexed: 11/27/2022]
Abstract
BACKGROUND An evidence-based measure of implementation (EBMI) is an implementation outcome measure shown to have predictive validity with one or more future-measured constructs of importance. The current study sought to identify correlates and predictors of an EBMI called procedure exposure. Garner et al. (2016) found procedure exposure to be an EBMI for the Adolescent Community Reinforcement Approach (A-CRA). METHODS The dataset included 76 community-based substance use treatment organizations located across the United States. Organizational-level regression analyses, which were framed within the context of Chaudoir et al. (2013) framework for predicting implementation outcomes, were used to examine predictors of A-CRA procedure exposure RESULTS: The Washington Circle's treatment initiation performance measure (B=5.05 [SE=1.60], p=0.002), as well as session exposure (B=0.18 [SE=0.06], p=0.003), were significant predictors of A-CRA procedure exposure in the backward stepwise regression analysis (Adjusted R-square=0.55). The Washington Circle's treatment engagement performance measure (B=7.93 [SE=0.77], p<0.001), as well as time-to-proficiency (B=-0.04 [SE=0.02], p=0.02), each had significant bivariate relationships with A-CRA procedure exposure but were not retained in the final model. CONCLUSIONS Organizations implementing A-CRA are encouraged to make the following high priorities: (a) scheduling and completing a subsequent treatment session within 14days of their index session (treatment initiation) and (b) providing a targeted number of treatment sessions to each client (session exposure). To the extent organizations do this, they may be more likely to achieve higher levels of A-CRA procedure exposure.
Collapse
Affiliation(s)
- Bryan R. Garner
- RTI International, P. O. Box 12194, Research Triangle Park, NC 27709, USA,Corresponding author. (B.R. Garner)
| | | | | | - Bing Han
- RAND Corp, P.O. Box 2138, Santa Monica, CA 90407, USA.
| | - Susan H. Godley
- Chestnut Health Systems, 448 Wylie Dr., Normal, IL 61761, USA
| |
Collapse
|
13
|
King AA, Baumann AA. Sickle cell disease and implementation science: A partnership to accelerate advances. Pediatr Blood Cancer 2017; 64:10.1002/pbc.26649. [PMID: 28556441 PMCID: PMC6026013 DOI: 10.1002/pbc.26649] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/17/2017] [Revised: 04/05/2017] [Accepted: 04/24/2017] [Indexed: 12/11/2022]
Abstract
Sickle cell disease (SCD) results in end organ damage and a shortened lifespan. Both the pathophysiology of the disease and the social determinants of health affect patient outcomes. Randomized controlled trials have been completed among this population and resulted in medical advances; however, the gestation of these advances and the lack of penetrance into clinical practice have limited advancements in clinical improvements for many people with SCD. We discuss the role of implementation science in SCD and highlight the need for this science to shorten the length of time to implement evidence-based care for more people with SCD.
Collapse
Affiliation(s)
- Allison A. King
- Program in Occupational Therapy, Washington University School of Medicine, St. Louis, Missouri
- Division of Pediatric Hematology/Oncology, Department of Pediatrics, Washington University School of Medicine, St. Louis, Missouri
- Division of Public Health Sciences, Department of Surgery, Washington University School of Medicine, St. Louis, Missouri
- Division of Hematology, Department of Medicine, Washington University School of Medicine, St. Louis, Missouri
| | - Ana A. Baumann
- Brown School, Washington University, St. Louis, Missouri
| |
Collapse
|