Measuring sustainability of a grassroots program in a large integrated health care delivery system: the Warrior to Soul Mate Program

Hospital Adaptions to Mitigate the COVID-19 Pandemic Effects on MARQUIS Toolkit Implementation and Sustainability

OBJECTIVE

To explore the perceived effects of COVID-19 on MARQUIS toolkit implementation and sustainability, challenges faced by hospitals in sustaining medication reconciliation efforts, and the strategies used to mitigate the negative effects of the pandemic.

DATA SOURCES AND STUDY SETTINGS

Primary qualitative data were extracted from a Web-based survey. Data were collected from hospitals that participated in MARQUIS2 ( n = 18) and the MARQUIS Collaborative ( n = 5).

STUDY DESIGN

A qualitative, cross-sectional study was conducted.

DATA COLLECTION/DATA EXTRACTION

Qualitative data were extracted from a Research Electronic Data Capture survey databased and uploaded into an Excel data analysis template. Two coders independently coded the data with a third coder resolving discrepancies.

PRINCIPAL FINDINGS

Thirty-one team members participated, including pharmacists ( n = 20; 65%), physicians ( n = 9; 29%), or quality-improvement (QI) specialists ( n = 2; 6%) with expertise in medication reconciliation (MedRec) (14; 45%) or QI (10; 32%). Organizational resources were limited, including funding, staffing, and access to pharmacy students. To support program continuation, hospitals reallocated staff and used new MedRec order sets. Telemedicine, workflow adaptations, leadership support, QI team involvement, and ongoing audits and feedback promoted toolkit sustainability.

CONCLUSIONS

COVID-19 affected the capacity of hospitals to sustain the MARQUIS toolkit. However, hospitals adapted various strategies to sustain the toolkit.

Using the RE-AIM framework to assess national teledermatology expansion

Background

Teledermatology has been utilized in the United States Department of Veterans Affairs (VA) for decades but continues to have incomplete penetration. VA has funded an initiative to enhance access to dermatology services since 2017 to support asynchronous teledermatology for Veterans living in rural areas. As part of an ongoing evaluation of this program, we assessed the teledermatology activity between the fiscal years 2020 and 2022. We focused on the second cohort of the initiative, comprising six VA facilities and their 54 referral clinics.

Methods

We studied teledermatology programs at cohort facilities using the reach, effectiveness, adoption, implementation, and maintenance framework. We used a mixed-methods design including annual online reports completed by participating facilities and VA administrative data. When possible, we compared the data from the 3 years of teledermatology funding with the baseline year prior to the start of funding.

Findings

Reach: Compared with the baseline year, there was a 100% increase in encounters and a 62% increase in patients seen at the funded facilities. Over 500 clinicians and support staff members were trained. Effectiveness: In FY 2022, primary or specialty care clinics affiliated with the funded facilities had more dermatology programs than primary or specialty care clinics across the VA (83% vs. 71% of sites). Adoption: By the end of the funding period, teledermatology constituted 16% of dermatology encounters at the funded facilities compared with 12% nationally. This reflected an increase from 9.2% at the funded facilities and 10.3% nationally prior to the funding period. Implementation: The continued funding for staff and equipment facilitated the expansion to rural areas. Maintenance: By the end of the funding period, all facilities indicated that they had fully implemented their program for patients of targeted primary care providers. The Program Sustainability Index scores generally increased during the funding period.

Conclusions

Targeted funding to support asynchronous teledermatology implementation for rural Veterans increased its reach, adoption, and implementation, ultimately improving access. Providing program guidance with staffing and training resources can increase the impact of these programs. Ongoing efforts to maintain and increase communication between primary care and dermatology will be needed to sustain success.

Sustainment of diverse evidence-informed practices disseminated in the Veterans Health Administration (VHA): initial development and piloting of a pragmatic survey tool

BACKGROUND

There are challenges associated with measuring sustainment of evidence-informed practices (EIPs). First, the terms sustainability and sustainment are often falsely conflated: sustainability assesses the likelihood of an EIP being in use in the future while sustainment assesses the extent to which an EIP is (or is not) in use. Second, grant funding often ends before sustainment can be assessed. The Veterans Health Administration (VHA) Diffusion of Excellence (DoE) program is one of few large-scale models of diffusion; it seeks to identify and disseminate practices across the VHA system. The DoE sponsors "Shark Tank" competitions, in which leaders bid on the opportunity to implement a practice with approximately 6 months of implementation support. As part of an ongoing evaluation of the DoE, we sought to develop and pilot a pragmatic survey tool to assess sustainment of DoE practices.

METHODS

In June 2020, surveys were sent to 64 facilities that were part of the DoE evaluation. We began analysis by comparing alignment of quantitative and qualitative responses; some facility representatives reported in the open-text box of the survey that their practice was on a temporary hold due to COVID-19 but answered the primary outcome question differently. As a result, the team reclassified the primary outcome of these facilities to Sustained: Temporary COVID-Hold. Following this reclassification, the number and percent of facilities in each category was calculated. We used directed content analysis, guided by the Consolidated Framework for Implementation Research (CFIR), to analyze open-text box responses.

RESULTS

A representative from forty-one facilities (64%) completed the survey. Among responding facilities, 29/41 sustained their practice, 1/41 partially sustained their practice, 8/41 had not sustained their practice, and 3/41 had never implemented their practice. Sustainment rates increased between Cohorts 1-4.

CONCLUSIONS

The initial development and piloting of our pragmatic survey allowed us to assess sustainment of DoE practices. Planned updates to the survey will enable flexibility in assessing sustainment and its determinants at any phase after adoption. This assessment approach can flex with the longitudinal and dynamic nature of sustainment, including capturing nuances in outcomes when practices are on a temporary hold. If additional piloting illustrates the survey is useful, we plan to assess the reliability and validity of this measure for broader use in the field.

Dynamic Diffusion Network: Advancing moral injury care and suicide prevention using an innovative model

Healthcare providers across a wide variety of settings face a common challenge: the need to provide real time care for complex problems that are not adequately addressed by existing protocols. In response to these intervention gaps, frontline providers may utilize existing evidence to develop new approaches that are tailored to specific problems. It is imperative that such approaches undergo some form of evaluation, ensuring quality control while permitting ongoing adaptation and refinement. “Dynamic diffusion” is an innovative approach to intervention improvement and dissemination whereby care practices are delivered and continuously evaluated under real-world conditions as part of a structured network experience. This “dynamic diffusion network” (DDN) promotes cross-pollination of ideas and shared learning to generate relatively rapid improvements in care. The pilot Mental Health and Chaplaincy DDN was developed to advance suicide prevention efforts and moral injury care practices being conducted by 13 chaplain-mental health professional teams across the Veterans Health Administration. Lessons learned from the pilot DDN include the importance of the following: geographic and cultural diversity among innovation collaborators to ensure the broadest possible relevance of solutions; leadership support to facilitate engagement of frontline providers in quality improvement efforts; and participation in a community of practice to motivate providers and offer opportunities for direct collaboration and cross-pollination of ideas.

Implementation and sustainment of diverse practices in a large integrated health system: a mixed methods study

BACKGROUND

One goal of health systems seeking to evolve into learning health systems is to accelerate the implementation and sustainment of evidence-based practices (EBPs). As part of this evolution, the Veterans Health Administration (VHA) developed the Innovation Ecosystem, which includes the Diffusion of Excellence (DoE), a program that identifies and diffuses Gold Status Practices (GSPs) across facilities. The DoE hosts an annual "Shark Tank" competition in which leaders bid on the opportunity to implement a GSP with 6 months of implementation support. Over 750 diverse practices were submitted in cohorts 2 and 3 of Shark Tank; 23 were designated GSPs and were implemented in 31 VA networks or facilities. As part of a national evaluation of the DoE, we identified factors contributing to GSP implementation and sustainment.

METHODS

Our sequential mixed methods evaluation of cohorts 2 and 3 of Shark Tank included semi-structured interviews with at least one representative from 30/31 implementing teams (N = 78/105 people invited) and survey responses from 29/31 teams (N = 39/47 invited). Interviews focused on factors influencing implementation and future sustainment. Surveys focused on sustainment 1.5-2 years after implementation. The Consolidated Framework for Implementation Research (CFIR) informed data collection and directed content analysis. Ordinal scales were developed inductively to rank implementation and sustainment outcomes.

RESULTS

Over 50% of teams (17/30) successfully implemented their GSP within the 6-month implementation period. Despite extensive implementation support, significant barriers related to centralized decision-making, staffing, and resources led to partial (n = 6) or no (n = 7) implementation for the remaining teams. While 12/17 initially successful implementation teams reported sustained use of their GSP, over half of the initially unsuccessful teams (n = 7/13) also reported sustained GSP use 1.5 years after the initial implementation period. When asked at 6 months, 18/27 teams with complete data accurately anticipated their future sustainability based on reported sustainment an average of 1.5 years later.

CONCLUSIONS

Most teams implemented within 6 months and/or sustained their GSP 1.5 years later. High levels of implementation and sustainment across diverse practices and teams suggest that VHA's DoE is a successful large-scale model of diffusion. Team predictions about sustainability after the first 6 months of implementation provide a promising early assessment and point of intervention to increase sustainability.