Implementation fidelity trajectories of a health promotion program in multidisciplinary settings: managing tensions in rehabilitation care

Champion and audit and feedback strategy fidelity and their relationship to depression intervention fidelity: A mixed method study

Background

Globally, mental health disorders rank as the greatest cause of disability. Low and middle-income countries (LMICs) hold a disproportionate share of the mental health burden, especially as it pertains to depression. Depression is highly prevalent among those with non-communicable diseases (NCDs), creating a barrier to successful treatment. While some treatments have proven efficacy in LMIC settings, wide dissemination is challenged by multiple factors, leading researchers to call for implementation strategies to overcome barriers to care provision. However, implementation strategies are often not well defined or documented, challenging the interpretation of study results and the uptake and replication of strategies in practice settings. Assessing implementation strategy fidelity (ISF), or the extent to which a strategy was implemented as designed, overcomes these challenges. This study assessed fidelity of two implementation strategies (a 'basic' champion strategy and an 'enhanced' champion + audit and feedback strategy) to improve the integration of a depression intervention, measurement based care (MBC), at 10 NCD clinics in Malawi. The primary goal of this study was to assess the relationship between the implementation strategies and MBC fidelity using a mixed methods approach.

Methods

We developed a theory-informed mixed methods fidelity assessment that first combined an implementation strategy specification technique with a fidelity framework. We then created corresponding fidelity indicators to strategy components. Clinical process data and one-on-one in-depth interviews with 45 staff members at 6 clinics were utilized as data sources. Our final analysis used descriptive statistics, reflexive-thematic analysis (RTA), data merging, and triangulation to examine the relationship between ISF and MBC intervention fidelity.

Results

Our mixed methods analysis revealed how ISF may moderate the relationship between the strategies and MBC fidelity. Leadership engagement and implementation climate were critical for clinics to overcome implementation barriers and preserve implementation strategy and MBC fidelity. Descriptive statistics determined champion strategy fidelity to range from 61 to 93% across the 10 clinics. Fidelity to the audit and feedback strategy ranged from 82 to 91% across the 5 clinics assigned to that condition. MBC fidelity ranged from 54 to 95% across all clinics. Although correlations between ISF and MBC fidelity were not statistically significant due to the sample of 10 clinics, associations were in the expected direction and of moderate effect size. A coefficient for shared depression screening among clinicians had greater face validity compared to depression screening coverage and functioned as a proximal indicator of implementation strategy success.

Conclusion

Fidelity to the basic and enhanced strategies varied by site and were influenced by leadership engagement and implementation climate. Champion strategies may benefit from the addition of leadership strategies to help address implementation barriers outside the purview of champions. ISF may moderate the relationship between strategies and implementation outcomes.

Evaluation of Health Promotion Program at the Community Level in Mexico: Is the Program Efficient and Effective?

CONTEXT

Health promotion programs have been encouraged in the Americas since 1990. In Mexico, health program promotion at the community level was implemented by the Ministry of Health in 2001 to encourage community health status improvement. Despite the longtime of its implementation, evaluations of its efficiency and effectiveness are scarce.

OBJECTIVE

To evaluate the efficiency of the Healthy Environments and Communities Program (HECP) through 2 means: (1) efficiency of the implementation and (2) technical efficiency, as well as to evaluate its effectiveness.

SETTING

Target communities of the HECP of 32 Mexican states during 2013-2017. The HECP developed community interventions to improve community organization, health behaviors, and family and communitarian sanitation.

METHODOLOGY

We conducted a cross-sectional study to evaluate the efficiency of HECP implementation and estimated the inclusion of target communities (focalization index), the retention of communities (continuity index), and the desertion of communities in the program (desertion index). To evaluate the adequate use of the program resources (technical efficiency), we used data-enveloped analysis and the Tobit regression model to identify external factors that can influence results. Finally, to evaluate the program's effectiveness, we estimated the index of the communities that improved their health indicators and were certified as healthy (community certification).

RESULTS

The median rate of focalization was 3.44 (1.31-85.13); the continuity of communities' rate was 0.50 (0.16-2.67). Regarding technical efficiency to reach healthy communities, only 2 states reached the optimal efficiency (score 1); where the efficiency was adjusted for external factors, 6 states reached a score of 1. The median of global effectiveness was 0.19 (0.01-0.78). We found differences in efficiency and effectiveness scores among states.

CONCLUSION

We found lower efficiency of the implementation and technical efficiency, as well as poor effectiveness of the program to reach healthy communities. To achieve HECP purpose, it is necessary to revise its guidelines, improve its strategies to work in communities, and establish the right mechanisms to monitor its implementation. It is essential to focus on the resources used to enhance technical efficiency and effectiveness at the community level.

Implementation fidelity to a behavioral diabetes prevention intervention in two New York City safety net primary care practices

BACKGROUND

It is critical to assess implementation fidelity of evidence-based interventions and factors moderating fidelity, to understand the reasons for their success or failure. However, fidelity and fidelity moderators are seldom systematically reported. The study objective was to conduct a concurrent implementation fidelity evaluation and examine fidelity moderators of CHORD (Community Health Outreach to Reduce Diabetes), a pragmatic, cluster-randomized, controlled trial to test the impact of a Community Health Workers (CHW)-led health coaching intervention to prevent incident type 2 Diabetes Mellitus in New York (NY).

METHODS

We applied the Conceptual Framework for Implementation Fidelity to assess implementation fidelity and factors moderating it across the four core intervention components: patient goal setting, education topic coaching, primary care (PC) visits, and referrals to address social determinants of health (SDH), using descriptive statistics and regression models. PC patients with prediabetes receiving care from safety-net patient-centered medical homes (PCMHs) at either, VA NY Harbor or at Bellevue Hospital (BH) were eligible to be randomized into the CHW-led CHORD intervention or usual care. Among 559 patients randomized and enrolled in the intervention group, 79.4% completed the intake survey and were included in the analytic sample for fidelity assessment. Fidelity was measured as coverage, content adherence and frequency of each core component, and the moderators assessed were implementation site and patient activation measure.

RESULTS

Content adherence was high for three components with nearly 80.0% of patients setting ≥ 1 goal, having ≥ 1 PC visit and receiving ≥ 1 education session. Only 45.0% patients received ≥ 1 SDH referral. After adjusting for patient gender, language, race, ethnicity, and age, the implementation site moderated adherence to goal setting (77.4% BH vs. 87.7% VA), educational coaching (78.9% BH vs. 88.3% VA), number of successful CHW-patient encounters (6 BH vs 4 VA) and percent of patients receiving all four components (41.1% BH vs. 25.7% VA).

CONCLUSIONS

The fidelity to the four CHORD intervention components differed between the two implementation sites, demonstrating the challenges in implementing complex evidence-based interventions in different settings. Our findings underscore the importance of measuring implementation fidelity in contextualizing the outcomes of randomized trials of complex multi-site behavioral interventions.

TRIAL REGISTRATION

The trial was registered with ClinicalTrials.gov on 30/12/2016 and the registration number is NCT03006666 .

A continuity of care programme for women at risk of preterm birth in the UK: Process evaluation of a hybrid randomised controlled pilot trial

BACKGROUND

The development and evaluation of specific maternity care packages designed to address preterm birth remains a public health priority. We aim to evaluate the implementation, context, and potential mechanisms of action, of a new care pathway that combined midwifery continuity of care with a specialist obstetric clinic for women at risk of preterm birth (POPPIE) in London (UK).

METHODS

We did a multiphase mixed method triangulation evaluation nested within a hybrid type 2, randomised controlled trial in London (United Kingdom). Pregnant women with identified risk factors for preterm birth were eligible for trial participation and randomly assigned (1:1) to either midwifery continuity of care linked to a specialist obstetric clinic (POPPIE group) or standard maternity care. The primary outcome was a composite of appropriate and timely interventions for the prevention and/or management of preterm labour and birth, analysed according to intention to treat. Clinical and process outcome data were abstracted from medical records and electronic data systems, and coded by study team members, who were masked to study group allocation. Implementation data were collected from meeting records and key documents, postnatal surveys (n = 164), semi-structured interviews with women (n = 30), healthcare providers and stakeholders (n = 24) pre-, mid and post implementation. Qualitative and quantitative data from meeting records and key documents were examined narratively. Qualitative data from interviews were analysed using three thematic frameworks: Proctor's (for implementation outcomes: appropriateness, adoption, feasibility, acceptability, fidelity, penetration, sustainability), the Consolidated Framework for Implementation Research (for determinants of implementation), and published program theories of continuity models (for potential mechanisms). Data triangulation followed a convergent parallel and pragmatic approach which brought quantitative and qualitative data together at the interpretation stage. We averaged individual implementation measures across all domains to give a single composite implementation strength score which was compared to the primary outcome.

RESULTS

Between May 9, 2017, and Sep 30, 2018, 553 women were assessed for eligibility and 334 were enrolled with less than 6% of loss to follow up (169 were assigned to the POPPIE group; 165 were to the standard group). There was no difference in the primary outcome (POPPIE group 83·3% versus standard group 84·7%; risk ratio 0·98 [95% CI 0·90 to 1·08]). Appropriateness and adoption: The introduction of the POPPIE model was perceived as a positive fundamental change for local maternity services. Partnership working and additional funding were crucial for adoption. Fidelity: More than 75% of antenatal and postnatal visits were provided by a named or partner midwife, and a POPPIE midwife was present in more than 80% of births. Acceptability: Nearly 98% of women who responded to the postnatal survey were very satisfied with POPPIE model. Quantitative fidelity and acceptability results were supported by the qualitative findings. Penetration and sustainability: Despite delays (likely associated with lack of existing continuity models at the hospital), the model was embedded within established services and a joint decision was made to sustain and adapt the model after the trial (strongly facilitated by national maternal policy on continuity pathways). Potential mechanisms of impact identified included e.g. access to care, advocacy and perceptions of safety and trust. There was no association between implementation measures and the primary outcome.

CONCLUSIONS

The POPPIE model of care was a feasible and acceptable model of care that was implemented with high fidelity and sustained in maternity services. Larger powered trials are feasible and needed in other settings, to evaluate the impact and implementation of continuity programmes in other communities affected by preterm birth and women who experience social disadvantage and vulnerability.

TRIAL REGISTRATION

UKCRN Portfolio Database (prospectively registered, 24 April 2017): 31951. ISRCTN registry (retrospectively registered, 21 August 2017): ISRCTN37733900.

Repeated measures of implementation variables

It is commonly acknowledged that implementation work is long-term and contextual in nature and often takes years to accomplish. Repeated measures are needed to study the trajectory of implementation variables over time. To be useful in typical practice settings, measures that are relevant, sensitive, consequential, and practical are needed to inform planning and action. If implementation independent variables and implementation dependent variables are to contribute to a science of implementation, then measures that meet these criteria must be established. This exploratory review was undertaken to "see what is being done" to evaluate implementation variables and processes repeatedly in situations where achieving outcomes was the goal (i.e., more likely to be consequential). No judgement was made about the adequacy of the measure (e.g., psychometric properties) in the review. The search process resulted in 32 articles that met the criteria for a repeated measure of an implementation variable. 23 different implementation variables were the subject of repeated measures. The broad spectrum of implementation variables identified in the review included innovation fidelity, sustainability, organization change, and scaling along with training, implementation teams, and implementation fidelity. Given the long-term complexities involved in providing implementation supports to achieve the full and effective use of innovations, repeated measurements of relevant variables are needed to promote a more complete understanding of implementation processes and outcomes. Longitudinal studies employing repeated measures that are relevant, sensitive, consequential, and practical should become common if the complexities involved in implementation are to be understood.

What can healthcare systems learn from looking at tensions in innovation processes? A systematic literature review

Background

Until now, scholarship on innovation processes in healthcare systems lack an in-depth appreciation of tensions. Tensions often revolve around barriers and result from individual assessments and prioritizations that guide actions to eventually overcome these barriers. In order to develop a more differentiated understanding of tensions’ role in healthcare innovation processes, this paper aims to shed light on the multifaceted ways in which tensions emerge, are being dealt with, and how they hinder or, at times, facilitate innovation processes.

Methods

A systematic review of published and grey literature was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guideline. The review involved searching three databases for original research articles and manually searching citations. Twenty-nine original full texts were identified, evaluated, and coded. These include papers on innovation in healthcare systems that investigated innovation-related organizational tensions. The findings were synthesized into different types of tensions in healthcare system innovation and the descriptions of the conflicting elements. We also analyzed the investigated innovations by type, process stages, and across different countries and healthcare systems.

Results

A total of forty-two tensions were identified and grouped into nine categories. Organizing tensions were predominant, followed by learning/belonging, performing, and performing/organizing tensions. Tensions most frequently occurred in the implementation phase and in the form of a dilemma. Included studies were conducted mainly in government-funded healthcare systems.

Conclusion

Our data suggest that innovation processes in healthcare systems are impaired by conflicts between contradictory elements, working cultures, and convictions and the organizational and regulatory context. Since the majority of the tensions we collected in our study can be addressed, future policy-making and research should take advantage of this fact and develop strategies that significantly influence the successful management of tensions and thus improve the implementation of innovations.

Implementation fidelity in a multifaceted program to foster rational antibiotics use in primary care: an observational study

BACKGROUND

The ARena study (Sustainable Reduction of Antimicrobial Resistance in German Ambulatory Care) is a three-arm, cluster randomized trial to evaluate a multifaceted implementation program in a German primary care setting. In the context of a prospective process evaluation conducted alongside ARena, this study aimed to document and explore fidelity of the implementation program.

METHODS

This observational study is based on data generated in a three-wave survey of 312 participating physicians in the ARena program and attendance documentation. Measures concerned persistence of participation in the ARena program and adherence to intervention components (thematic quality circles, e-learning, basic expenditure reimbursements, additional bonus payments and a computerized decision support system). Participants' views on five domains of the implementation were also measured. Binary logistic and multiple linear regression analyses were used to explore which views on the implementation were associated with participants' adherence to quality circles and use of additional bonus compensation.

RESULTS

The analysis of fidelity showed overall high persistence of participation in the intervention components across the three intervention arms (90,1%; 97,9%; 92,9%). 96.4% of planned quality circles were delivered to study participants and, across waves, 30.4% to 93% of practices participated; 56.1% of physicians attended the maximum of four quality circles. 84% of the practices (n = 158) with a minimum of one index patient received a performance-based additional bonus payment at least once. In total, bonus compensation was triggered for 51.8% of affected patients. Participation rate for e-learning (a prerequisite for reimbursement of project-related expenditure) covered 90.8% of practices across all intervention arms, with the highest rate in arm II (96.5%). Uptake of expenditure reimbursement was heterogeneous across study arms, with a mean rate of 86.5% (89.1% in arm I, 96.4% in arm II and 74.1% in arm III). Participants' views regarding participant responsiveness (OR = 2.298) 95% CI [1.598, 3.305] and Context (OR = 2.146) 95% CI [1.135, 4.055] affected additional bonus payment. Participants' views on participant responsiveness (Beta = 0.718) 95% CI [0.479, 0.957], Context (Beta = 0.323) 95% CI [0.055, 0.590] and Culture of shared decision-making (Beta = -0.334) 95% CI [-0.614, -0.053] affected quality circle attendance.

CONCLUSION

This study showed an overall high fidelity to the implementation program. Participants' views on the implementation were associated with degree of intervention fidelity.

TRIAL REGISTRATION

ISRCTN, ISRCTN58150046.

Organisational structures and processes for health and well-being: insights from work integration social enterprise

Background

Previous research on employee well-being for those who have experienced social and economic disadvantage and those with previous or existing mental health conditions has focused mainly on programmatic interventions. The purpose of this research was to examine how organisational structures and processes (such as policies and culture) influence well-being of employees from these types of backgrounds.

Methods

A case study ethnographic approach which included in-depth qualitative analysis of 93 semi-structured interviews of employees, staff, and managers, together with participant observation of four social enterprises employing young people.

Results

The data revealed that young people were provided a combination of training, varied work tasks, psychosocial support, and encouragement to cultivate relationships among peers and management staff. This was enabled through the following elements: structure and space; funding, finance and industry orientation; organisational culture; policy and process; and fostering local service networks.. The findings further illustrate how organisational structures at these workplaces promoted an inclusive workplace environment in which participants self-reported a decrease in anxiety and depression, increased self-esteem, increased self-confidence and increased physical activity.

Conclusions

Replicating these types of organisational structures, processes, and culture requires consideration of complex systems perspectives on implementation fidelity which has implications for policy, practice and future research.

Rehabilitation: mobility, exercise & sports; a critical position stand on current and future research perspectives

BACKGROUND

Human movement, rehabilitation, and allied sciences have embraced their ambitions within the cycle of "RehabMove" congresses over the past 30 years. This combination of disciplines and collaborations in the Netherlands has tried to provide answers to questions in the fields of rehabilitation and adapted sports, while simultaneously generating new questions and challenges. These research questions help us to further deepen our understanding of (impaired) human movement and functioning, with and without supportive technologies, and stress the importance of continued multidisciplinary (inter)national collaboration.

METHODS

This position stand provides answers that were conceived by the authors in a creative process underlining the preparation of the 6th RehabMove Congress.

RESULTS

The take-home message of the RehabMove2018 Congress is a plea for continued multidisciplinary research in the fields of rehabilitation and adapted sports. This should be aimed at more individualized notions of human functioning, practice, and training, but also of performance, improved supportive technology, and appropriate "human and technology asset management" at both individual and organization levels and over the lifespan.

CONCLUSIONS

With this, we anticipate to support the development of rehabilitation sciences and technology and to stimulate the use of rehabilitation notions in general health care. We also hope to help ensure a stronger embodiment of preventive and lifestyle medicine in rehabilitation practice. Indeed, general health care and rehabilitation practice require a healthy and active lifestyle management and research agenda in the context of primary, secondary, and tertiary prevention.IMPLICATIONS FOR REHABILITATIONContinued multidisciplinary (international) collaboration will stimulate the development of rehabilitation and human movement sciences.Notions from "human and technology asset management and ergonomics" are fundamental to rehabilitation practice and research.The rehabilitation concept will further merge into general health care and the quality there-off.

Evaluating process fidelity during the implementation of Group Antenatal Care in Mexico

BACKGROUND

CenteringPregnancy (CP) is a group antenatal care (G-ANC) model that has proven beneficial for mothers and their newborns. We conducted a feasibility study beginning in 2016 as part of the Mexican effort to implement G-ANC locally. This study reports on fidelity to the essential elements of CP during its implementation in Mexico.

METHODS

We collected prospective data using a standardized checklist at four primary-care centers that implemented our adapted G-ANC model. We performed a descriptive analysis of fidelity to 28 processes per G-ANC session (71 sessions made up of 10 groups and 129 women across 4 health centers). We calculated fidelity to each process as a proportion with 95% confidence intervals. We present overall results and stratified by health center and by facilitation team.

RESULTS

Overall fidelity to the G-ANC intervention was 82%, with variability by health center (78-88%). The elements with the highest fidelity were having space for activities such as checking vital signs, conversation in a circle, and medical check-ups (100% each) and the element with the lowest fidelity was using music to enhance privacy (27.3%). Fidelity was not significantly different by center.

CONCLUSIONS

Our study suggests good model fidelity during the implementation of G-ANC in Mexico. Our findings also contribute useful information about where to focus efforts in the future to maintain and improve G-ANC model fidelity.

The implementation of a physical activity counseling program in rehabilitation care: findings from the ReSpAct study

PURPOSE

This study aimed to evaluate the implementation of a physical activity counseling program in rehabilitation and to study heterogeneity in received counseling and investigate its association with changes in patients' physical activity outcomes.

METHODS

This prospective cohort study was conducted in 18 rehabilitation institutions. Data were collected using surveys completed by professionals (n= ±70) and patients (n = 1719). Implementation was evaluated using different process outcomes: reach, dosage, satisfaction, maintenance. Patients' physical activity outcomes included changes in total minutes/week of physical activity. Latent class analyses were conducted to identify profiles of received counseling characteristics and multilevel models were used to investigate associations with physical activity outcomes.

RESULTS

5873 Patients were provided with motivational interviewing-based counseling after rehabilitation. Professionals and patients were positive about the program. Sixteen institutions (89%) formally agreed to continue the program. The four identified profiles of counseling characteristics illustrate a large variation in received counseling among patients. No substantial differences in physical activity outcomes were found between profiles.

CONCLUSION

After a three-year program period, the physical activity counseling centers were sustainably implemented in Dutch rehabilitation care. This study illustrated an innovative approach to assess heterogeneity in implementation outcomes (e.g., counseling profiles) in relation to program outcomes (e.g., physical activity).Implications for rehabilitationPhysical activity counseling after rehabilitation is important to support people with disabilities in making the step from rehabilitation-based physical activities to community-based physical activities.Establishing "Physical Activity Counseling Centers" is a promising "disability-overarching" strategy to promote physical activity after rehabilitation.Although the actual received counseling (dosage) varied among patients, this did not coincide with large differences in physical activity outcomes.The training in Motivational Interviewing, the financial incentives, and the advisory support were considered as important or essential ingredients for a successful implementation of the counseling program in rehabilitation practice.

Implementation evaluation of academic detailing on naloxone prescribing trends at the United States Veterans Health Administration

OBJECTIVE

Academic detailing in partnership with the Opioid Overdose Education and Naloxone Distribution (OEND) program was implemented to increase naloxone access for the prevention of opioid overdose mortality in veterans at the U.S. Department of Veterans Affairs (VA). However, implementation was not uniform leading to varying levels of intervention exposure potentially impacting naloxone prescribing. We examined the impact of implementation strength (proportion of providers exposed to academic detailing) at each station on naloxone prescribing from September 2014 to December 2017.

STUDY DESIGN AND SETTING

Retrospective cohort design with fixed effects models at the VA.

DATA COLLECTION/EXTRACTION METHODS

We used VA Corporate Data Warehouse for data on pharmacy dispensing, station-, provider- and patient-level characteristics. OEND-specific academic detailing activities came from data recorded by academic detailers using Salesforce.com.

PRINCIPAL FINDINGS

VA stations wherein 100 percent of providers exposed to an OEND-related academic detailing educational outreach visit experienced an increased incident rate of naloxone prescribing that was 5.52 times the incident rate of stations where no providers were exposed; alternatively, this is equivalent to an average monthly increase of 2.60 naloxone prescriptions per 1000 population at risk for opioid overdose.

CONCLUSIONS

Our findings highlight the importance of academic detailing's implementation strength on naloxone prescribing. Decision makers must carefully consider the implementation process to achieve the greatest effectiveness from the intervention.

The influence of contextual factors on implementation fidelity in a whole school approach to prevention of violence against women

ISSUE ADDRESSED

Implementation fidelity relates to the degree of adherence to implementation protocols and content and helps to guide replication of evidence-based programs. In settings-based research, notions of fidelity have been applied more often to delivery of education content rather than whole of setting change. The aims of this paper were firstly, to analyse how contextual factors influenced implementation of a whole school program on respectful relationships education, and secondly given the findings, discuss whether a more flexible approach to implementation fidelity may have yielded increased school engagement.

METHODS

The project was conducted in 19 secondary schools in Victoria in 2015. This paper reports on focus groups and interviews which were conducted with 81 school staff and 28 staff from the lead agency, community agency partners, and departmental staff to understand the contextual factors influencing implementation.

RESULTS

The program followed a traditional implementation fidelity approach of considering core elements and some minor scope for adaption which engaged some regions and schools but not others.

CONCLUSION

An alternative notion of implementation fidelity, "fidelity to function," may have permitted increased flexibility to tailor the intervention components to suit different school and community contexts and potentially, increasing both the reach and impact of the program. SO WHAT?: Understanding how to apply notions of fidelity to guide whole of setting change are important considerations if programs are to be replicated to have wider reach and greater impact and effectiveness.