Process evaluation on quality improvement interventions

Process evaluation of a data-driven quality improvement program within a cluster randomised controlled trial to improve coronary heart disease management in Australian primary care

BACKGROUND

This study evaluates primary care practices' engagement with various features of a quality improvement (QI) intervention for patients with coronary heart disease (CHD) in four Australian states.

METHODS

Twenty-seven practices participated in the QI intervention from November 2019 -November 2020. A combination of surveys, semi-structured interviews and other materials within the QUality improvement in primary care to prevent hospitalisations and improve Effectiveness and efficiency of care for people Living with heart disease (QUEL) study were used in the process evaluation. Data were summarised using descriptive statistical and thematic analyses for 26 practices.

RESULTS

Sixty-four practice team members and Primary Health Networks staff provided feedback, and nine of the 63 participants participated in the interviews. Seventy-eight percent (40/54) were either general practitioners or practice managers. Although 69% of the practices self-reported improvement in their management of heart disease, engagement with the intervention varied. Forty-two percent (11/26) of the practices attended five or more learning workshops, 69% (18/26) used Plan-Do-Study-Act cycles, and the median (Interquartile intervals) visits per practice to the online SharePoint site were 170 (146-252) visits. Qualitative data identified learning workshops and monthly feedback reports as the key features of the intervention.

CONCLUSION

Practice engagement in a multi-featured data-driven QI intervention was common, with learning workshops and monthly feedback reports identified as the most useful features. A better understanding of these features will help influence future implementation of similar interventions.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry (ANZCTR) number ACTRN12619001790134.

Optimal designs using generalized estimating equations in cluster randomized crossover and stepped wedge trials

Cluster randomized crossover and stepped wedge cluster randomized trials are two types of longitudinal cluster randomized trials that leverage both the within- and between-cluster comparisons to estimate the treatment effect and are increasingly used in healthcare delivery and implementation science research. While the variance expressions of estimated treatment effect have been previously developed from the method of generalized estimating equations for analyzing cluster randomized crossover trials and stepped wedge cluster randomized trials, little guidance has been provided for optimal designs to ensure maximum efficiency. Here, an optimal design refers to the combination of optimal cluster-period size and optimal number of clusters that provide the smallest variance of the treatment effect estimator or maximum efficiency under a fixed total budget. In this work, we develop optimal designs for multiple-period cluster randomized crossover trials and stepped wedge cluster randomized trials with continuous outcomes, including both closed-cohort and repeated cross-sectional sampling schemes. Local optimal design algorithms are proposed when the correlation parameters in the working correlation structure are known. MaxiMin optimal design algorithms are proposed when the exact values are unavailable, but investigators may specify a range of correlation values. The closed-form formulae of local optimal design and MaxiMin optimal design are derived for multiple-period cluster randomized crossover trials, where the cluster-period size and number of clusters are decimal. The decimal estimates from closed-form formulae can then be used to investigate the performances of integer estimates from local optimal design and MaxiMin optimal design algorithms. One unique contribution from this work, compared to the previous optimal design research, is that we adopt constrained optimization techniques to obtain integer estimates under the MaxiMin optimal design. To assist practical implementation, we also develop four SAS macros to find local optimal designs and MaxiMin optimal designs.

Effect and Process Evaluation of an Intervention to Improve Hand Hygiene Compliance in Long-Term Care Facilities

OBJECTIVES

This study evaluated the effect of a tailored, multifaceted improvement strategy on hand hygiene compliance in long-term care facilities (LTCFs). We also performed a process evaluation to explore the mechanisms through which our strategy brought about change.

DESIGN

We conducted a stepped-wedge cluster-randomized controlled trial with a sequential rollout of the improvement strategy to all participating LTCFs. The strategy consisted of education, training, reminders, observation sessions (including feedback), and team meetings (including feedback).

SETTING AND PARTICIPANTS

The study included nursing professionals from 14 LTCFs (23 wards) in the Netherlands.

METHODS

Hand hygiene compliance was observed during 5 measurement periods using WHO's "Five Moments for Hand Hygiene." Multilevel analyses and corresponding tests were completed on an intention-to-treat basis.

RESULTS

The absolute intervention effect of overall hand hygiene compliance (primary outcome measure) was 13% (95% CI 9.3-16.7, P < .001), adjusted for time and clustering. The adjusted absolute effect was 23% (95% CI 7-39, P < .002) before a clean and aseptic procedure, 18% (95% CI 10-26, P < .001) after touching a resident, 14% (95% CI 7-22, P < .003) before touching a resident, 10% (95% CI 5-15, P < .001) after contact with body fluid, and 1% (95% CI -11 to 13, P = .8) after touching a resident's surroundings. With the exception of leadership, participants at LTCFs with more exposure to the intervention components showed statistically significantly more improvement than those at facilities with lower exposure scores.

CONCLUSIONS AND IMPLICATIONS

Our strategy was successful in improving hand hygiene compliance. LTCFs with more team members exposed to the different intervention components, demonstrated a greater effect from the intervention. To strengthen the impact of our intervention, we recommend that future improvement strategies provide more support to managers to ensure they are better equipped to take on their leadership roles and enable their teams to improve and maintain hand hygiene compliance.

Implementing a new living concept for persons with dementia in long-term care: evaluation of a quality improvement process

BACKGROUND

Improving quality of nursing home care for residents is a constant focus of stakeholders involved within quality improvement projects. Though, achieving change in long-term care is challenging. Process evaluations provide insight into the nature, exposure and experiences of stakeholders and influencing mechanisms for implementation. The aim of this study is to gain insight into the process and facilitating and hindering mechanisms of implementing a quality improvement project that seeks to create a dementia-friendly community with a nursing home at its core.

METHODS

For the process evaluation we planned a case study design with an ethnographic approach. Various research methods were used: qualitative observations, focus groups, interviews and questionnaires for various stakeholders and document review. Data collection and analyses in this study is based on the Consolidated Framework for Implementation Research.

RESULTS

Four main lessons were learned. Firstly, nursing staff are crucial to achieve more freedom for residents. Secondly, high-impact changes in daily care need strong and sustainable focus from the care organisation. Thirdly, dementia-friendly societies should be deployed from multiple actors, which entails long-term collaborations with external stakeholders. Fourthly, the transition to a dementia-friendly society requires meeting spaces for and a focus on both residents and people from the community. Consequently, local residents are shifting from external to internal stakeholders, extending beyond the regular involvement of informal carers and volunteers within the nursing home.

CONCLUSIONS

Nursing homes are part of the local community and provide opportunities to collaborate on a dementia-friendly society. However, the change that is required (promoting freedom, residents' autonomy and the redesign of care processes) is complex and influenced by various mechanisms. Understanding these mechanisms can benefit other care organisations that strive to implement a similar initiative.

Champion and audit and feedback strategy fidelity and their relationship to depression intervention fidelity: A mixed method study

Background

Globally, mental health disorders rank as the greatest cause of disability. Low and middle-income countries (LMICs) hold a disproportionate share of the mental health burden, especially as it pertains to depression. Depression is highly prevalent among those with non-communicable diseases (NCDs), creating a barrier to successful treatment. While some treatments have proven efficacy in LMIC settings, wide dissemination is challenged by multiple factors, leading researchers to call for implementation strategies to overcome barriers to care provision. However, implementation strategies are often not well defined or documented, challenging the interpretation of study results and the uptake and replication of strategies in practice settings. Assessing implementation strategy fidelity (ISF), or the extent to which a strategy was implemented as designed, overcomes these challenges. This study assessed fidelity of two implementation strategies (a 'basic' champion strategy and an 'enhanced' champion + audit and feedback strategy) to improve the integration of a depression intervention, measurement based care (MBC), at 10 NCD clinics in Malawi. The primary goal of this study was to assess the relationship between the implementation strategies and MBC fidelity using a mixed methods approach.

Methods

We developed a theory-informed mixed methods fidelity assessment that first combined an implementation strategy specification technique with a fidelity framework. We then created corresponding fidelity indicators to strategy components. Clinical process data and one-on-one in-depth interviews with 45 staff members at 6 clinics were utilized as data sources. Our final analysis used descriptive statistics, reflexive-thematic analysis (RTA), data merging, and triangulation to examine the relationship between ISF and MBC intervention fidelity.

Results

Our mixed methods analysis revealed how ISF may moderate the relationship between the strategies and MBC fidelity. Leadership engagement and implementation climate were critical for clinics to overcome implementation barriers and preserve implementation strategy and MBC fidelity. Descriptive statistics determined champion strategy fidelity to range from 61 to 93% across the 10 clinics. Fidelity to the audit and feedback strategy ranged from 82 to 91% across the 5 clinics assigned to that condition. MBC fidelity ranged from 54 to 95% across all clinics. Although correlations between ISF and MBC fidelity were not statistically significant due to the sample of 10 clinics, associations were in the expected direction and of moderate effect size. A coefficient for shared depression screening among clinicians had greater face validity compared to depression screening coverage and functioned as a proximal indicator of implementation strategy success.

Conclusion

Fidelity to the basic and enhanced strategies varied by site and were influenced by leadership engagement and implementation climate. Champion strategies may benefit from the addition of leadership strategies to help address implementation barriers outside the purview of champions. ISF may moderate the relationship between strategies and implementation outcomes.

Online remote behavioural intervention for tics in 9- to 17-year-olds: the ORBIT RCT with embedded process and economic evaluation

Background

Behavioural therapy for tics is difficult to access, and little is known about its effectiveness when delivered online.

Objective

To investigate the clinical and cost-effectiveness of an online-delivered, therapist- and parent-supported therapy for young people with tic disorders.

Design

Single-blind, parallel-group, randomised controlled trial, with 3-month (primary end point) and 6-month post-randomisation follow-up. Participants were individually randomised (1 : 1), using on online system, with block randomisations, stratified by site. Naturalistic follow-up was conducted at 12 and 18 months post-randomisation when participants were free to access non-trial interventions. A subset of participants participated in a process evaluation.

Setting

Two hospitals (London and Nottingham) in England also accepting referrals from patient identification centres and online self-referrals.

Participants

Children aged 9-17 years (1) with Tourette syndrome or chronic tic disorder, (2) with a Yale Global Tic Severity Scale-total tic severity score of 15 or more (or > 10 with only motor or vocal tics) and (3) having not received behavioural therapy for tics in the past 12 months or started/stopped medication for tics within the past 2 months.

Interventions

Either 10 weeks of online, remotely delivered, therapist-supported exposure and response prevention therapy (intervention group) or online psychoeducation (control).

Outcome

Primary outcome: Yale Global Tic Severity Scale-total tic severity score 3 months post-randomisation; analysis done in all randomised patients for whom data were available. Secondary outcomes included low mood, anxiety, treatment satisfaction and health resource use. Quality-adjusted life-years are derived from parent-completed quality-of-life measures. All trial staff, statisticians and the chief investigator were masked to group allocation.

Results

Two hundred and twenty-four participants were randomised to the intervention (n = 112) or control (n = 112) group. Participants were mostly male (n = 177; 79%), with a mean age of 12 years. At 3 months the estimated mean difference in Yale Global Tic Severity Scale-total tic severity score between the groups adjusted for baseline and site was -2.29 points (95% confidence interval -3.86 to -0.71) in favour of therapy (effect size -0.31, 95% confidence interval -0.52 to -0.10). This effect was sustained throughout to the final follow-up at 18 months (-2.01 points, 95% confidence interval -3.86 to -0.15; effect size -0.27, 95% confidence interval -0.52 to -0.02). At 18 months the mean incremental cost per participant of the intervention compared to the control was £662 (95% confidence interval -£59 to £1384), with a mean incremental quality-adjusted life-year of 0.040 (95% confidence interval -0.004 to 0.083) per participant. The mean incremental cost per quality-adjusted life-year gained was £16,708. The intervention was acceptable and delivered with high fidelity. Parental engagement predicted child engagement and more positive clinical outcomes.

Harms

Two serious, unrelated adverse events occurred in the control group.

Limitations

We cannot separate the effects of digital online delivery and the therapy itself. The sample was predominately white and British, limiting generalisability. The design did not compare to face-to-face services.

Conclusion

Online, therapist-supported behavioural therapy for young people with tic disorders is clinically and cost-effective in reducing tics, with durable benefits extending up to 18 months.

Future work

Future work should compare online to face-to-face therapy and explore how to embed the intervention in clinical practice.

Trial registration

This trial is registered as ISRCTN70758207; ClinicalTrials.gov (NCT03483493). The trial is now complete.

Funding

This project was funded by the National Institute for Health and Care Research (NIHR) Health and Technology Assessment programme (project number 16/19/02) and will be published in full in Health and Technology Assessment; Vol. 27, No. 18. See the NIHR Journals Library website for further project information.

Systems analysis and improvement approach to improve naloxone distribution within syringe service programs: study protocol of a randomized controlled trial

BACKGROUND

More than half a million Americans died of an opioid-related overdose between 1999 and 2020, the majority occurring between 2015 and 2020. The opioid overdose mortality epidemic disproportionately impacts Black, Indigenous, and people of color (BIPOC): since 2015, overdose mortality rates have increased substantially more among Black (114%) and Latinx (97%) populations compared with White populations (32%). This is in part due to disparities in access to naloxone, an opioid antagonist that can effectively reverse opioid overdose to prevent death. Our recent pilot work determined that many barriers to naloxone access can be identified and addressed by syringe service programs (SSPs) using the Systems Analysis and Improvement Approach to Naloxone distribution (SAIA-Naloxone). This randomized controlled trial will test SAIA-Naloxone's ability to improve naloxone distribution in general and among BIPOC specifically.

METHODS

We will conduct a trial with 32 SSPs across California, randomly assigning 16 to the SAIA-Naloxone arm and 16 to receive implementation as usual. SAIA-Naloxone is a multifaceted, multilevel implementation strategy through which trained facilitators work closely with SSPs to (1) assess organization-level barriers, (2) prioritize barriers for improvement, and (3) test solutions through iterative change cycles until achieving and sustaining improvements. SSPs receiving SAIA-Naloxone will work with a trained facilitator for a period of 12 months. We will test SAIA-Naloxone's ability to improve SSPs' naloxone distribution using an interrupted time series approach. Data collection will take place during a 3-month lead-in period, the 12-month active period, and for an additional 6 months afterward to determine whether impacts are sustained. We will use a structured approach to specify SAIA-Naloxone to ensure strategy activities are clearly defined and to assess SAIA-Naloxone fidelity to aid in interpreting study results. We will also assess the costs associated with SAIA-Naloxone and its cost-effectiveness.

DISCUSSION

This trial takes a novel approach to improving equitable distribution of naloxone amid the ongoing epidemic and associated racial disparities. If successful, SAIA-Naloxone represents an important organizational-level solution to the multifaceted and multilevel barriers to equitable naloxone distribution.

An overview of optimal designs under a given budget in cluster randomized trials with a binary outcome

Cluster randomized trial design may raise financial concerns because the cost to recruit an additional cluster is much higher than to enroll an additional subject in subject-level randomized trials. Therefore, it is desirable to develop an optimal design. For local optimal designs, optimization means the minimum variance of the estimated treatment effect under the total budget. The local optimal design derived from the variance needs the input of an association parameter ρ in terms of a "working" correlation structure R ( ρ ) in the generalized estimating equation models. When the range of ρ instead of an exact value is available, the parameter space is defined as the range of ρ and the design space is defined as enrollment feasibility, for example, the number of clusters or cluster size. For any value ρ within the range, the optimal design and relative efficiency for each design in the design space is obtained. Then, for each design in the design space, the minimum relative efficiency within the parameter space is calculated. MaxiMin design is the optimal design that maximizes the minimum relative efficiency among all designs in the design space. Our contributions are threefold. First, for three common measures (risk difference, risk ratio, and odds ratio), we summarize all available local optimal designs and MaxiMin designs utilizing generalized estimating equation models when the group allocation proportion is predetermined for two-level and three-level parallel cluster randomized trials. We then propose the local optimal designs and MaxiMin designs using the same models when the group allocation proportion is undecided. Second, for partially nested designs, we develop the optimal designs for three common measures under the setting of equal number of subjects per cluster and exchangeable working correlation structure in the intervention group. Third, we create three new Statistical Analysis System (SAS) macros and update two existing SAS macros for all the optimal designs. We provide two examples to illustrate our methods.

Prescription drug monitoring programs evaluation: A systematic review of reviews

BACKGROUND

Prescription drug monitoring programs (PDMPs) are used to mitigate harms from high-risk medicines including misuse, prescription shopping, overdoses, and death. Previous systematic reviews report inconsistent findings. We undertook a systematic review of reviews to 1) describe and identify the methods and outcome measures used to evaluate PDMPs, 2) summarise existing evidence on outcomes and factors that influence PDMP success or benefit realisation.

METHODS

MEDLINE, EMBASE, Scopus, Cochrane Database of Systematic Reviews, and PROSPERO were used to identify systematic reviews on PDMPs. Twelve papers met the inclusion criteria. Data extracted included review aim, study designs, settings, outcome measures, and key findings. Quality was assessed using AMSTAR 2 quality assessment tool.

RESULTS

Review papers were categorised as outcome or process evaluation reviews. Process evaluation reviews described implementation processes, barriers and facilitators to PDMP use and/or implementation. Most (57%) papers described barriers which frequently included usability and data integration. Outcome evaluation papers reported impact of PDMPs on outcomes, which were opioid-focused, and findings were highly variable. Most reviews (67%) were rated as low quality, limiting the conclusions that can be drawn.

CONCLUSIONS

Inconsistent methods and outcome measures were used to evaluate PDMPs. No economic evaluations of PDMPs were found. Standardising assessment and reporting of results may improve the quality and confidence in an evidence-base to inform future roll-out and evaluation of PDMPs. Targeting barriers such as system-related challenges and negative end-user perceptions could improve sustained uptake of PDMPs, and potentially facilitate benefits realisation, including mitigating harms of high-risk prescription medicines.

Training the Next Generation of Diabetes Researchers: Evaluation of the Diabetes Action Canada Training and Mentoring Program

BACKGROUND

The Diabetes Action Canada Training and Mentoring (DAC-TM) Program launched in June 2017, with the goal of building capacity in the next generation of diabetes researchers in Canada in patient-oriented research (POR).

METHODS

We conducted a program evaluation of the DAC-TM program using a sequential, mixed-methods research design.

RESULTS

Our analysis of 82 surveys and 22 in-depth interviews from a wide range of DAC-TM Program stakeholders revealed consistent patterns in experience with the program. The training sessions were perceived to be well-organized, convenient educational opportunities to gain new knowledge about POR and become integrated into a community of practice of POR researchers who study diabetes and its complications in Canada. The content of the training was perceived to be useful and relevant to participants, although improvements could be made to help address the training needs of the broader DAC community. There is broad support for and appreciation of the mentorship awards, which were perceived to be appropriately targeted to early-career investigators. The mentor-mentee relationships were perceived to be positive, productive and career-advancing overall, but could benefit from a more strategic design and promote better connectivity to foster mentor-mentee relationships. In addition, feedback about opportunities to network and forge new connections was mixed and represents another opportunity for improvement to strengthen capacity building.

CONCLUSIONS

Findings from this formative evaluation study show key strengths and opportunities to improve the DAC-TM Program, which can be used to help enhance its function and promote its long-term sustainability.

What do we want to get out of this? a critical interpretive synthesis of the value of process evaluations, with a practical planning framework

BACKGROUND

Process evaluations aim to understand how complex interventions bring about outcomes by examining intervention mechanisms, implementation, and context. While much attention has been paid to the methodology of process evaluations in health research, the value of process evaluations has received less critical attention. We aimed to unpack how value is conceptualised in process evaluations by identifying and critically analysing 1) how process evaluations may create value and 2) what kind of value they may create.

METHODS

We systematically searched for and identified published literature on process evaluation, including guidance, opinion pieces, primary research, reviews, and discussion of methodological and practical issues. We conducted a critical interpretive synthesis and developed a practical planning framework.

RESULTS

We identified and included 147 literature items. From these we determined three ways in which process evaluations may create value or negative consequences: 1) through the socio-technical processes of 'doing' the process evaluation, 2) through the features/qualities of process evaluation knowledge, and 3) through using process evaluation knowledge. We identified 15 value themes. We also found that value varies according to the characteristics of individual process evaluations, and is subjective and context dependent.

CONCLUSION

The concept of value in process evaluations is complex and multi-faceted. Stakeholders in different contexts may have very different expectations of process evaluations and the value that can and should be obtained from them. We propose a planning framework to support an open and transparent process to plan and create value from process evaluations and negotiate trade-offs. This will support the development of joint solutions and, ultimately, generate more value from process evaluations to all.

Multilevel impacts of a pediatric early warning system in resource-limited pediatric oncology hospitals

Background

Pediatric Early Warning Systems (PEWS) reduce clinical deterioration, improve interdisciplinary communication, and provide cost savings; however, little is known about how these impacts are achieved or related. This study evaluates the multi-level impacts of PEWS in resource-limited pediatric oncology centers.

Methods

We conducted 71 semi-structured interviews including physicians (45%), nurses (45%), and administrators (10%) from 5 resource-limited pediatric oncology centers in 4 Latin American countries. Interviews were conducted in Spanish, transcribed, and translated into English. A code book was developed using a priori and inductively derived codes. Transcripts were independently coded by 2 coders, achieving a kappa of 0.8-0.9. Thematic content analysis explored perceived impacts of PEWS at the level of the patient, clinician, healthcare team, and institution.

Results

PEWS improved the quality of attention for patients, reducing morbidity and mortality. Clinicians felt more knowledgeable, confident, and empowered providing patient care, resulting in greater job satisfaction. PEWS affected team dynamics by improving interdisciplinary (ward and intensive care unit) and interprofessional (physicians and nurses) relationships and communication. This ultimately led to institutional culture change with emphasis on patient safety, collaboration with other centers, and receipt of institutional awards. Together, these impacts led to hospital-wide support of ongoing PEWS use.

Conclusions

In resource-limited hospitals, PEWS use results in multi-level positive impacts on patients, clinicians, teams, and institutions, creating a feedback loop that further supports ongoing PEWS use. These findings can guide advocacy for PEWS to various stakeholders, improve PEWS effectiveness, and inform assessment of other interventions to improve childhood cancer outcomes.

Facilitating future implementation and translation to clinical practice: The Implementation Planning Assessment Tool for clinical trials

Implementation assessment plans are crucial for clinical trials to achieve their full potential. Without a proactive plan to implement trial results, it can take decades for one-fifth of effective interventions to be adopted into routine care settings. The Veterans Health Administration Office of Research and Development is undergoing a systematic transformation to embed implementation planning in research protocols through the Cooperative Studies Program, its flagship clinical research program. This manuscript has two objectives: 1) to introduce an Implementation Planning Assessment (IPA) Tool that any clinical trialist may use to facilitate post-trial implementation of interventions found to be effective and 2) to provide a case study demonstrating the IPA Tool's use. The IPA Tool encourages study designers to initially consider rigorous data collection to maximize acceptability of the intervention by end-users. It also helps identify and prepare potential interested parties at local and national leadership levels to ensure, upon trial completion, interventions can be integrated into programs, technologies, and policies in a sustainable way. The IPA Tool can alleviate some of the overwhelming nature of implementation science by providing a practical guide based on implementation science principles for researchers desiring to scale up and spread effective, clinical trial-tested interventions to benefit patients.

Strengthening respectful communication with patients and colleagues in neonatal units - developing and evaluating a communication and emotional competence training for nurse managers in Kenya

Introduction: Effective communication is essential to delivering compassionate, high-quality nursing care. The intensive, stressful and technical environment of a new-born unit (NBU) in a low-resource setting presents communication-related challenges for nurses, with negative implications for nurse well-being, team relationships and patient care. We adapted a pre-existing communication and emotional competence course with NBU nurse managers working in Kenya, explored its' value to participants and developed a theory of change to evaluate its' potential impact. Methods: 18 neonatal nurse managers from 14 county referral hospitals helped adapt and participated in a nine-month participatory training process. Training involved guided 'on the job' self-observation and reflection to build self-awareness, and two face-to-face skills-building workshops. The course and potential for future scale up was assessed using written responses from participant nurses (baseline questionnaires, reflective assignments, pre and post workshop questionnaires), workshop observation notes, two group discussions and nine individual in-depth interviews. Results: Participants were extremely positive about the course, with many emphasizing its direct relevance and applicability to their daily work. Increased self-awareness and ability to recognize their own, colleagues' and patients' emotional triggers, together with new knowledge and practical skills, reportedly inspired nurses to change; in turn influencing their ability to provide respectful care, improving their confidence and relationships and giving them a stronger sense of professional identity. Conclusion: Providing respectful care is a major challenge in low-resource, high-pressure clinical settings but there are few strategies to address this problem. The participatory training process examined addresses this challenge and has potential for positive impacts for families, individual workers and teams, including worker well-being. We present an initial theory of change to support future evaluations aimed at exploring if and how positive gains can be sustained and spread within the wider system.

Nationwide implementation of a multifaceted tailored strategy to improve uptake of standardized structured reporting in pathology: an effect and process evaluation

Background

Implementation strategies are aimed at improving guideline adherence. Both effect and process evaluations are conducted to provide insights into the success or failure of these strategies. In our study, we evaluate the nationwide implementation of standardized structured reporting (SSR) in pathology.

Methods

An interrupted time series analysis was conducted to evaluate the effect of a previously developed implementation strategy, which consisted of various digitally available elements, on SSR in pathology laboratories. A segmented regression analysis was performed to analyze the change in mean SSR percentages directly after the strategy introduction for pathology reporting and specific subcategories. In addition, we analyzed the change in trend in the weekly percentages after strategy introduction, also for subgroups of tumor groups, retrieval methods, and type of laboratory. The change in SSR use after the strategy introduction was determined for all pathology laboratories. We further conducted a process evaluation in which the exposure to the strategy elements was determined. Experiences of the users with all strategy elements and the remaining barriers and potential strategy elements were evaluated through an eSurvey. We also tested whether exposure to a specific element and a combination of elements resulted in a higher uptake of SSR after strategy introduction.

Results

There was a significant increase in an average use of SSR after the strategy introduction for reporting of gastrointestinal (p=.018) and urological (p=.003) oncological diagnoses. A significant increase was present for all oncological resections as a group (p=.007). Thirty-three out of 42 pathology laboratories increased SSR use after the strategy introduction. The “Feedback button”, an option within the templates for SSR to provide feedback to the provider and one of the elements of the implementation strategy, was most frequently used by the SSR users, and effectiveness results showed that it increased average SSR use after the strategy introduction. Barriers were still present for SSR implementation.

Conclusions

Nationwide SSR implementation improved for specific tumor groups and retrieval methods. The next step will be to further improve the use of SSR and, simultaneously, to further develop potential benefits of high SSR use, focusing on re-using discrete pathology data. In this way, we can facilitate proper treatment decisions in oncology.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-022-01224-5.

Data-driven quality improvement program to prevent hospitalisation and improve care of people living with coronary heart disease: Protocol for a process evaluation

BACKGROUND

Practice-level quality improvement initiatives using rapidly advancing technology offers a multidimensional approach to reduce cardiovascular disease burden. For the "QUality improvement in primary care to prevent hospitalisations and improve Effectiveness and efficiency of care for people Living with heart disease" (QUEL) cluster randomised controlled trial, a 12-month quality improvement intervention was designed for primary care practices to use data and implement progressive changes using "Plan, Do, Study, Act" cycles within their practices with training in a series of interactive workshops. This protocol aims to describe the systematic methods to conduct a process evaluation of the data-driven intervention within the QUEL study.

METHODS

A mixed-method approach will be used to conduct the evaluation. Quantitative data collected throughout the intervention period, via surveys and intervention materials, will be used to (1) identify the key elements of the intervention and how, for whom and in what context it was effective; (2) determine if the intervention is delivered as intended; and (3) describe practice engagement, commitment and capacity associated with various intervention components. Qualitative data, collected via semi-structured interviews and open-ended questions, will be used to gather in-depth understanding of the (1) satisfaction, utility, barriers and enablers; (2) acceptability, uptake and feasibility, and (3) effect of the COVID-19 pandemic on the implementation of the intervention.

CONCLUSION

Findings from the evaluation will provide new knowledge on the implementation of a complex, multi-component intervention at practice-level using their own electronic patient data to enhance secondary prevention of cardiovascular disease.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry (ANZCTR) number ACTRN12619001790134.

Development and delivery of an allied health team intervention for older adults in the emergency department: A process evaluation

Background

There is encouraging evidence that interdisciplinary teams of Health and Social Care Professionals (HSCPs) can enhance patient care in the Emergency Department (ED), especially for older adults with complex needs. However, no formal process evaluations of implementations of ED-based HSCP interventions are available. The study aimed to evaluate the development and delivery of a HSCP team intervention for older adults in the ED of a large Irish teaching hospital.

Methods

Using the Medical Research Council (MRC) Framework for process evaluations, we investigated implementation and delivery, mechanisms of impact, and contextual influences on implementation by analysing the HSCP team’s activity notes and participant recruitment logs, and by carrying out six interviews and four focus groups with 26 participants (HSCP team members, ED doctors and nurses, hospital staff). Qualitative insights were analysed thematically.

Results

The implementation process had three phases (pre-implementation, piloting, and delivery), with the first two described as pivotal to optimise care procedures and build positive stakeholders’ involvement. The team’s motivation and proactive communication were key to promote acceptability and integration in the ED (Theme 1); also, their specialised skills and interdisciplinary approach enhanced patient and staff’s ED experience (Theme 2). The investment and collaboration of multiple stakeholders were described as essential contextual enablers of implementation (Theme 4). Delivering the intervention within a randomised controlled trial fostered credibility but caused frustration among patients and staff (Theme 3).

Discussion

This process evaluation is the first to provide in-depth and practical insights on the complexities of developing and delivering an ED-based HSCP team intervention for older adults. Our findings highlight the importance of establishing a team of HSCPs with a strong interdisciplinary ethos to ensure buy-in and integration in the ED processes. Also, actively involving relevant stakeholders is key to facilitate implementation.

Trial registration

ClinicalTrials.gov, NCT03739515; registered on 12^th November 2018.

Accreditation of Simulation Fellowships and Training Programs: More Checkboxes or Elevating the Field?

SUMMARY STATEMENT

As the field of healthcare simulation matures, formal accreditation for simulation fellowships and training programs has become increasingly available and touted as a solution to standardize the education of those specializing in healthcare simulation. Some simulation experts hold opposing views regarding the potential value of simulation fellowship program accreditation. We report on the proceedings of a spirited debate at the 20th International Meeting on Simulation in Healthcare in January 2020. Pro arguments view accreditation as the logical evolution of a maturing profession: improving training quality through standard setting, providing external validation for individual programs, and enhancing the program's return on investment. Con arguments view accreditation as an incompletely formulated construct; burdensome to the "financially strapped" fellowship director, misaligned with simulation fellows' needs and expectations, and confusing to administrators mistakenly equating accreditation with credentialing. In addition, opponents of accreditation postulate that incorporating curricular standards, practice guidelines, and strategies derived and implemented without rigor, supporting evidence and universal consensus is premature. This narrative review of our debate compares and contrasts contemporary perspectives on simulation fellowship program accreditation, concluding with formal recommendations for learners, administrators, sponsors, and accrediting bodies.

Process evaluation of a multicentre randomised clinical trial of substituting surgical excisions of low-risk basal cell carcinomas from secondary to primary care

OBJECTIVES

In 2016, the SKINCATCH Trial, a clustered multi-centre randomised trial, was initiated to assess whether low-risk basal cell carcinomas (BCCs) can be treated by general practitioners (GPs) without loss of quality of care. The trial intervention consisted of a tailored 2-day educational course on skin cancer management. The aim of this process evaluation was to investigate GPs' exposure to the intervention, implementation of the intervention and experiences with the intervention and trial.

RESEARCH DESIGN AND METHODS

Data on exposure to the intervention, implementation and experiences were obtained at several points during the trial. Complementary quantitative components (ie, surveys, database analysis, medical record analysis) and qualitative components (ie, interviews and focus groups) were used. Quantitative data were analysed using descriptive statistics; qualitative data were summarised (barrier interviews) or audiorecorded, transcribed verbatim and thematically analysed using Atlas.Ti (focus groups).

RESULTS

Following a 100% intervention exposure, results concerning the implementation of the trial showed that aside from the low inclusion rate of patients with low-risk BCCs (n=54), even less excisions of low-risk BCCs were performed (n=40). Although the intervention was experienced as highly positive, several barriers were mentioned regarding the trial including administrative challenges, lack of time and high workload of GPs, low volume of BCC patients and patients declining to participate or requesting a referral to a dermatologist.

CONCLUSIONS

Although GPs' participation in the highly valued training was optimal, several barriers may have contributed to the low inclusion and excision rate of low-risk BCCs. While some of the issues were trial-related, other barriers such as low patient-volume and patients requesting referrals are applicable outside the trial setting as well. This may question the feasibility of substitution of surgical excisions of low-risks BCCs from secondary to primary care in the current Dutch setting.

TRIAL REGISTRATION NUMBER

Trial NL5631 (NTR5746).

In It for the Long Haul: RE-AIM Evaluation of a Preschool Programme Implementing and Maintaining Adult-Initiated Motor Skill Development and Physical Activity across a Two-Year Period

Good motor skills (MS) are considered important for children's social, psychological and physical development and general physical activity (PA) levels. The Motor skill in Preschool study (MiPS) aimed to optimize children's MS through weekly PA sessions. The aim of this study is to use the RE-AIM framework to report the two-year implementation process of MiPS since the programme's initiation. Data were collected through a staff questionnaire based on the RE-AIM framework. Data were collected at three months, one year and two years after initiation. Results show that the pedagogical staff believes that the programme promotes MS in children. Implementation measures only showed medium to low fidelity concerning the core element of performing adult-initiated PA sessions with a duration of at least 45 min 4 days a week. The largest barrier was finding the time to plan these PA sessions. Still, the content of the PA sessions achieved high fidelity scores and the programme was deemed suitable for staff's everyday practice and in alignment with the stated pedagogical goals. The mandatory competence development course was highly valued as strong implementation support. It is notable that there is a large variation in the implementation among the preschools with some struggling more than others.

Process evaluations undertaken alongside randomised controlled trials in the hospital setting: A scoping review

Background

There is increasing recognition of the importance of undertaking process evaluations alongside implementation of health interventions by examining mechanisms of impact and contextual factors. However, a comprehensive synthesis of process evaluations undertaken alongside clinical trials in hospital settings is lacking. We undertook a scoping review to address this gap.

Methods

This review was guided by the methodological framework for scoping studies. Studies were identified using four databases; Ovid Medline, EBSCO CINAHL, EMBASE and Scopus. Two authors independently screened all titles and available abstracts, with a third author available to adjudicate. Studies were eligible for inclusion if they described a process evaluation undertaken alongside a randomised controlled trial in the hospital setting. Data were abstracted by one author and checked by two others and analysed both descriptively and using inductive content analysis.

Results

Data were extracted from 30 articles reporting on 15 trials, most of which were cluster randomised trials (c-RTs) (n = 12). The most common data collection methods used in process evaluations were interviews, questionnaires or surveys, and records or logs. Data analysis revealed three themes relative to how authors: use process data to interpret, understand and explain trial outcomes; evaluate responses to the intervention; and consider the implementation context.

Conclusions

Findings from this review demonstrate the complex nature of intervention implementation in the hospital setting. Overall, there is need for standardised reporting of process evaluations and more explicit descriptions of how authors use frameworks to guide their evaluation.

Process Evaluation of Lumbar Interbody Fusion Surgeries in Five Dutch Hospitals: A Qualitative Analysis

Background and Objectives: Only limited qualitative research concerning instrumented spine surgeries has been published, despite the increasing number of these surgeries and the evident importance of qualitative analysis of the processes surrounding these complex interventions. Current qualitative research is mainly limited to the experiences, emotions and expectations of patients. Insight into the full process, including experiences from the perspective of informal caregivers and healthcare professionals, remains scarce. Materials and Methods: Data were gathered by means of semi-structured face-to-face interviews. In total, there were 27 participants, including 11 patients, 7 informal caregivers and 9 healthcare professionals. The interview process was audiotaped, and each interview was transcribed verbatim. To systematically analyse the gathered data, software for qualitative analysis (NVivo) was used. After immersion in the raw data of transcripts and field notes, a list of broad categories for organising the data into meaningful clusters for analysis was developed. All interviews were coded by the first author, and 25% was independently assessed by the second author. Results: The results of our study describe several promoting and limiting factors concerning the process of lumbar fusion surgery from the perspective of patients, informal caregivers and healthcare providers. The most frequently mentioned promoting factors were: information and opportunities to ask questions during consultations; multidisciplinary consultations; good communication and guidance during hospitalization; and follow-up appointments. The most frequently mentioned limiting factors were: lack of educational material; lack of guidance and communication prior to, during and after hospitalisation. Conclusion: Overall, participants were satisfied with the current healthcare-process in lumbar fusion surgery. However, we found that lack of educational material and guidance during the process led to insecurity about complaints, surgery and recovery. To improve the process of lumbar interbody fusion and to increase patient satisfaction, healthcare providers should focus on guiding and educating patients and informal caregivers about the pre-operative trajectory, the surgery and the recovery. From the healthcare providers’ perspective, the process could be improved by multidisciplinary consultations and a dedicated spine team in the operation room. Although this study focusses on lumbar fusion surgery, results could be translated to other fields of spine surgery and surgery in general.

Participants' Engagement With Telephone Support Interventions to Promote Healthy Feeding Practices and Obesity-Protective Behaviours for Infant Obesity Prevention

BACKGROUND

Participant engagement with program interventions is vital to support intended behaviour changes and outcomes. The aim of this research was to investigate participant engagement with the Communicating Healthy Beginnings Advice by Telephone (CHAT) program, an early childhood obesity prevention program that included interventions for promoting healthy infant feeding practices and obesity-protective behaviours via telephone, and whether engagement with the telephone support program varied by participants' sociodemographic characteristics.

METHODS

This study used de-identified CHAT program data of participants who received the interventions via telephone. Data analysed included 1) participant engagement in telephone support from late pregnancy to 12 months of child's age, 2) demographic characteristics collected at late pregnancy and 3) intervention providers' observations and notes (qualitative data) for 10 participants from each engagement group (low, medium, high) to explore issues discussed during telephone support.

RESULTS

Call completion rate by participants was above sixty percent for all six stages of the telephone support program with more than half of the participants (57%) demonstrating high level of engagement. We found that participants' country of birth, employment status and annual household income were predictors of engagement with the telephone support provided in the CHAT program. The odds of participants' engagement with the telephone support program were 1.68 times higher for Australian born (95% CI 1.07 - 2.62), 1.63 times higher for participants who were employed (95% CI 1.01 - 2.66) and 1.63 times higher for participants with annual household income ≥AUD$80,000 (95% CI 1.02 - 2.60).

CONCLUSIONS

Participant engagement with the program interventions was good. Participants' engagement with the telephone support program was significantly associated with certain socio-demographic characteristics. Australian born participants, and participants associated with higher household income and employment engaged significantly more with the telephone support provided in the CHAT program. Additionally, the program engaged more participants older than 30 years of age and those who spoke English at home. The program provided unintended personal benefits to some participants with high engagement level due to their various psychosocial needs such as domestic violence, mental health and sleep related issues. Although not an intended benefit of the intervention, psychosocial needs of participants were met which was a likely factor for mothers' engagement with the program. This is an important factor that needs to be considered while implementing future programs or scale up of this program.

Optimizing red blood cell transfusion practices in the intensive care unit: a multi-phased health technology reassessment

Background

Health technology reassessment (HTR) is a process to manage existing health technologies to ensure ongoing optimal use. A model to guide HTR was developed; however, there is limited practical experience. This paper addresses this knowledge gap through the completion of a multi-phase HTR of red blood cell (RBC) transfusion practices in the intensive care unit (ICU).

Objective

The HTR consisted of three phases and here we report on the final phase: the development, implementation, and evaluation of behavior change interventions aimed at addressing inappropriate RBC transfusions in an ICU.

Methods

The interventions, comprised of group education and audit and feedback, were co-designed and implemented with clinical leaders. The intervention was evaluated through a controlled before-and-after pilot feasibility study. The primary outcome was the proportion of potentially inappropriate RBC transfusions (i.e., with a pre-transfusion hemoglobin of 70 g/L or more).

Results

There was marked variability in the monthly proportion of potentially inappropriate RBC transfusions. Relative to the pre-intervention phase, there was no significant difference in the proportion of potentially inappropriate RBC transfusions post-intervention. Lessons from this work include the importance of early and meaningful engagement of clinical leaders; tailoring the intervention modalities; and, efficient access to data through an electronic clinical information system.

Conclusions

It was feasible to design, implement, and evaluate a tailored, multi-modal behavior change intervention in this small-scale pilot study. However, early evaluation of the intervention revealed no change in technology use leading to reflection on the important question of how the HTR model needs to be improved.

Lessons learned during implementation of MR-guided High-Intensity Focused Ultrasound treatment of uterine fibroids

BACKGROUND

Although promising results have been reported for Magnetic Resonance image-guided High-Intensity Focused Ultrasound (MR-HIFU) treatment of uterine fibroids, this treatment is not yet widely implemented in clinical practice. During the implementation of a new technology, lessons are learned and an institutional learning-curve often has to be completed. The primary aim of our prospective cohort study was to characterize our learning-curve based on our clinical outcomes. Secondary aims included identifying our lessons learned during implementation of MR-HIFU on a technical, patient selection, patient counseling, medical specialists and organizational level.

RESULTS

Our first seventy patients showed significant symptom reduction and improvement of quality of life at 3, 6 and 12 months after MR-HIFU treatment compared to baseline. After the first 25 cases, a clear plateau phase was reached in terms of failed treatments. The median non-perfused volume percentage of these first 25 treatments was 44.6% (range: 0-99.7), compared to a median of 74.7% (range: 0-120.6) for the subsequent treatments.

CONCLUSIONS

Our findings describe the learning-curve during the implementation of MR-HIFU and include straightforward suggestions to shorten learning-curves for future users. Moreover, the lessons we learned on technique, patient selection, patient counseling, medical specialists and organization, together with the provided supplements, may be of benefit to other institutions aiming to implement MR-HIFU treatment of uterine fibroids. Trial registration ISRCTN14634593. Registered January 12, 2021-Retrospectively registered, https://www.isrctn.com/ISRCTN14634593 .

Mixed-methods evaluation of the NHS Genomic Medicine Service for paediatric rare diseases: study protocol

Background

A new nationally commissioned NHS England Genomic Medicine Service (GMS) was recently established to deliver genomic testing with equity of access for patients affected by rare diseases and cancer. The overarching aim of this research is to evaluate the implementation of the GMS during its early years, identify barriers and enablers to successful implementation, and provide recommendations for practice. The focus will be on the use of genomic testing for paediatric rare diseases.

Methods

This will be a four-year mixed-methods research programme using clinic observations, interviews and surveys. Study 1 consists of qualitative interviews with designers/implementers of the GMS in Year 1 of the research programme, along with documentary analysis to understand the intended outcomes for the Service. These will be revisited in Year 4 to compare intended outcomes with what happened in practice, and to identify barriers and facilitators that were encountered along the way. Study 2 consists of clinic observations (pre-test counselling and results disclosure) to examine the interaction between health professionals and parents, along with follow-up interviews with both after each observation. Study 3 consists of a longitudinal survey with parents at two timepoints (time of testing and 12 months post-results) along with follow-up interviews, to examine parent-reported experiences and outcomes. Study 4 consists of qualitative interviews and a cross-sectional survey with medical specialists to identify preparedness, facilitators and challenges to mainstreaming genomic testing. The use of theory-based and pre-specified constructs will help generalise the findings and enable integration across the various sub-studies.

Dissemination

We will disseminate our results to policymakers as findings emerge, so any suggested changes to service provision can be considered in a timely manner. A workshop with key stakeholders will be held in Year 4 to develop and agree a set of recommendations for practice.

Implementation of music in the perioperative standard care of colorectal surgery: study protocol of the IMPROVE Study

INTRODUCTION

Perioperative music intervention has been proven effective in reducing anxiety, pain, neurohormonal stress response and medication requirement. Unfortunately, there is a gap between new effective interventions and their (interventions) integration in standard care protocols. The aim of this preimplementation and postimplementation study is to investigate the adherence to a music intervention in a tailored, multilevel, systematic implementation strategy and the initial impact of the implementation on postoperative pain in patients undergoing colorectal surgery.

METHODS AND DESIGN

A monocentre prospective preimplementation and postimplementation study was set up using The Consolidated Framework for Implementation Research to conduct a systematic implementation of music intervention in three phases. Primary outcomes are adherence to the music intervention and the initial impact of the music intervention implementation on postoperative pain scores on the first postoperative day. At least 100 patients will be included in the study. Secondary outcomes include adherence to the implementation strategy, penetration of music intervention in the standard care, assessment of the determinants for implementation, impact of the implemented music intervention on pain during the entire admission, anxiety, medication usage, complications, hospital and intensive care unit (ICU) length of stay.

ETHICS AND DISSEMINATION

The Medical Ethics Review Board of Erasmus MC University Medical Centre Rotterdam, The Netherlands, has approved this protocol. The study is being conducted in accordance with the Declaration of Helsinki. Results of this trial will be published in peer-reviewed scientific journals and conference presentations.

TRIAL REGISTRATION

Dutch Trial Register NL8071.

Guide to Enhanced Recovery for Cancer Patients Undergoing Surgery: ERAS and Oesophagectomy

Enhanced Recovery After Surgery (ERAS) protocols are widely used in oesophageal cancer surgery. Multiple studies have demonstrated that ERAS protocols are associated with a shorter length of stay and a reduction in the incidence of post-operative complications after oesophagectomy. However, there is substantial heterogeneity in the content of ERAS protocols and the delivery of these pathways can be challenging. This paper discusses the key recommendations for ERAS protocols in oesophageal cancer surgery and the barriers and facilitating factors for their successful implementation.

The Transmural Trauma Care Model can be implemented well but some barriers and facilitators should be considered during implementation: a mixed methods study

QUESTIONS

What is the reach, dose delivered, dose received and fidelity of the Transmural Trauma Care Model (TTCM)? What are the barriers and facilitators associated with the implementation of the TTCM?

DESIGN

Mixed-methods process evaluation with quantitative evaluation of the extent to which the TTCM was implemented as intended and qualitative evaluation of barriers and facilitators to its implementation.

PARTICIPANTS

Focus group participants included trauma patients, trauma surgeons, hospital-based physiotherapists and primary care network physiotherapists.

OUTCOME MEASURES

Implementation was assessed with reach, dose delivered, dose received and fidelity.

DATA ANALYSIS

A framework method was used to analyse the focus groups and the 'constellation approach' was used to categorise barriers and facilitators into three categories: structure, culture and practice.

RESULTS

The TTCM's reach was 81%, its dose delivered was 99% and 100%, and its dose received was 95% and 96% for the multidisciplinary TTCM consultation hours at the outpatient clinic for trauma patients and the primary care network physiotherapists, respectively. Various fidelity scores ranged from 66 to 93%. Numerous barriers and facilitators associated with the implementation of the TTCM were identified and categorised.

CONCLUSION

This process evaluation showed that the TTCM was largely implemented as intended. Furthermore, various facilitators and barriers were identified that need to be considered when implementing the TTCM more widely. Differences were found among stakeholders but they were generally of the opinion that if the barriers were overcome, the quality of care and patient satisfaction were likely to improve significantly after implementing the TTCM.

REGISTRATION

NTR5474.

Integrated Management of Neonatal and Childhood Illness Training in Lao PDR: A Pilot Study of an Adaptable Approach to Training and Supervision

Integrated Management of Neonatal and Childhood Illness (IMNCI) has been part of the national strategy for child health in Lao Peoples Democratic Republic since 2003. The program, while running for an extended period, has faced multiple challenges including maintaining the teaching quality for the implementation of the IMNCI guidelines and a structure to enable and support healthcare workers trained to apply the training in their workplace. A revised training model that focused on building skills for teaching according to adult learning principles in a pool of facilitators, a practical and hands-on training workshop for healthcare workers, and the establishment of a program of health center supervision was developed and implemented in three provinces. Participants in the revised model reported increased confidence in implementing IMNCI guidelines, they demonstrated competence in the steps of IMNCI and on follow-up assessment at a supervision visit were found to have improved patient care through the measurement of pediatric case management scores. This study highlights the importance of a focus on education to ensure the translation of guidelines into practice and thereby lead to improvements in the quality of pediatric care. The IMNCI training approach is acceptable and valued by healthcare worker participants.

Implementing a multidisciplinary psychotropic medication review among nursing home residents with dementia: a process evaluation

OBJECTIVES

Before drawing conclusions on the contribution of an effective intervention to daily practice and initiating dissemination, its quality and implementation in daily practice should be optimal. The aim of this process evaluation was to study these aspects alongside a randomized controlled trial investigating the effects of a multidisciplinary biannual medication review in long-term care organizations (NTR3569).

DESIGN

Process evaluation with multiple measurements.

SETTING

Thirteen units for people with dementia in six long-term care organizations in the Netherlands.

PARTICIPANTS

Physicians, pharmacists, and nursing staff of participating units.

INTERVENTION

The PROPER intervention is a structured and biannually repeated multidisciplinary medication review supported by organizational preparation and education, evaluation, and guidance.

MEASUREMENTS

Web-based questionnaires, interviews, attendance lists of education sessions, medication reviews and evaluation meetings, minutes, evaluation, and registration forms.

RESULTS

Participation rates in education sessions (95%), medication reviews (95%), and evaluation meetings (82%) were high. The intervention's relevance and feasibility and applied implementation strategies were highly rated. However, the education sessions and conversations during medication reviews were too pharmacologically oriented for several nursing staff members. Identified barriers to implementation were required time, investment, planning issues, and high staff turnover; facilitators were the positive attitude of professionals toward the intervention, the support of higher management, and the appointment of a local implementation coordinator.

CONCLUSION

Implementation was successful. The commitment of both higher management and professionals was an important factor. This may partly have been due to the subject being topical; Dutch long-term-care organizations are pressed to lower inappropriate psychotropic drug use.

Types of implementation of the dementia-specific case conference concept WELCOME-IdA in nursing homes: a qualitative process evaluation of the FallDem effectiveness trial

Background

Dementia is regularly associated with behavioral and psychological symptoms of dementia (BPSD, also referred to as challenging behavior). Structured dementia-specific case conferences (DSCCs) enable nursing staff in nursing homes (NHs) to analyze and handle the BPSD of residents with dementia. The FallDem trial estimated the effectiveness of the structured DSCC intervention WELCOME-IdA (Wittener model of case conferences for people with dementia – the Innovative dementia-oriented Assessment tool) in NHs in Germany. No significant change in the overall prevalence of challenging behavior was found. A multipart process evaluation was conducted to explain this result.

Methods

This qualitative process evaluation of the response of individuals, perceived maintenance, effectiveness, and unintended consequences was part of the multipart process evaluation that followed the framework by Grant et al. (Trials 14: 15, 2013). It used the data from semi-structured telephone interviews and focus group interviews with nurses and managers as secondary data. Selected domains of the Consolidated Framework for Implementation Research (CFIR) were used as deductive categories for a directed content analysis.

Results

The interviewees in all NHs appraised WELCOME-IdA as generating positive change, although it proved important that some adjustments were made to the intervention and the organization. Thirteen CFIR constructs out of the domains intervention characteristics, inner setting, and process proved to be essential for understanding the different course that the implementation of WELCOME-IdA took in each of the four NHs. This is reflected in three types of WELCOME-IdA implementation: (1) priority on adjusting the intervention to fit the organization, (2) priority on adjusting the organization to fit the intervention, and (3) no setting of priorities in adjusting either the organization or the intervention.

Conclusion

The unsatisfying results of the FallDem effectiveness trial can in part be explained with regard to the interplay between the intervention and the implementation which was revealed in the processes that occurred in the organizations during the implementation of the WELCOME-IdA intervention. Future implementation of WELCOME-IdA should be tailored based on an analysis of the organization’s readiness, resources, and capacities and should also define custom-made intervention and implementation outcomes to measure success. Furthermore, our results confirm that the CFIR can be used beneficially to conduct process evaluations.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-021-00191-0.

Strategies to enhance routine physical activity in care home residents: the REACH research programme including a cluster feasibility RCT

Background

Care home residents are mainly inactive, leading to increased dependency and low mood. Although exercise classes may increase activity, a more sustainable model is to engage staff and residents in increasing routine activity.

Objectives

The objectives were to develop and preliminarily test strategies to enhance the routine physical activity of care home residents to improve their physical, psychological and social well-being through five overlapping workstreams.

Design

This trial had a mixed-methods research design to develop and test the feasibility of undertaking an evaluative study consisting of gaining an understanding of the opportunities for and barriers to enhancing physical activity in care homes (workstream 1); testing physical activity assessment instruments (workstream 2); developing an intervention through a process of intervention mapping (workstream 3); refining the provisional intervention in the care home setting and clarifying outcome measurement (workstream 4); and undertaking a cluster randomised feasibility trial of the intervention [introduced via three facilitated workshops at baseline (with physiotherapist input), 2 weeks (with artist input) and 2 months], with embedded process and health economic evaluations (workstream 5).

Setting

The trial was set in 12 residential care homes differing in size, location, ownership and provision in Yorkshire, UK.

Participants

The participants were elderly residents, carers, managers and staff of care homes.

Intervention

The intervention was MoveMore, designed for the whole home, to encourage and support the movement of residents in their daily routines.

Main outcome measures

The main outcome measures related to the feasibility and acceptability of implementing a full-scale trial in terms of recruitment and retention of care homes and residents, intervention delivery, completion and reporting of baseline data and outcomes (including hours of accelerometer wear, hours of sedentary behaviour and hours and type of physical activity), and safety and cost data (workstream 5).

Results

Workstream 1 – through a detailed understanding of life in a care home, a needs assessment was produced, and barriers to and facilitators of activity were identified. Key factors included ethos of care; organisation, management and delivery of care; use of space; and the residents’ daily routines. Workstream 2 – 22 (73.3%) out of 30 residents who wore a hip accelerometer had valid data (≥ 8 hours on ≥ 4 days of the week). Workstream 3 – practical mechanisms for increasing physical activity were developed, informed by an advisory group of stakeholders and outputs from workstreams 1 and 2, framed by the process of intervention mapping. Workstream 4 – action groups were convened in four care homes to refine the intervention, leading to further development of implementation strategies. The intervention, MoveMore, is a whole-home intervention involving engagement with a stakeholder group to implement a cyclical process of change to encourage and support the movement of residents in their daily routines. Workstream 5 – 12 care homes and 153 residents were recruited to the cluster randomised feasibility trial. Recruitment in the care homes varied (40–89%). Five care homes were randomised to the intervention and seven were randomised to usual care. Predetermined progression criteria were recruitment of care homes and residents (green); intervention delivery (amber); and data collection and follow-up – 52% of residents provided usable accelerometer data at 9 months (red), > 75% of residents had reported outcomes at 9 months (green, but self-reported resident outcomes were red), 26% loss of residents to follow-up at 9 months [just missing green criterion (no greater than 25%)] and safety concerns (green).

Limitations

Observations of residents’ movements were not conducted in private spaces. Working with care home residents to identify appropriate outcome measures was challenging. Take-up of the intervention was suboptimal in some sites. It was not possible to make a reliably informed decision on the most appropriate physical activity end point(s) for future use in a definitive trial.

Conclusions

A whole-home intervention was developed that was owned and delivered by staff and was informed by residents and staff. The feasibility of conducting a cluster randomised controlled trial was successfully tested: the target numbers of care homes and residents were recruited, demonstrating that it is possible to recruit care home residents to a cluster randomised trial, although this process was time-consuming and resource heavy. A large data set was collected, which provided a comprehensive picture of the environment, residents and staff in care homes. Extensive quantitative and qualitative work comprehensively explored a neglected area of health and social care research. Completion of ethnographic work in a range of settings enabled the production of an in-depth picture of life in care homes that will be helpful for other researchers considering organisational change in this setting.

Future work

The content and delivery of the intervention requires optimisation and the outcome measurement requires further refinement prior to undertaking a full trial evaluation. Consideration could be given to a recommended, simplified, core outcome set, which would facilitate data collection in this population.

Trial registration

Current Controlled Trials ISRCTN16076575.

Funding

This project was funded by the National Institute for Health Research (NIHR) Programme Grant for Applied Research programme and will be published in full in Programme Grant for Applied Research; Vol. 9, No. 9. See the NIHR Journals Library website for further project information.

A multicentre cluster-randomized clinical trial to improve antibiotic use and reduce length of stay in hospitals: comparison of three measurement and feedback methods

BACKGROUND

Various metrics of hospital antibiotic use might assist in guiding antimicrobial stewardship (AMS).

OBJECTIVES

To compare patient outcomes in association with three methods to measure and feedback information on hospital antibiotic use when used in developing an AMS intervention.

METHODS

Three methods were randomly allocated to 42 clusters from 21 Dutch hospitals: (1) feedback on quantity of antibiotic use [DDD, days-of-therapy (DOT) from hospital pharmacy data], versus feedback on (2) validated, or (3) non-validated quality indicators from point prevalence studies. Using this feedback together with an implementation tool, stewardship teams systematically developed and performed improvement strategies. The hospital length of stay (LOS) was the primary outcome and secondary outcomes included DOT, ICU stay and hospital mortality. Data were collected before (February-May 2015) and after (February-May 2017) the intervention period.

RESULTS

The geometric mean hospital LOS decreased from 9.5 days (95% CI 8.9-10.1, 4245 patients) at baseline to 9.0 days (95% CI 8.5-9.6, 4195 patients) after intervention (P < 0.001). No differences in effect on LOS or secondary outcomes were found between methods. Feedback on quality of antibiotic use was used more often to identify improvement targets and was preferred over feedback on quantity of use. Consistent use of the implementation tool seemed to increase effectiveness of the AMS intervention.

CONCLUSIONS

The decrease in LOS versus baseline likely reflects improvement in the quality of antibiotic use with the stewardship intervention. While the outcomes with the three methods were otherwise similar, stewardship teams preferred data on the quality over the quantity of antibiotic use.

Improving Management of Infantile Spasms by Adopting Implementation Science

Over the last several decades, significant progress has been made in the discovery of appropriate therapy in the management of infantile spasms (IS). Based on several well-controlled studies, the American Academy of Neurology and the Child Neurology Society have published the current best practice parameters for the treatment of IS. However, dissemination and implementation of evidence-based guidelines remain a significant challenge. Though the number of well-performed controlled trials and systematic reviews is increasing exponentially, the proportion of valuable new information subsequently embedding into the routine clinical care is significantly lower. Planned and systematic implementation of evidence-based interventions in a given health care structure may outstrip the benefits of discovering a new insight, procedure, or drug in another controlled setting. Implementation problems can be broad-ranging to hinder effective, efficient, safe, timely, and patient-centered care without significant variation. The first part of this review article provides a detailed summary of some crucial comparative treatment studies of IS available in the literature. In the second part, practical challenges to mitigate the gap between knowledge and practice to improve outcomes in the management of IS has been explored, and a consolidated framework approach for systematic implementation research methodology has been discussed to implement evidence-based guidelines for the management of IS. Although large multicenter controlled studies will help gather quality evidence in the treatment of IS, a more comprehensive range of scientific methodologies, including qualitative research and mixed research methodologies, will hold the more considerable promise for implementing evidence-based practices in the health care system.

Evaluating the CYP-IAPT transformation of child and adolescent mental health services in Cambridgeshire, UK: a qualitative implementation study

Background

The Children and Young People's Improving Access to Psychological Therapies (CYP-IAPT) programme was introduced to transform Child and Adolescent Mental Health Services (CAMHS) across England. The programme comprised a set of principles that local CAMHS partnerships were expected to operationalise and embed with the aim of increasing access to services and improving the quality of care. This study explored how the implementation of the CYP-IAPT programme was executed and experienced by CAMHS professionals in the county of Cambridgeshire (UK), and the extent to which the CYP-IAPT principles were perceived to be successfully embedded into everyday practice.

Methods

We analysed 275 documents relating to the CYP-IAPT programme issued between 2011 and 2015. We also conducted a thematic analysis of 20 qualitative interviews, undertaken at two time points, with professionals from three CAMHS teams in Cambridgeshire. Analysis was informed by implementation science frameworks.

Results

Document analysis suggested that the CYP-IAPT programme was initially not clearly defined and lacked guidance on how to operationalise key programme principles and apply them in everyday practice. There was also a degree of programme evolution over time, which made it difficult for local stakeholders to understand the scope and aims of CYP-IAPT. Interviews with staff showed low coherent understanding of the programme, variable levels of investment among stakeholders and difficulties in collaborative working. Barriers and facilitators to programme implementation were identified at individual, service and strategic levels. These in turn impacted the local implementation efforts and sustainability of the programme in Cambridgeshire.

Conclusions

We identified factors relating to programme design and national and local implementation planning, as well as features of inner and outer context, which impacted on the delivery and sustainability of the programme. These findings can be drawn upon to inform the development and delivery of other local and national quality improvement (QI) initiatives relating to children and young people's mental health.

Practical generic guidelines for paediatric-to-adult transition for adolescents with chronic disease

BACKGROUND

The last 20 years have seen many attempts to improve transition to adult healthcare for adolescents with chronic disease, but there is currently no established consensus on generic practices. Our goal was to identify relevant and pragmatic guidelines for transition practice for each step of this process (before, during and after transfer), applicable to a wide range of chronic illnesses and health services, via a participatory approach involving all the key stakeholders.

METHODS

We conducted interviews and a literature review to elaborate a questionnaire for use in an online 2-round Delphi survey. The survey panel included 36 French health and social professionals from different care settings, and young adults and parents with an experience of healthcare transition related to all types of chronic disease.

RESULTS

The survey consensus identified 19 items on feasibility and relevance criteria, which form the guidelines. It is composed of five practices to be adopted during preparation in paediatrics, seven practices in the active phase of transition and seven in adult care. Two guidelines achieved complete consensus: having a longer consultation for the first appointment with the adult doctor, and keeping the same adult doctor throughout follow-up. A further 36 items met the criterion of relevance, but were deemed unfeasible.

CONCLUSIONS

Taking into account all stakeholder views and the real-world applicability of care practices enabled us to elaborate consensual guidelines whose implementation requires no additional health service resources.

Process Evaluation of a Community-Based Microbial Larviciding Intervention for Malaria Control in Rural Tanzania

Microbial larviciding can be an effective component of integrated vector management malaria control schemes, although it is not commonly implemented. Moreover, quality control and evaluation of intervention activities are essential to evaluate the potential of community-based larviciding interventions. We conducted a process evaluation of a larval source management intervention in rural Tanzania where local staff were employed to apply microbial larvicide to mosquito breeding habitats with the aim of long-term reductions in malaria transmission. We developed a logic model to guide the process evaluation and then established quantitative indicators to measure intervention success. Quantitative analysis of intervention reach, exposure, and fidelity was performed to assess larvicide application, and interviews with larviciding staff were reviewed to provide context to quantitative results. Results indicate that the intervention was successful in terms of reach, as staff applied microbial larvicide at 80% of identified mosquito breeding habitats. However, the dosage of larvicide applied was sufficient to ensure larval elimination at only 26% of sites, which does not meet the standard set for intervention fidelity. We propose that insufficient training and protocol adaptation, environment and resource issues, and human error contributed to low larvicide application rates. This demonstrates how several small, context-specific details in sum can result in meaningful differences between intervention blueprint and execution. These findings may serve the design of other larval source management interventions by demonstrating the value of additional training, supervision, and measurement and evaluation of protocol adherence.

Process and outcome evaluation of a CBME intervention guided by program theory

RATIONALE

Competency-based medical education (CBME) has gained momentum as an improved training model, but literature on outcomes of CBME, including evaluation of implementation processes, is minimal. We present a case for the following: (a) the development of a program theory is essential prior to or in the initial stages of implementation of CBME; (b) the program theory should guide the strategies and methods for evaluation that will answer questions about anticipated and unintended outcomes; and (c) the iterative process of testing assumptions and hypotheses will lead to modifications to the program theory to inform best practices of implementing CBME.

METHODS

We use the Triple C Competency-based Curriculum as a worked example to illustrate how process and outcome evaluation, guided by a program theory, can lead to meaningful enhancement of CBME curriculum, assessment, and implementation strategies. Using a mixed methods design, the processes and outcomes of Triple C were explored through surveys, interviews, and historical document review, which captured the experiences of various stakeholders.

FINDINGS

The theory-led program evaluation process was able to identify areas that supported CBME implementation: the value of a strong nondirective national vertical core supporting the transformation in education, program autonomy, and adaptability to pre-existing local context. Areas in need of improvement included the need for ongoing support from College of Family Physicians of Canada (CFPC) and better planning for shifts in program leadership over time.

CONCLUSIONS

Deliberately pairing evaluation alongside change is an important activity and, when accomplished, yields valuable information from the experiences of those implementing and experiencing a program. Evaluation and the development of an updated program theory facilitate the introduction of new changes and theories that build on these findings, which also supports the desired goal of contributing toward cumulative science rather than "reinventing the wheel."

Effects of de-implementation strategies aimed at reducing low-value nursing procedures: a systematic review and meta-analysis

Background

In the last decade, there is an increasing focus on detecting and compiling lists of low-value nursing procedures. However, less is known about effective de-implementation strategies for these procedures. Therefore, the aim of this systematic review was to summarize the evidence of effective strategies to de-implement low-value nursing procedures.

Methods

PubMed, Embase, Emcare, CINAHL, PsycINFO, Cochrane Central Register of Controlled Trials, Web of Science, and Google Scholar were searched till January 2020. Additionally, reference lists and citations of the included studies were searched. Studies were included that described de-implementation of low-value nursing procedures, i.e., procedures, test, or drug orders by nurses or nurse practitioners. PRISMA guideline was followed, and the ‘Cochrane Effective Practice and Organisation of Care’ (EPOC) taxonomy was used to categorize de-implementation strategies. A meta-analysis was performed for the volume of low-value nursing procedures in controlled studies, and Mantel–Haenszel risk ratios (95% CI) were calculated using a random effects model.

Results

Twenty-seven studies were included in this review. Studies used a (cluster) randomized design (n = 10), controlled before-after design (n = 5), and an uncontrolled before-after design (n = 12). Low-value nursing procedures performed by nurses and/or nurse specialists that were found in this study were restraint use (n = 20), inappropriate antibiotic prescribing (n = 3), indwelling or unnecessary urinary catheters use (n = 2), ordering unnecessary liver function tests (n = 1), and unnecessary antipsychotic prescribing (n = 1). Fourteen studies showed a significant reduction in low-value nursing procedures. Thirteen of these 14 studies included an educational component within their de-implementation strategy. Twelve controlled studies were included in the meta-analysis. Subgroup analyses for study design showed no statistically significant subgroup effect for the volume of low-value nursing procedures (p = 0.20).

Conclusions

The majority of the studies with a positive significant effect used a de-implementation strategy with an educational component. Unfortunately, no conclusions can be drawn about which strategy is most effective for reducing low-value nursing care due to a high level of heterogeneity and a lack of studies. We recommend that future studies better report the effects of de-implementation strategies and perform a process evaluation to determine to which extent the strategy has been used.

Trial registration

The review is registered in Prospero (CRD42018105100).

Process evaluation of the effects of patient safety auditing in hospital care (part 2)

Objective

To identify factors that explain the observed effects of internal auditing on improving patient safety.

Design setting and participants

A process evaluation study within eight departments of a university medical centre in the Netherlands.

Intervention(s)

Internal auditing and feedback for improving patient safety in hospital care.

Main outcome measure(s)

Experiences with patient safety auditing, percentage implemented improvement actions tailored to the audit results and perceived factors that hindered or facilitated the implementation of improvement actions.

Results

The respondents had positive audit experiences, with the exception of the amount of preparatory work by departments. Fifteen months after the audit visit, 21% of the intended improvement actions based on the audit results were completely implemented. Factors that hindered implementation were short implementation time: 9 months (range 5–11 months) instead of the 15 months’ planned implementation time; time-consuming and labour-intensive implementation of improvement actions; and limited organizational support for quality improvement (e.g. insufficient staff capacity and time, no available quality improvement data and information and communication technological (ICT) support).

Conclusions

A well-constructed analysis and feedback of patient safety problems is insufficient to reduce the occurrence of poor patient safety outcomes. Without focus and support in the implementation of audit-based improvement actions, quality improvement by patient safety auditing will remain limited.

Use of concurrent evaluation to improve implementation of a home fortification programme in Bangladesh: a methodological innovation

Objective:

This paper focuses on the use of ‘concurrent evaluation’ to evaluate a nationally scaled-up programme in Bangladesh that was implemented by BRAC (an international development organisation) using Shasthya Shebika (SS) – volunteer community health workers – to promote home fortification with micronutrient powders (MNP) for children under-five.

Design:

We developed a programme impact pathway to conceptualise the implementation and evaluation strategy and developed a strategic partnership among the key programme stakeholders for better use of evaluation evidence. We developed a multi-method concurrent evaluation strategy to provide insights into the BRAC programme and created provision for course correction to the implementation plan while it was in operation.

Setting:

One hundred sixty-four sub-districts and six urban slums in Bangladesh.

Participants:

Caregivers of children 6–59 months, SS and BRAC’s staff members.

Results:

The evaluation identified low awareness about home fortification among caregivers, inadequate supply and frequent MNP stockouts, and inadequate skills of BRAC’s SS to promote MNP at the community level as hindrances to the achievement of programme goals. The partners regularly discussed evaluation results during and after implementation activities to assess progress in programme coverage and any needs for modification. BRAC initiated a series of corrections to the original implementation plan to address these challenges, which improved the design of the MNP programme; this resulted in enhanced programme outcomes.

Conclusions:

Concurrent evaluation is an innovative approach to evaluate complex real-world programmes. Here it was utilised in implementing a large-scale nutrition programme to measure implementation process and effectiveness.

Using clinical simulation to study how to improve quality and safety in healthcare

Simulation can offer researchers access to events that can otherwise not be directly observed, and in a safe and controlled environment. How to use simulation for the study of how to improve the quality and safety of healthcare remains underexplored, however. We offer an overview of simulation-based research (SBR) in this context. Building on theory and examples, we show how SBR can be deployed and which study designs it may support. We discuss the challenges of simulation for healthcare improvement research and how they can be tackled. We conclude that using simulation in the study of healthcare improvement is a promising approach that could usefully complement established research methods.

Using User-Feedback to Develop a Website: MyT1DHope, for Parents of Children with T1D

Effectively caring for a child with type-1 diabetes (T1D) can be extremely demanding and stressful for the entire family. The difficulties of caring for a child with T1D impact the psychological and physical well-being of the parents. Interventions aimed at reducing stress and increasing coping may improve parental outcomes. This project sought to develop a resource to be used as a preliminary intervention step for these parents. This research was conducted in two parts. Part 1 includes three stages of formative evaluation, including a focus group, interviews, and a survey with parents to examine the current landscape of diabetes websites as a resource. Using data from Part 1, Part 2 of the study included the identification of a theory to guide the design of a website, the development of the website, and an 8-week pilot study of the website with parents with a child with T1D. Significant improvements were found in diabetes knowledge and caregivers' satisfaction of life. No changes were found in social support, self-efficacy, or quality of life. Overall, the data show that the parents were generally satisfied with the website. Through this work, researchers were able to develop a deeper understanding of the needs of parents caring for a child with T1D to develop the initial component of a larger intervention.

Qualitative instruments involving clients as co-researchers to assess and improve the quality of care relationships in long-term care: an evaluation of instruments to enhance client participation in quality research

OBJECTIVES

Enhancing the active involvement of clients as co-researchers is seen as a promising innovation in quality research. The aim of this study was to assess the feasibility and usability of five qualitative instruments used by co-researchers for assessing the quality of care relationships in long-term care.

DESIGN AND SETTING

A qualitative evaluation was performed in three care organisations each focused on one of the following three client groups: frail older adults, people with mental health problems and people with intellectual disabilities. A total of 140 respondents participated in this study. The data comprised observations by researchers and experiences from co-researchers, clients and professionals.

RESULTS

Two instruments scored best on feasibility and usability and can therefore both be used by co-researchers to monitor the quality of care relationships from the client perspective in long-term care.

CONCLUSIONS

The selected instruments let co-researchers interview other clients about their experiences with care relationships. The study findings are useful for long-term care organisations and client councils who are willing to give clients an active role in quality improvement.

Validation of the Chinese Non-pharmacological Therapy Experience Scale in persons with intellectual disability

Aim

To validate the 4- and 7-point Chinese Non-pharmacological Therapy Experience Scales and test the psychometric properties of the scales on persons with intellectual disability.

Design

A validation study.

Methods

Sixty-seven persons with intellectual disability were recruited from six hostels or centres for persons with intellectual disability in Hong Kong. A total of 1,163 and 1,161 observations were collected by the trained observers with the 4-point and 7-point scales, respectively. The floor and ceiling effects, inter-rater reliability, internal consistency, responsiveness of both scales and the scale equivalence were examined.

Results

The Cronbach's α of the 4- and 7-point scales was .762 and .797, respectively. The correlation between the two scales was 0.906. The inter-rater reliability of the 4- and 7-point scales was 0.774 and 0.835, respectively. Neither scale had the floor or ceiling effects. The effect size of the 7-point scale was consistently higher than that of the 4-point scale.

Process evaluation of a tailored work-related support intervention for patients diagnosed with gastrointestinal cancer

Purpose

To perform a process evaluation of a tailored work-related support intervention for patients diagnosed with gastrointestinal cancer.

Methods

The intervention comprised three tailored psychosocial work-related support meetings. To outline the process evaluation of this intervention, we used six key components: recruitment, context, reach, dose delivered, dose received and fidelity. Data were collected using questionnaires, checklists and research logbooks and were analysed both quantitatively and qualitatively.

Results

In total, 16 hospitals, 33 nurses and 7 oncological occupational physicians (OOPs) participated. Analysis of the six key components revealed that the inclusion rate of eligible patients was 47%. Thirty-eight intervention patients were included: 35 actually had a first meeting, 32 had a second and 17 had a third. For 31 patients (89%), the first meeting was face to face, as per protocol. However, in only 32% of the cases referred to support type A (oncological nurse) and 13% of the cases referred to support type B (OOP), the first meeting was before the start of the treatment, as per protocol. The average duration of the support type A meetings was around the pre-established 30 min; for the OOPs, the average was 50 min. Protocol was easy to follow according to the healthcare professionals. Overall, the patients considered the intervention useful.

Conclusions

This study has shown that the strategy of tailored work-related support is appreciated by both patients and healthcare professionals and applicable in clinical practice.

Implications for Cancer survivors

The intervention was appreciated by patients; however, whether the timing of the work-related support was adequate (i.e. before treatment was started) requires further research.

Trial registration

NTR5022.

Electronic supplementary material

The online version of this article (10.1007/s11764-019-00797-3) contains supplementary material, which is available to authorized users.

Reducing waste: a guidelines-based approach to reducing inappropriate vitamin D and TSH testing in the inpatient rehabilitation setting

Background

Laboratory overutilisation increases healthcare costs, and can lead to overdiagnosis, overtreatment and negative health outcomes. Discipline-specific guidelines do not support routine testing for Vitamin D and thyroid-stimulating hormone (TSH) in the inpatient rehabilitation setting, yet 94% of patients had Vitamin D and TSH tests on admission to inpatient rehabilitation at our institution. Our objective was to reduce Vitamin D and TSH testing by 25% on admission to inpatient Stroke, Spinal Cord Injury, Acquired Brain Injury and Amputee Rehabilitation units.

Methods

A fishbone framework for root cause analysis revealed potential causes underlying overutilisation of Vitamin D and TSH testing. A series of Plan-Do-Study-Act (PDSA) cycles were introduced to target remediable factors, starting with an academic detailing intervention with key stakeholders that reviewed applicable clinical guidelines for each patient care discipline and the rationale for reducing admission testing. Simultaneously, computerised clinical decision support (CCDS) limited Vitamin D testing to specific criteria. Audit and feedback were used in a subsequent PDSA cycle. Frequency of Vitamin D and TSH testing on admission was the primary outcome measure. The number of electronic admission order caresets containing automatic Vitamin D and/or TSH orders before and after the interventions was the process measure. Rate of Vitamin D supplementation and changes in thyroid-related medication were the balancing measures.

Results

After implementation, 2.9% of patients had admission Vitamin D testing (97% relative reduction) and 53% of patients had admission TSH testing (43% relative reduction). Admission order caresets with prepopulated Vitamin D and TSH orders decreased from 100% (n=6) to 0%. The interventions were successful; similar to previous literature, CCDS was more effective than education and audit and feedback interventions alone. The interventions represent >$9000 annualised savings.

A process evaluation of the quality improvement collaborative for a community-based family planning learning site in Uganda

Background: High-quality family planning (FP) services have been associated with increased FP service demand and use, resulting in improved health outcomes for women. Community-based family planning (CBFP) is a key strategy in expanding access to FP services through community health workers or Village Health Team (VHTs) members in Uganda. We established the first CBFP learning site in Busia district, Uganda, using a quality improvement collaborative (QIC) model. This process evaluation aims to understand the QIC adaptation process, supportive implementation factors and trends in FP uptake and retention.

Methods: We collected data from two program districts: Busia (learning site) and Oyam (scale-up). We used a descriptive mixed-methods process evaluation design: desk review of program documents, program monitoring data and in-depth interviews and focus group discussions.

Results: The quality improvement (QI) process strengthened linkages between health services provided in communities and health centers. Routine interaction of VHTs, clients and midwives generated improvement ideas. Participants reported increased learning through midwife mentorship of VHTs, supportive supervision, monthly meetings, data interpretation and learning sessions. Three areas for potential sustainability and institutionalization of the QI efforts were identified: the integration of QI into other services, district-level plans and support for the QIC and motivation of QI teams. Challenges in the replication of this model include the community-level capacity for data recording and interpretation, the need to simplify QI terminology and tools for VHTs and travel reimbursements for meetings. We found positive trends in the number of women on an FP method, the number of returning clients and the number of couples counseled.

Conclusions: A QIC can be a positive approach to improve VHT service delivery. Working with VHTs on QI presents specific challenges compared to working at the facility level. To strengthen the implementation of this CBFP QIC and other community-based QICs, we provide program-relevant recommendations.