Systematic review of the use of process evaluations in knowledge translation research

Process Evaluations of Diabetes Self-Management Programs: A Systematic Review of the Literature

OBJECTIVE

To conduct a systematic review of process evaluations (PEs) of diabetes self-management programs (DSMPs).

DATA SOURCE

An electronic search using Medline (Ovid), Embase (Ovid), CINAHL (Ensco), Academic Search (Ebsco), and APA PsycInfo (Ebsco).

STUDY INCLUSION AND EXCLUSION CRITERIA

Peer-reviewed, empirical quantitative, qualitative, or mixed-method studies were included if they (1) were a traditional, group-based DSMP, (2) involved adults at least 18 years with T1DM or T2DM, (3) were a stand-alone or embedded PE, and (4) published in English.

DATA EXTRACTION

The following process evaluation outcomes were extracted: fidelity, dose delivered, dose received, reach, recruitment, retention, and context. Additional items were extracted, (eg, process evaluation type, data collection methods; theories; frameworks or conceptual models used to guide the process evaluation, and etc).

DATA SYNTHESIS

Due to heterogeneity across studies, studies were synthesized qualitatively (narratively).

RESULTS

Sixty-eight studies (k) in 78 articles (n) (k = 68; n = 78) were included. Most were mixed methods of low quality. Studies were typically integrated into outcome evaluations vs being stand-alone, lacked theoretical approaches to guide them, and incorporated limited outcomes such as dose received, reach, and retention.

CONCLUSION

Future research should 1) implement stand-alone theoretically grounded PE studies and 2) provide a shared understanding of standardized guidelines to conduct PEs. This will allow public health practitioners and researchers to assess and compare the quality of different programs to be implemented.

Process evaluation of a parent-child communication intervention for adolescent sexual and reproductive health in Uganda

BACKGROUND

Previous initiatives concerning adolescent sexual and reproductive health (SRH) education in Low-or-Middle Income Countries (LMICs) have been limited by cultural norms and misinformation perpetuated within families. Responding to the paucity of research on the implementation of SRH interventions in LMICs and limited knowledge regarding their mechanisms, this study undertakes a process evaluation of a parent-focused intervention to promote parent-adolescent communication about SRH in Uganda.

METHODS

This paper explores the implementation, contextual factors and mechanisms of impact of the intervention, using the Medical Research Council (MRC) guidelines for process evaluations. Implementation was evaluated through indicators of dose, fidelity and adaptations, acceptability and feasibility. The contextual factors and mechanisms of impact were evaluated to refine the intervention's causal assumptions. Data was collected during April - October 2021 in South-Western Uganda using a mixed-methods approach, including document analysis, intervention observations, interviews, focus group discussions and most significant change stories.

RESULTS

The acceptability of the intervention was related to its community engagement, the strong rapport with delivery agents, and individual characteristics of participants. Five contextual factors influencing implementation were highlighted; (i) cultural norms, (ii) perceptions about youth SRH, (iii) poverty, (iv) Covid-19 pandemic, and (v) prior research projects in the community. When considering the intervention's mechanisms of impact, four causal pathways were identified; (i) Awareness of SRH needs helped parents overcome stigma, (ii) Parenting skills training improved SRH communication, (iii) Group learning stimulated shared parenting, and (iv) Group learning improved co-parenting.

CONCLUSION

The paper presented three key learnings and corresponding recommendations for future research. Firstly, implementation success was credited to meaningful community engagement which improved acceptability and uptake. Secondly, the complex influences of contextual factors highlighted the need for contextual analysis in research studies to inform intervention design. Finally, this evaluation recognised the interplay between mechanisms of impact and suggested further research consider such combined impacts when designing intervention content.

Primary health care roles of community pharmacists in low- and middle-income countries: A protocol for a mixed methods systematic review

BACKGROUND

The roles of community pharmacists in primary health care cannot be overemphasized, these professionals have evolved from the traditional dispensing to providing health promotion, vaccination services, and medication therapy management as well as other areas of patient care needs. The objectives of this review are to explore evidence of the provision of primary health care services by community pharmacists in low- and middle-income countries (LMICs) and identify barriers to these services and how these barriers can be mitigated.

METHOD

A systematic search will be conducted in Medline, CINAHL, Google scholar, Global Index Medicus, LILACS, and CENTRAL to identify potentially relevant primary literature for inclusion. Also, ProQuest Dissertations and Theses will be search to identify relevant gray literature. The review will be guided by the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols checklist for systematic reviews. All primary research articles regardless of study design exploring or reporting on community pharmacists' involvement in primary health care in LMICs will be considered for possible inclusion. A summary of the included studies will be provided and a quality assessment will be conducted using the 2018 version of the mixed methods analytical tool.

EXPECTED RESULTS

Findings from this review will add to the evidence in the literature and cause the need for policy change to empower the community pharmacy practice by providing a friendlier legal framework for delivering services in tandem with international best practices. PROSPERO registration ID: CRD42023357312.

Why Is It So Hard to Evaluate Knowledge Exchange? Comment on "Sustaining Knowledge Translation Practices: A Critical Interpretive Synthesis"

Despite a growth in knowledge translation (KT) or exchange activities, and a smaller growth in their evaluations, it remains challenging to identify evidence of efficacy. This could be due to well-documented political and logistical difficulties involved in evaluating knowledge exchange interventions. By bringing in theory from science and technology studies (STS), Borst et al1 offer a new way of thinking about this problem. Most KT evaluations draw on health research traditions; centralising comparability, efficacy, and so on. Borst et al propose focusing on the work it takes to move knowledge over boundaries between these communities, seeing relationships as interactions, not just conduits for evidence. They show how 'context' can be understood as a mutual creation, not a static environment; and that institutions shape behaviours, rather than merely being sites or platforms for evidence mobilisation. Seeing KT as a creative, active practice opens new ways to design and evaluate KT mechanisms.

Implementation of a frailty intervention in the transition from hospital to home: a realist process evaluation protocol for the FORTRESS trial

INTRODUCTION

Frailty in Older people: Rehabilitation, Treatment, Research Examining Separate Settings (the FORTRESS study) is a multisite, hybrid type II, stepped wedge, cluster, randomised trial examining the uptake and outcomes of a frailty intervention. The intervention is based on the 2017 Asia Pacific Clinical Practice Guidelines for the Management of Frailty and begins in the acute hospital setting and transitions to the community. The success of the intervention will require individual and organisational behaviour change within a dynamic health system. This process evaluation will examine the multiple variables at play in the context and mechanism of the frailty intervention to enhance understanding of the outcomes of the FORTRESS study and how the outcomes can be translated from the trial into broader practice.

METHODS AND ANALYSIS

The FORTRESS intervention will recruit participants from six wards in New South Wales and South Australia, Australia. Participants of the process evaluation will include trial investigators, ward-based clinicians, FORTRESS implementation clinicians, general practitioners and FORTRESS participants. The process evaluation has been designed using realist methodology and will occur in parallel to the FORTRESS trial. A mixed-method approach will be used with qualitative and quantitative data collected from interviews, questionnaires, checklists and outcome assessments. Qualitative and quantitative data will be examined for CMOCs (Context, Mechanism, Outcome Configurations) and programme theories will be developed, tested and refined. This will facilitate development of more generalisable theories to inform translation of frailty intervention within complex healthcare systems.

ETHICS AND DISSEMINATION

Ethical approval for the FORTRESS trial, inclusive of the process evaluation, has been obtained from the Northern Sydney Local Health District Human Research Ethics Committees reference number 2020/ETH01057. Recruitment for the FORTRESS trial uses opt-out consent. Dissemination will be via publications, conferences and social media.

TRIAL REGISTRATION NUMBER

ACTRN12620000760976p (FORTRESS trial).

An implementation evaluation of the physical activity counseling for in-patients with major depressive disorder (PACINPAT) intervention: a randomized controlled trial

BACKGROUND

The physical activity counseling for in-patients with major depression (PACINPAT) randomized controlled trial was launched to tackle physical inactivity for in-patients with major depressive disorder. Evidence shows that despite potential treatment effects, physical inactivity is prevalent in this population. To contribute to the assessment of how this in-person and remote, theory-based, individually tailored intervention was designed, received and effected behavior, the aim of this study was to evaluate its implementation.

METHODS

This implementation evaluation was conducted within a multi-center randomized controlled trial according to the Process Evaluation Framework by the Medical Research Council including the analysis of reach, dose, fidelity and adaptation. Data were collected from the implementers and the participants randomized to the intervention group of the trial.

RESULTS

The study sample comprised 95 physically inactive in-patients (mean age: 42 years, 53% women) with diagnosed major depressive disorder. The intervention reached the intended population (N = 95 in-patients enrolled in the study). The intervention dose varied between early dropouts (counseling sessions, M = 1.67) and study completers with some participants receiving a low dose (counseling sessions, M = 10.05) and high dose (counseling sessions, M = 25.37). Differences in the attendance groups were recognizable in the first two counseling sessions (duration of counseling session about 45 min in early dropouts versus 60 min for study completers). Fidelity of the in-person counseling content was partly achieved and adapted, whereas that of the remote counseling content was well achieved. Participants (86% at follow up) reported satisfaction with the implementers of the intervention. Adaptations were made to content, delivery mode and dose.

CONCLUSION

The PACINPAT trial was implemented in the intended population, in varying doses and with adaptations made to in-person counseling content and remote counseling dose. These findings are key to understanding outcome analyses within the PACINPAT trial, further developing interventions and contributing to implementation research among in-patients with depressive disorders.

TRIAL REGISTRATION

ISRCTN, ISRCTN10469580 , registered on 3^rd September 2018.

A systematic review and narrative synthesis of the experiences with caring for older people living with dementia in Sub-Saharan Africa

BACKGROUND

In low- and middle-income countries, including sub-Saharan Africa little is known about the experiences with caring for people living with dementia. The purpose of this systematic review and narrative synthesis was to examine the experiences with caring for older people living with dementia at home.

RESULTS

In total, 366 abstracts were identified and following screening, 19 studies were included in the synthesis. Six themes were identified: conceptualising dementia, caregiving arrangements, the impact of caregiving, caregiver identity and role, managing caregiving, unmet caregiver needs.

CONCLUSION

There is a dearth of research in relation to caregiving for older people living with dementia in sub-Saharan Africa. There is need for better information campaigns and support programs directed at family and professional caregivers in this context.

Process evaluation of complex interventions in non-communicable and neglected tropical diseases in low- and middle-income countries: a scoping review

OBJECTIVES

The aim of this review is to map out the use of process evaluation (PE) in complex interventions that address non-communicable diseases (NCDs) and neglected tropical diseases (NTDs) to identify gaps in the design and conduct, as well as strengths, limitations and implications, of this type of research in low- and middle-income countries (LMICs).

DESIGN

Scoping review of PE studies of complex interventions implemented in LMICs. Six databases were searched focused on studies published since 2008.

DATA SOURCES

Embase, PubMed, EbscoHost, Web of Science (WOS), Virtual Health Library (VHL) Regional Portal and Global Index Medicus: Regional Indexes AIM (AFRO), LILACS (AMRO/PAHO), IMEMR (EMRO), IMSEAR (SEARO), WPRIM (WPRO) Global Index Regional Indexes, MEDLINE, SciELO.

ELIGIBILITY CRITERIA

Studies conducted in LMICs on PEs of randomised controlled trials (RCTs) and non-RCTs published between January 2008 and January 2020. Other criteria were studies of interventions for people at risk or having physical and mental NCDs, and/or NTDs, and/or their healthcare providers and/or others related to achieve better health for these two disease groups. Studies were excluded if they were not reported in English or Spanish or Portuguese or French, not peer-reviewed articles, not empirical research and not human research.

DATA EXTRACTION AND SYNTHESIS

Data extracted to be evaluated were: available evidence in the utilisation of PE in the areas of NCDs and NTDs, including frameworks and theories used; methods applied to conduct PEs; and in a subsample, the barriers and facilitators to implement complex interventions identified through the PE. Variables were extracted and categorised. The information was synthesised through quantitative analysis by reporting frequencies and percentages. Qualitative analysis was also performed to understand facilitators and barriers presented in these studies. The implications for PEs, and how the information from the PE was used by researchers or other stakeholders were also assessed in this approach.

RESULTS

303 studies were identified, 79% were for NCDs, 12% used the label 'PE', 27% described a theory or framework for the PE, and 42% used mixed methods to analyse their findings. Acceptability, barriers and facilitators to implement the interventions, experiences and perceptions, and feasibility were the outcomes most frequently evaluated as part of the PEs. Barriers and facilitators themes identified were contextual factors, health system factors, human resources, attitudes and policy factors.

CONCLUSIONS

PEs in NCDs and NTDs are used in LMICs with a wide variety of methods. This review identified many PEs that were not labelled by the authors as such, as well as a limited application of PE-related theories and frameworks, and heterogeneous reporting of this type of study.

Exploring healthcare providers’ perceptions regarding the prevention and treatment of chronic pain in breast cancer survivors: A qualitative analysis among different disciplines

Background

The prevention and treatment of chronic pain problems in breast cancer follow-up care require an adequate response from healthcare providers. Generally, this involves the uptake of evidence-based principles regarding pain management in everyday practice. However, despite the extensive literature on effective pain interventions, systematic and coordinated follow-up care is lacking for breast cancer survivors with pain problems in Flanders, Belgium.

Objective

This study aimed to gather insight into healthcare providers’ perceptions of pain prevention and treatment in breast cancer follow-up care, particularly with attention to the multilevel influences on pain follow-up.

Methods

We conducted four online focus groups with twenty-two healthcare providers from different disciplines such as oncologists, pharmacists, nurses, physiotherapists, and psychologists. Data analysis was guided by the Qualitative Analysis Guide of Leuven. This guide is inspired by the constant comparison method, based on Grounded Theory.

Results

The identified influencing factors were thematically grouped into four levels: at the level of the individual healthcare provider, in interaction with the patient, in interaction with colleagues, and at the context level. At each level, we distinguished factors related to healthcare providers’ perceptions such as awareness, knowledge, attitudes, beliefs, experiences, and intentions. For example, because of a lack of knowledge and certain beliefs among healthcare providers, referral to other disciplines often does not happen in the context of pain.

Conclusion

This study points out the need to explore the prevention and treatment of chronic pain after breast cancer from a multidimensional point of view. This involves not only the characteristics of individual healthcare providers but is also inherently interactional and system-like in nature. This analysis provides opportunities for the development of interventions that target the influencing factors of prevention and treatment of chronic pain in breast cancer survivors.

Implementation of a Health Promotion Practice Using Individually Targeted Lifestyle Interventions in Primary Health Care: Protocol for the "Act in Time" Mixed Methods Process Evaluation Study

Background

There is growing evidence that noncommunicable diseases (NCDs) can be attributable to unhealthy lifestyle habits. However, there has been little application of this knowledge in primary health care (PHC).

Objective

This study aims to evaluate the process and outcomes of a multifaceted implementation strategy for a healthy lifestyle-promoting practice in a PHC setting. This practice is based on national guidelines targeting unhealthy lifestyle habits with a potential risk for NCDs.

Methods

A pre-post implementation study design with a control group is used in a PHC setting in central Sweden. The Medical Research Council guidelines for process evaluation of complex interventions will be applied. The implementation process and outcomes will be assessed using a mix of qualitative and quantitative methods. A strategic sample of up to 6 PHC centers will be included as intervention centers, which will receive a 12-month multifaceted implementation strategy. Up to 6 matched PHC centers will serve as controls. Core components in the implementation strategy are external and internal facilitators in line with the integrated-Promoting Action on Research Implementation in Health Services (i-PARIHS) framework and the Astrakan change leadership model. Data will be collected at baseline, during the implementation phase, and 4-6 months after the implementation strategy. Questionnaires will be sent to roughly 500 patients in every PHC center and 200 health care professionals (HCPs) before and after implementation. In addition, purposeful sampling will be used for interviews and focus group discussions with managers, HCPs, patient representatives, and internal and external facilitators. Use of data from medical records and activity logs will be an additional data source.

Results

Recruitment of PHC centers began in March 2021 and ended in Spring 2022. Based on the planned timeline with the 12-month implementation strategy and 4-6-month follow-up, we expect to collect the final data in Summer 2023.

Conclusions

This study will explain implementation process and outcomes using a multifaceted implementation strategy for a healthy lifestyle-promoting practice in a real-world PHC context. The study is expected to provide new knowledge about the role of facilitators and their contribution to implementation outcomes. These findings can guide policy makers, managers, and PHC staff to integrate health promotion and disease prevention in PHC and provide methodological support to facilitators.

Trial Registration

ClinicalTrials.gov NCT04799860; https://clinicaltrials.gov/ct2/show/NCT04799860

International Registered Report Identifier (IRRID)

DERR1-10.2196/37634

Use of an Electronic Medication Management Support System in Patients with Polypharmacy in General Practice: A Quantitative Process Evaluation of the AdAM Trial

Polypharmacy is associated with a risk of negative health outcomes. Potentially inappropriate medications, interactions resulting from contradicting medical guidelines, and inappropriate monitoring, all increase the risk. This process evaluation (PE) of the AdAM study investigates implementation and use of a computerized decision-support system (CDSS). The CDSS analyzes medication appropriateness by including claims data, and hence provides general practitioners (GPs) with full access to patients’ medical treatments. We based our PE on pseudonymized logbook entries into the CDSS and used the four dimensions of the Medical Research Council PE framework. Reach, which examines the extent to which the intended study population was included, and Dose, Fidelity, and Tailoring, which examine how the software was actually used by GPs. The PE was explorative and descriptive. Study participants were representative of the target population, except for patients receiving a high level of nursing care, as they were treated less frequently. GPs identified and corrected inappropriate prescriptions flagged by the CDSS. The frequency and intensity of interventions documented in the form of logbook entries lagged behind expectations, raising questions about implementation barriers to the intervention and the limitations of the PE. Impossibility to connect the CDSS to GPs’ electronic medical records (EMR) of GPs due to technical conditions in the German healthcare system may have hindered the implementation of the intervention. Data logged in the CDSS may underestimate medication changes in patients, as documentation was voluntary and already included in EMR.

Process Evaluation of an Implementation Trial: Design, Rationale, and Early Lessons Learnt From an International Cluster Clinical Trial in Intracerebral Hemorrhage

Background

The third INTEnsive care bundle with blood pressure Reduction in Acute Cerebral Hemorrhage Trial (INTERACT3) is an ongoing, international, multicenter, stepped-wedge cluster, prospective, randomized, open, blinded endpoint assessed trial evaluating the effectiveness of a quality improvement “care bundle” for the management of patients with acute spontaneous intracerebral hemorrhage (ICH) in low- and middle-income countries (LMICs). An embedded process evaluation aims to explore the uptake and implementation of the intervention, and understand the context and stakeholder perspectives, for interpreting the trial outcomes.

Methodology

The design was informed by Normalization Process Theory and the UK Medical Research Council process evaluation guidance. Mixed methods are used to evaluate the implementation outcomes of fidelity, reach, dose, acceptability, appropriateness, adoption, sustainability, and relevant contextual factors and mechanisms affecting delivery of the care bundle. Semi-structured interviews and non-participant observations are conducted with the primary implementers (physicians and nurses) and patients/carers to explore how the care bundle was integrated into routine care. Focus group discussions are conducted with investigators and project operational staff to understand challenges and possible solutions in the organization of the trial. Data from observational records, surveys, routine monitoring data, field notes and case report forms, inform contextual factors, and adoption of the intervention. Purposive sampling of sites according to pre-specified criteria is used to achieve sample representativeness.

Discussion

Implementation outcomes, and relevant barriers and facilitators to integrating the care bundle into routine practice, will be reported after completion of the process evaluation. The embedded process evaluation will aid understanding of the causal mechanisms between care bundle elements and clinical outcomes within complex health systems across diverse LMIC settings.

Trial Registration

The INTERACT3 study is registered at ClinicalTrials.gov (NCT03209258).

Process evaluations of mental health and psychosocial support interventions for populations affected by humanitarian crises

BACKGROUND

Randomised controlled trials (RCTs) have been increasingly used to test the effectiveness of mental health and psychosocial support(MHPSS) interventions for populations affected by humanitarian crises. Process evaluations are often integrated within RCTs of psychological interventions to investigate the implementation of the intervention, the impact of context, and possible mechanisms of action. We aimed to explore limitations and strengths of how process evaluations are currently conceptualised and implemented within MHPSS RCTs specifically.

METHODS

In April-June 2021 we conducted semi-structured interviews with 24 researchers involved in RCTs of MHPSS interventions in 23 different countries. Participants were selected based on systematic reviews of MHPSS interventions, funders' databases, and personal networks. Data were analysed using codebook thematic analysis.

RESULTS

The conduct of process evaluations was characterized by high heterogeneity in perceived function, implementation outcomes assessed, and methods used. While process evaluations were overwhelmingly considered as an important component of an RCT, there were different opinions on their perceived quality. This could be explained by the varying prioritization of effectiveness data over implementation data, confusion around the nature of process evaluations, and challenges in the collection and analysis of process data in humanitarian settings. Various practical recommendations were made by participants to improve future process evaluations in relation to: (i) study design (e.g., embedding process evaluations in study protocol and overall study objectives); (ii) methods (e.g., use of mixed methods); and (iii) increased financial and human resources dedicated to process evaluations.

CONCLUSION

The current state of process evaluations in MHPSS RCTs is heterogeneous. The quality of process evaluations should be improved to strengthen implementation science of the growing number of evidence-informed MHPSS interventions.

Risk factors associated with quadbike crashes: a systematic review

Background

Quadbikes or all-terrain vehicles are known for their propensity for crashes resulting in injury, disability, and death. The control of these needless losses resulting from quadbike crashes has become an essential contributor to sustainable development goals. Understanding the risk factors for such injuries is essential for developing preventive policies and strategies. The aim of this review was to identify the risk factors associated with quadbike crashes at multiple levels through a systematic review of a wide range of study designs.

Methods

The study incorporated a mixed-method systematic review approach and followed the PRISMA 2020 guidelines for reporting systematic reviews, including a peer reviewed  protocol. This systematic review included observational studies investigating the risk factors associated with quadbike crashes, injuries, or deaths. Seven electronic databases were searched from inception to October 2021. Studies were screened and extracted by three researchers. Quality appraisal was conducted using the Mixed Methods Appraisal Tool (MMAT). Due to extensive heterogeneity, meta-analysis was not conducted. All the risk factors have been presented in a narrative synthesis for discussion following the guidelines for Synthesis without Meta-analysis (SWiM).

Results

Thirty-nine studies combining an aggregate of 65,170 participants were included in this systematic review. The results indicate that modifiable risk factors, such as the increasing age of driving initiation, reducing substance use, and the use of organized riding parks, could reduce quadbike injuries. Riding practices such as avoiding passengers, avoiding nighttime riding, and using helmets could significantly reduce crashes and injuries among riders. Vehicle modifications such as increasing the wheelbase and limiting engine displacement could also help reduce crash incidence. Traditional interventional methods, such as legislation and training, had a weak influence on reducing quadbike injuries.

Conclusion

Multiple risk factors are associated with quadbike injuries, with most of them modifiable. Strengthening policies and awareness to minimize risk factors would help in reducing accidents associated with quadbikes.

PROSPERO registration number CRD42020170245

Evaluation of Electronic Service-Learning (e-Service-Learning) Projects in Mainland China under COVID-19

The use of electronic service-learning (e-Service-Learning or e-SL) is valuable under COVID-19 because we can provide the service without physical contact. Unfortunately, evaluation of e-SL is not widespread and there is no known study in different Chinese societies. Besides, there are many methodological limitations of the existing studies in the field. In this paper, we evaluated e-SL projects implemented in summer 2020 and 2021 in three sites in China. First, we examined service providers' changes based on pretest and posttest scores (i.e., objective outcome evaluation) and their perceptions of the e-SL projects (i.e., subjective outcome evaluation based on the service providers). Second, graduate student assessors in Chinese mainland universities and teachers of primary school students (i.e., service recipients) rated the SL program quality, service providers' performance and benefits to the service recipients after program completion (i.e., subjective outcome evaluation of SL projects based on other stakeholders). Third, trained graduate student assessors evaluated service quality during the implementation process (i.e., process evaluation). We found that university students (i.e., service providers) showed higher posttest scores in positive youth development attributes, leadership attributes and life satisfaction relative to pretest scores. Besides, service providers showed positive perceptions of their learning experience, own performance, benefits to the service recipients and themselves in the SL projects. Similarly, other stakeholders also had positive evaluation of the SL projects and related benefits. Finally, trained graduate student assessors had positive assessment of the quality of program implementation. The findings underscore the utility of e-SL involving both online teaching and learning as well as online service, particularly in a Chinese context.

Comparing process evaluations of motivational interviewing interventions for managing health conditions and health promotions: A scoping review

OBJECTIVE

To explore how process evaluations were conducted alongside randomised controlled trials (RCTs) involving motivational interviewing (MI) as an intervention to manage health conditions.

METHODS

A scoping review was conducted. We searched 7 databases (to May 2021) for studies that incorporated at least one aspect of process evaluation of RCTs using MI to manage a health condition. Two reviewers screened the studies for eligibility and extracted data according to Medical Research Council framework.

RESULTS

Of the 123 studies included, 85% lacked a theoretical framework for process evaluation. Most studies reported fidelity, but dose was underreported. Sixty-five studies reported mechanism of impact, but only twelve used participant experiences to understand how MI works. Only thirty used true mediation analysis. Context (n = 33) was the least reported aspect of process evaluation.

CONCLUSION

Process evaluations of MI to manage health conditions often consist of fragmented reports of implementation, mechanisms, and context. Using validated measures of fidelity, reporting dose, and using mediation analysis alongside qualitative exploration of participant and stakeholder insights will improve our understanding of how MI works.

PRACTICE IMPLICATIONS

Robust and comprehensive process evaluations will inform MI researchers to design more rigorous trials and for clinicians to implement more effective interventions for their clients.

A systematic review of de-escalation strategies for redeployed staff and repurposed facilities in COVID-19 intensive care units (ICUs) during the pandemic

BACKGROUND

Intensive care units (ICUs) experienced a surge in patient cases during the COVID-19 pandemic. Demand was managed by redeploying healthcare workers (HCWs) and restructuring facilities. The rate of ICU admissions has subsided in many regions, with the redeployed workforce and facilities returning to usual functions. Previous literature has focused on the escalation of ICUs, limited research exists on de-escalation. This study aimed to identify the supportive and operational strategies used for the flexible de-escalation of ICUs in the context of COVID-19.

METHODS

The systematic review was developed by searching eight databases in April and November 2021. Papers discussing the return of redeployed staff and facilities and the training, wellbeing, and operational strategies were included. Excluded papers were non-English and unrelated to ICU de-escalation. Quality was assessed using the mixed methods appraisal tool (MMAT) and authority, accuracy, coverage, objectivity, date, and significance (AACODS) checklist, findings were developed using narrative synthesis and thematic analysis.

FINDINGS

Fifteen papers were included from six countries covering wellbeing and training themes encompassing; time off, psychological follow-up, gratitude, identification of training needs, missed training catch-up, and continuation of ICU and disaster management training. Operational themes included management of rotas, retainment of staff, division of ICU facilities, leadership changes, traffic light systems, and preparation for re-expansion.

INTERPRETATION

The review provided an overview of the landscape of de-escalation strategies that have taken place in six countries. Limited empirical evidence was available that evaluated the effectiveness of such strategies. Empirical and evaluative research from a larger array of countries is needed to be able to make global recommendations on ICU de-escalation practices.

Features of Emergency Medical System calls that facilitate or inhibit Emergency Medical Dispatcher recognition that a patient is in, or at imminent risk of, cardiac arrest: A systematic mixed studies review

Aim

To identify and appraise evidence relating to the features of an Emergency Medicine System call interaction that enable, or inhibit, an Emergency Medical Dispatcher’s recognition that a patient is in out-of-hospital cardiac arrest, or at imminent risk of out-of-hospital cardiac arrest.

Methods

All study designs were eligible for inclusion. Data sources included Medline, BNI, CINAHL, EMBASE, PubMed, Cochrane Database of Systematic Reviews, AMED and OpenGrey. Stakeholder resources were screened and experts in resuscitation were asked to review the studies identified. Studies were appraised using the Mixed Methods Appraisal Tool. Synthesis was completed using a segregated mixed research synthesis approach.

Results

Thirty-two studies were included in the review. Three main themes were identified: Key features of the Emergency Medical Service call interaction; Managing the Emergency Medical Service call; Emotional distress.

Conclusion

A dominant finding is the difficulty in recognising abnormal/agonal breathing during the Emergency Medical Service call. The interaction between the caller and the Emergency Medical Dispatcher is critical in the recognition of patients who suffer an out-of-hospital cardiac arrest. Emergency Medical Dispatchers adapt their approach to the Emergency Medical Service call, and regular training for Emergency Medical Dispatchers is recommended to optimise out-of-hospital cardiac arrest recognition. Further research is required with a focus on the Emergency Medical Service call interaction of patients who are alive at the time of the Emergency Medical Service call and who later deteriorate into OHCA.

PROSPERO registration: CRD42019155458.

Process evaluation of a cluster randomised controlled trial to improve bronchiolitis management - a PREDICT mixed-methods study

BACKGROUND

Bronchiolitis is the most common reason for hospitalisation in infants. All international bronchiolitis guidelines recommend supportive care, yet considerable variation in practice continues with infants receiving non-evidence based therapies. We developed six targeted, theory-informed interventions; clinical leads, stakeholder meeting, train-the-trainer, education delivery, other educational materials, and audit and feedback. A cluster randomised controlled trial (cRCT) found the interventions to be effective in reducing use of five non-evidence based therapies in infants with bronchiolitis. This process evaluation paper aims to determine whether the interventions were implemented as planned (fidelity), explore end-users' perceptions of the interventions and evaluate cRCT outcome data with intervention fidelity data.

METHODS

A pre-specified mixed-methods process evaluation was conducted alongside the cRCT, guided by frameworks for process evaluation of cRCTs and complex interventions. Quantitative data on the fidelity, dose and reach of interventions were collected from the 13 intervention hospitals during the study and analysed using descriptive statistics. Qualitative data identifying perception and acceptability of interventions were collected from 42 intervention hospital clinical leads on study completion and analysed using thematic analysis.

RESULTS

The cRCT found targeted, theory-informed interventions improved bronchiolitis management by 14.1%. The process evaluation data found variability in how the intervention was delivered at the cluster and individual level. Total fidelity scores ranged from 55 to 98% across intervention hospitals (mean = 78%; SD = 13%). Fidelity scores were highest for use of clinical leads (mean = 98%; SD = 7%), and lowest for use of other educational materials (mean = 65%; SD = 19%) and audit and feedback (mean = 65%; SD = 20%). Clinical leads reflected positively about the interventions, with time constraints being the greatest barrier to their use.

CONCLUSION

Our targeted, theory-informed interventions were delivered with moderate fidelity, and were well received by clinical leads. Despite clinical leads experiencing challenges of time constraints, the level of fidelity had a positive effect on successfully de-implementing non-evidence-based care in infants with bronchiolitis. These findings will inform widespread rollout of our bronchiolitis interventions, and guide future practice change in acute care settings.

TRIAL REGISTRATION

Australian and New Zealand Clinical Trials Registry: ACTRN12616001567415 .

Process evaluation of complex interventions in chronic and neglected tropical diseases in low- and middle-income countries-a scoping review protocol

BACKGROUND

The use of process evaluations is a growing area of interest in research groups working on complex interventions. This methodology tries to understand how the intervention was implemented to inform policy and practice. A recent systematic review by Liu et al. on process evaluations of complex interventions addressing non-communicable diseases found few studies in low- and middle- income countries (LMIC) because it was restricted to randomized controlled trials, primary healthcare level and non-communicable diseases. Yet, LMICs face different barriers to implement interventions in comparison to high-income countries such as limited human resources, access to health care and skills of health workers to treat chronic conditions especially at primary health care level. Therefore, understanding the challenges of interventions for non-communicable diseases and neglected tropical diseases (diseases that affect poor populations and have chronic sequelae) will be important to improve how process evaluation is designed, conducted and used in research projects in LMICs. For these reasons, in comparison to the study of Liu et al., the current study will expand the search strategy to include different study designs, languages and settings.

OBJECTIVE

Map research using process evaluation in the areas of non-communicable diseases and neglected tropical diseases to inform the gaps in the design and conduct of this type of research in LMICs.

METHODS

Scoping review of process evaluation studies of randomized controlled trials (RCTs) and non-RCTs of complex interventions implemented in LMICs including participants with non-communicable diseases or neglected tropical diseases and their health care providers (physicians, nurses, technicians and others) related to achieve better health for all through reforms in universal coverage, public policy, service delivery and leadership. The aspects that will be evaluated are as follows: (i) available evidence of process evaluation in the areas of non-communicable diseases and neglected tropical diseases such as frameworks and theories, (ii) methods applied to conduct process evaluations and (iii) gaps between the design of the intervention and its implementation that were identified through the process evaluation. Studies published from January 2008. Exclusion criteria are as follows: not peer reviewed articles, not a report based on empirical research, not reported in English or Spanish or Portuguese or French, reviews and non-human research.

DISCUSSION

This scoping review will map the evidence of process evaluations conducted in LMICs. It will also identify the methods they used to collect and interpret data, how different theories and frameworks were used and lessons from the implementation of complex interventions. This information will allow researchers to conduct better process evaluations considering special characteristics from countries with limited human resources, scarce data available and limited access to health care.

Acceptability of Computerized Cognitive Behavioral Therapy for Adults: Umbrella Review

BACKGROUND

Mental ill-health presents a major public health problem. A potential part solution that is receiving increasing attention is computer-delivered psychological therapy, particularly during the COVID-19 pandemic as health care systems moved to remote service delivery. However, computerized cognitive behavioral therapy (cCBT) requires active engagement by service users, and low adherence may minimize treatment effectiveness. Therefore, it is important to investigate the acceptability of cCBT to understand implementation issues and maximize potential benefits.

OBJECTIVE

This study aimed to produce a critical appraisal of published reviews about the acceptability of cCBT for adults.

METHODS

An umbrella review informed by the Joanna Briggs Institute (JBI) methodology identified systematic reviews about the acceptability of cCBT for common adult mental disorders. Acceptability was operationalized in terms of uptake of, dropping out from, or completion of cCBT treatment; factors that facilitated or impeded adherence; and reports about user, carer, and health care professional experience and satisfaction with cCBT. Databases were searched using search terms informed by relevant published research. Review selection and quality appraisal were guided by the JBI methodology and the AMSTAR tool and undertaken independently by 2 reviewers.

RESULTS

The systematic searches of databases identified 234 titles, and 9 reviews (covering 151 unique studies) met the criteria. Most studies were comprised of service users with depression, anxiety, or specifically, panic disorder or phobia. Operationalization of acceptability varied across reviews, thereby making it difficult to synthesize results. There was a similar number of guided and unguided cCBT programs; 34% of guided and 36% of unguided users dropped out; and guidance included email, telephone, face-to-face, and discussion forum support. Guided cCBT was completed in full by 8%-74% of the participants, while 94% completed one module and 67%-84% completed some modules. Unguided cCBT was completed in full by 16%-66% of participants, while 95% completed one module and 54%-93% completed some modules. Guided cCBT appeared to be associated with adherence (sustained via telephone). A preference for face-to-face CBT compared to cCBT (particularly for users who reported feeling isolated), internet or computerized delivery problems, negative perceptions about cCBT, low motivation, too busy or not having enough time, and personal circumstances were stated as reasons for dropping out. Yet, some users favored the anonymous nature of cCBT, and the capacity to undertake cCBT in one's own time was deemed beneficial but also led to avoidance of cCBT. There was inconclusive evidence for an association between sociodemographic variables, mental health status, and cCBT adherence or dropping out. Users tended to be satisfied with cCBT, reported improvements in mental health, and recommended cCBT. Overall, the results indicated that service users' preferences were important considerations regarding the use of cCBT.

CONCLUSIONS

The review indicated that "one size did not fit all" regarding the acceptability of cCBT and that individual tailoring of cCBT is required in order to increase population reach, uptake, and adherence and therefore, deliver treatment benefits and improve mental health.

How, why and under what circumstances does a quality improvement collaborative build knowledge and skills in clinicians working with people with dementia? A realist informed process evaluation

In increasingly constrained health and aged care services, strategies are needed to improve quality and translate evidence into practice. In dementia care, recent failures in quality and safety have led the WHO to prioritise the translation of known evidence into practice. While quality improvement collaboratives have been widely used in healthcare, there are few examples in dementia care.

We describe a recent quality improvement collaborative to improve dementia care across Australia and assess the implementation outcomes of acceptability and feasibility of this strategy to translate known evidence into practice. A realist-informed process evaluation was used to analyse how, why and under what circumstances a quality improvement collaborative built knowledge and skills in clinicians working in dementia care.

This realist-informed process evaluation developed, tested and refined the programme theory of a quality improvement collaborative. Data were collected pre-intervention and post-intervention using surveys and interviews with participants (n=28). A combined inductive and deductive data analysis process integrated three frameworks to examine the context and mechanisms of knowledge and skill building in participant clinicians.

A refined program theory showed how and why clinicians built knowledge and skills in quality improvement in dementia care. Six mechanisms were identified: motivation, accountability, identity, collective learning, credibility and reflective practice. These mechanisms, in combination, operated to overcome constraints, role boundaries and pessimism about improved practice in dementia care.

A quality improvement collaborative designed for clinicians in different contexts and roles was acceptable and feasible in building knowledge, skills and confidence of clinicians to improve dementia care. Supportive reflective practice and a credible, flexible and collaborative process optimised quality improvement knowledge and skills in clinicians working with people with dementia.

Trial registration number

ACTRN12618000268246.

Process evaluation of the implementation of a decision support system to prevent and treat disease-related malnutrition in a hospital setting

Background

Malnutrition is present in 30% of hospitalized patients and has adverse outcomes for the patient and the healthcare system. The current practice for nutritional care is associated with many barriers. The MyFood decision support system was developed to prevent and treat malnutrition.

Methods

This paper reports on a process evaluation that was completed within an effectiveness trial. MyFood is a digital tool with an interface consisting of an app and a website. MyFood includes functions to record and evaluate dietary intake. It also provides reports to nurses, including tailored recommendations for nutritional treatment. We used an effectiveness-implementation hybrid design in a randomized controlled trial. The RE-AIM (Reach, Efficiency, Adoption, Implementation, Maintenance) framework was used to perform a process evaluation alongside the randomized controlled trial, using a combination of quantitative and qualitative methods. An implementation plan, including implementation strategies, was developed to plan and guide the study.

Results

Reach: In total, 88% of eligible patients consented to participate (n = 100). Adoption: Approximately 75% of the nurses signed up to use MyFood and 50% used the reports. Implementation: MyFood empowered the patients in their nutritional situation and acted as a motivation to eat to reach their nutritional target. The compliance of using MyFood was higher among the patients than the nurses. A barrier for use of MyFood among the nurses was different digital systems which were not integrated and the log-in procedure to the MyFood website. Despite limited use by some nurses, the majority of the nurses claimed that MyFood was useful, better than the current practice, and should be implemented in the healthcare system.

Conclusions

This study used a process evaluation to interpret the results of a randomized controlled trial more in-depth. The patients were highly compliant, however, the compliance was lower among the nurses. MyFood empowered the patients in their nutritional situation, the usability was considered as high, and the experiences and attitudes towards MyFood were primarily positive. Focus on strategies to improve the nurses’ compliance may in the future improve the MyFood system’s potential.

Trial registration

The trial was registered in ClinicalTrials.gov 26/01/2018 (NCT03412695).

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06236-3.

How digital health solutions align with the roles and functions that support hospital to home transitions for older adults: a rapid review study protocol

INTRODUCTION

Older adults may experience challenges during the hospital to home transitions that could be mitigated by digital health solutions. However, to promote adoption in practice and realise benefits, there is a need to specify how digital health solutions contribute to hospital to home transitions, particularly pertinent in this era of social distancing. This rapid review will: (1) elucidate the various roles and functions that have been developed to support hospital to home transitions of care, (2) identify existing digital health solutions that support hospital to home transitions of care, (3) identify gaps and new opportunities where digital health solutions can support these roles and functions and (4) create recommendations that will inform the design and structure of future digital health interventions that support hospital to home transitions for older adults (eg, the pre-trial results of the Digital Bridge intervention; ClinicalTrials.gov Identifier: NCT04287192).

METHODS AND ANALYSIS

A two-phase rapid review will be conducted to meet identified aims. In phase 1, a selective literature review will be used to generate a conceptual map of the roles and functions of individuals that support hospital to home transitions for older adults. In phase 2, a search on MEDLINE, EMBASE and CINAHL will identify literature on digital health solutions that support hospital to home transitions. The ways in which digital health solutions can support the roles and functions that facilitate these transitions will then be mapped in the analysis and generation of findings.

ETHICS AND DISSEMINATION

This protocol is a review of the literature and does not involve human subjects, and therefore, does not require ethics approval. This review will permit the identification of gaps and new opportunities for digital processes and platforms that enable care transitions and can help inform the design and implementation of future digital health interventions. Review findings will be disseminated through publications and presentations to key stakeholders.

Evidence for implementation of interventions to promote mental health in the workplace: a systematic scoping review protocol

BACKGROUND

Mental health problems are common in the working population and represent a growing concern internationally, with potential impacts on workers, organisations, workplace health and compensation authorities, labour markets and social policies. Workplace interventions that create workplaces supportive of mental health, promote mental health awareness, destigmatise mental illness and support those with mental disorders are likely to improve health and economical outcomes for employees and organisations. Identifying factors associated with successful implementation of these interventions can improve intervention quality and evaluation, and facilitate the uptake and expansion. Therefore, we aim to review research reporting on the implementation of mental health promotion interventions delivered in workplace settings, in order to increase understanding of factors influencing successful delivery.

METHODS AND ANALYSIS

A scoping review will be conducted incorporating a stepwise methodology to identify relevant literature reviews, primary research and grey literature. This review is registered with Research Registry (reviewregistry897). One reviewer will conduct the search to identify English language studies in the following electronic databases from 2008 through to July 1, 2020: Scopus, PROSPERO, Health Technology Assessments, PubMed, Campbell Collaboration, Joanna Briggs Library, PsycINFO, Web of Science Core Collection, CINAHL and Institute of Occupational Safety and Health (IOSH). Reference searching, Google Scholar, Grey Matters, IOSH and expert contacts will be used to identify grey literature. Two reviewers will screen title and abstracts, aiming for 95% agreement, and then independently screen full texts for inclusion. Two reviewers will assess methodological quality of included studies using the Mixed Methods Appraisal Tool and extract and synthesize data in line with the RE-AIM framework, Nielson and Randall's model of organisational-level interventions and Moore's sustainability criteria, if the data allows. We will recruit and consult with international experts in the field to ensure engagement, reach and relevance of the main findings.

DISCUSSION

This will be the first systematic scoping review to identify and synthesise evidence of barriers and facilitators to implementing mental health promotion interventions in workplace settings. Our results will inform future evaluation studies and randomised controlled trials and highlight gaps in the evidence base.

SYSTEMATIC REVIEW REGISTRATION

Research Registry ( reviewregistry897 ).

What theories underpin performance-based financing? A scoping review

PURPOSE

The study aims to explore the theoretical bases justifying the use of performance-based financing (PBF) in the health sector in low- and middle-income countries (LMICs).

DESIGN/METHODOLOGY/APPROACH

The authors conducted a scoping review of the literature on PBF so as to identify the theories utilized to underpin it and analyzed its theoretical justifications.

FINDINGS

Sixty-four studies met the inclusion criteria. Economic theories were predominant, with the principal-agent theory being the most commonly-used theory, explicitly referred to by two-thirds of included studies. Psychological theories were also common, with a wide array of motivation theories. Other disciplines in the form of management or organizational science, political and social science and systems approaches also contributed. However, some of the theories referred to contradicted each other. Many of the studies included only casually alluded to one or more theories, and very few used these theories to justify or support PBF. No theory emerged as a dominant, consistent and credible justification of PBF, perhaps except for the principal-agent theory, which was often inappropriately applied in the included studies, and when it included additional assumptions reflecting the contexts of the health sector in LMICs, might actually warn against adopting PBF.

PRACTICAL IMPLICATIONS

Overall, this review has not been able to identify a comprehensive, credible, consistent, theoretical justification for using PBF rather than alternative approaches to health system reforms and healthcare providers' motivation in LMICs.

ORIGINALITY/VALUE

The theoretical justifications of PBF in the health sector in LMICs are under-documented. This review is the first of this kind and should encourage further debate and theoretical exploration of the justifications of PBF.

Prevalence, Barriers, and Interventions Related to Medication Adherence Among Patients With Major Depressive Disorder: A Scoping Review

The current scoping review aimed to identify the prevalence, contributing factors, methods of measurement, and interventions related to medication adherence among patients with major depressive disorder (MDD). A total of 37 articles met inclusion criteria. The prevalence of medication adherence among patients with MDD ranged from 10.6% to 85.4%. Approximately 67% of studies used self-report data collection. Illness-related factors (e.g., onset of illness, duration of illness, symptoms, illness severity), medication-related factors (e.g., adverse reactions, duration of treatment, cost of treatment), and patient-related factors (e.g., beliefs, attitudes, knowledge, self-stigma) were the most reported factors associated with medication adherence. In addition, multi-faceted interventions were recommended over single-element interventions to enhance medication adherence. There is a need to integrate appropriate and effective assessment measures of medication adherence that lead to better health care outcomes, lower risk factors, and improved interventions related to medication adherence. [Journal of Psychosocial Nursing and Mental Health Services, 59(1), 39-51.].

Examining fidelity in the INFORM trial: a complex team-based behavioral intervention

Background

Fidelity in complex behavioral interventions is underexplored. This study examines the fidelity of the INFORM trial and explores the relationship between fidelity, study arm, and the trial’s primary outcome—care aide involvement in formal team communications about resident care.

Methods

A concurrent process evaluation of implementation fidelity was conducted in 33 nursing homes in Western Canada (Alberta and British Columbia). Study participants were from 106 clinical care units clustered in 33 nursing homes randomized to the Basic and Enhanced-Assisted Feedback arms of the INFORM trial.

Results

Fidelity of the INFORM intervention was moderate to high, with fidelity delivery and receipt higher than fidelity enactment for both study arms. Higher enactment teams experienced a significantly larger improvement in formal team communications between baseline and follow-up than lower enactment teams (F(1, 70) = 4.27, p = .042).

Conclusions

Overall fidelity enactment was associated with improvements in formal team communications, but the study arm was not. This suggests that the intensity with which an intervention is offered and delivered may be less important than the intensity with which intervention participants enact the core components of an intervention. Greater attention to fidelity assessment and publication of fidelity results through studies such as this one is critical to improving the utility of published trials.

Analysis of the implementation of a community-based intervention to control dengue fever in Burkina Faso

BACKGROUND

A community-based dengue fever intervention was implemented in Burkina Faso in 2017. The results achieved vary from one area to another. The objective of this article is to analyze the implementation of this intervention, to better understand the process, and to explain the contextual elements of performance variations in implementation.

METHODOLOGY

The research was conducted in the former sector 22 of the city of Ouagadougou. We adapted the Consolidated Framework for Implementation Research (CFIR) to take into account the realities of the context and the intervention. The data collected from the participants directly involved in the implementation using three techniques: document consultation, individual interview, and focus group.

RESULTS

Two dimensions of CFIR emerge from the results as having had a positive influence on the implementation: (i) the characteristics of the intervention and (ii) the processes of the intervention implementation. The majority of the CFIR constructions were considered to have had a positive effect on implementation. The quality and strength of the evidence received the highest score. The dimension of the external context had a negative influence on the implementation of the intervention.

CONCLUSION

The objective of the study was to analyze the influence of contextual elements on the implementation process of a community-based dengue fever intervention. We used the CFIR framework already used by many studies for implementation analysis. Although it was not possible to test this framework in its entirety, it is useful for the analysis of the implementation. Its use is simple and does not require any special skills from users. Usability is indeed an essential criterion for the relevance of using an analytical framework in implementation science.

Process evaluation of complex cardiovascular interventions: How to interpret the results of my trial?

Complex interventions of varying degrees of complexity are commonly used and evaluated in cardiovascular nursing and allied professions. Such interventions are increasingly tested using randomized trial designs. However, process evaluations are seldom used to better understand the results of these trials. Process evaluation aims to understand how complex interventions create change by evaluating implementation, mechanisms of impact, and the surrounding context when delivering an intervention. As such, this method can illuminate important mechanisms and clarify variation in results. In this article, process evaluation is described according to the Medical Research Council guidance and its use exemplified through a randomized controlled trial evaluating the effectiveness of a transition program for adolescents with chronic conditions.