How do medical schools influence their students' career choices? A realist evaluation

INTRODUCTION

The career choices of medical graduates vary widely between medical schools in the UK and elsewhere and are generally not well matched with societal needs. Research has found that experiences in medical school including formal, informal and hidden curricula are important influences. We conducted a realist evaluation of how and why these various social conditions in medical school influence career thinking.

METHODS

We interviewed junior doctors at the point of applying for speciality training. We selected purposively for a range of career choices. Participants were asked to describe points during their medical training when they had considered career options and how their thinking had been influenced by their context. Interview transcripts were coded for context-mechanism-outcome (CMO) configurations to test initial theories of how career decisions are made.

RESULTS

A total of 26 junior doctors from 12 UK medical schools participated. We found 14 recurring CMO configurations in the data which explained influences on career choice occurring during medical school.

DISCUSSION

Our initial theories about career decision-making were refined as follows: It involves a process of testing for fit of potential careers. This process is asymmetric with multiple experiences needed before deciding a career fits ('easing in') but sometimes only a single negative experience needed for a choice to be ruled out. Developing a preference for a speciality aligns with Person-Environment-Fit decision theories. Ruling out a potential career can however be a less thought-through process than rationality-based decision theories would suggest. Testing for fit is facilitated by longer and more authentic undergraduate placements, allocation of and successful completion of tasks, being treated as part of the team and enthusiastic role models. Informal career guidance is more influential than formal. We suggest some implications for medical school programmes.

Sharing is Caring: A Realist Evaluation of a Social Support Group for Individuals Who Have Been Bereaved by Suicide

AIM

To understand the experiences and perceived impact on the wellbeing of individuals attending a suicide bereavement social support group.

DESIGN

A qualitative study guided by a realist evaluation framework.

METHODS

Data were collected from May-July 2020 using online semistructured individual interviews with participants (N = 6), from the North West of England recruited from a suicide bereavement support group's social media. Data were analysed using thematic analysis informed by the realist framework.

RESULTS

Effective social support includes the prioritisation of building meaningful connections with like-minded individuals, providing a safe space for authentic self-expression aiding personal relationship maintenance. Contextual factors included: Societal and cultural stigma of suicide, self-stigma and gender norms. Mechanisms influencing support seeking include: Not wanting to burden loved ones due to judgement, and a lack of understanding.

IMPACT

Policymakers can reduce demand on healthcare systems by developing tailored support groups to suit individual needs.

A realist evaluation of how, why and when objective structured clinical exams (OSCEs) are experienced as an authentic assessment of clinical preparedness

INTRODUCTION

Whilst rarely researched, the authenticity with which Objective Structured Clinical Exams (OSCEs) simulate practice is arguably critical to making valid judgements about candidates' preparedness to progress in their training. We studied how and why an OSCE gave rise to different experiences of authenticity for different participants under different circumstances.

METHODS

We used Realist evaluation, collecting data through interviews/focus groups from participants across four UK medical schools who participated in an OSCE which aimed to enhance authenticity.

RESULTS

Several features of OSCE stations (realistic, complex, complete cases, sufficient time, autonomy, props, guidelines, limited examiner interaction etc) combined to enable students to project into their future roles, judge and integrate information, consider their actions and act naturally. When this occurred, their performances felt like an authentic representation of their clinical practice. This didn't work all the time: focusing on unavoidable differences with practice, incongruous features, anxiety and preoccupation with examiners' expectations sometimes disrupted immersion, producing inauthenticity.

CONCLUSIONS

The perception of authenticity in OSCEs appears to originate from an interaction of station design with individual preferences and contextual expectations. Whilst tentatively suggesting ways to promote authenticity, more understanding is needed of candidates' interaction with simulation and scenario immersion in summative assessment.

The Use of Vignettes to Improve the Validity of Qualitative Interviews for Realist Evaluation

Although realist evaluation (RE) requires multiple data collection methods, qualitative interviews are considered most valuable and are most frequently used. The guiding principles of RE may limit the emergence of new Context-Mechanism-Outcome (CMO) configurations by evoking particular underlying mechanisms. This paper proposes a new method for conducting semi-structured interviews in the RE context by drawing on the literature and examining the ability of vignettes to explore perceptions about specific situations. Vignettes are developed based on researchers' knowledge of the setting and program theory and are updated through an iterative process throughout data collection. Interviews focus on situations illustrated in the vignette to capture variations in interviewees' perceptions. This method constrains interviewees to using retroduction to identify the hidden underlying mechanisms that link contextual elements to outcomes based on their experiences. This method allows researchers to focus on CMO configurations without evoking mechanisms, which contributes to the rigor of the method.

Four System Enablers of Large-System Transformation in Health Care: A Mixed Methods Realist Evaluation

Policy Points The implementation of large-scale health care interventions relies on a shared vision, commitment to change, coordination across sites, and a spanning of siloed knowledge. Enablers of the system should include building an authorizing environment; providing relevant, meaningful, transparent, and timely data; designating and distributing leadership and decision making; and fostering the emergence of a learning culture. Attention to these four enablers can set up a positive feedback loop to foster positive change that can protect against the loss of key staff, the presence of lone disruptors, and the enervating effects of uncertainty.

CONTEXT

Large-scale transformative initiatives have the potential to improve the quality, efficiency, and safety of health care. However, change is expensive, complex, and difficult to implement and sustain. This paper advances system enablers, which will help to guide large-scale transformation in health care systems.

METHODS

A realist study of the implementation of a value-based health care program between 2017 and 2021 was undertaken in every public hospital (n = 221) in New South Wales (NSW), Australia. Four data sources were used to elucidate initial program theories beginning with a set of literature reviews, a program document review, and informal discussions with key stakeholders. Semistructured interviews were then conducted with 56 stakeholders to confirm, refute, or refine the theories. A retroductive analysis produced a series of context-mechanism-outcome (CMO) statements. Next, the CMOs were validated with three health care quality expert panels (n = 51). Synthesized data were interrogated to distill the overarching system enablers.

FINDINGS

Forty-two CMO statements from the eight initial program theory areas were developed, refined, and validated. Four system enablers were identified: (1) build an authorizing environment; (2) provide relevant, authentic, timely, and meaningful data; (3) designate and distribute leadership and decision making; and (4) support the emergence of a learning culture. The system enablers provide a nuanced understanding of large-system transformation that illustrates when, for whom, and in what circumstances large-system transformation worked well or worked poorly.

CONCLUSIONS

System enablers offer nuanced guidance for the implementation of large-scale health care interventions. The four enablers may be portable to similar contexts and provide the empirical basis for an implementation model of large-system value-based health care initiatives. With concerted application, these findings can pave the way not just for a better understanding of greater or lesser success in intervening in health care settings but ultimately to contribute higher quality, higher value, and safer care.

Leadership practices that enable healthful cultures in clinical practice: A realist evaluation

AIM

To generate, test and refine programme theories that emerged from a rapid realist review investigating practising UK Nurses' and Midwives' experiences of effective leadership strategies during the COVID-19 pandemic.

BACKGROUND

The realist review of literature generated six tentative theories of healthful leadership practices reflecting, working with people's beliefs and values; being facilitative; multiple means of communication and; practical support. The review yielded little insight into the actual impact of the leadership approaches advocated.

METHODS

A realist study, informed by person-centredness using mixed-methods. Online survey (n = 328) and semi-structured interviews (n = 14) of nurses and midwives across the UK in different career positions/specialities. Quantitative data analysed using descriptive statistics and exploratory factor analysis. Framework analysis for qualitative data using context (C), mechanism (M), outcome (O) configurations of the tentative theories.

RESULTS

Three refined theories were identified concerning: Visibility and availability; embodying values and; knowing self. Healthful leadership practices are only achievable within organisational cultures that privilege well-being.

CONCLUSIONS

Leaders should intentionally adopt practices that promote well-being. 'Knowing self' as a leader, coaching and mentoring practice development is important for leadership development.

IMPLICATIONS FOR CLINICAL PRACTICE

Nurses who feel valued, heard, cared for and safe are more likely to remain in clinical practice. Job satisfaction and being motivated to practice with confidence and competence will impact positively on patient outcomes.

IMPACT

The study addresses the role of leadership in developing healthful workplace cultures. The main findings were six leadership practices that promote healthful cultures. The research will have an impact on strategic and clinical leaders, nurses and midwives.

REPORTING METHOD

This study used EQUATOR checklist, RAMASES II as reporting standards for realist evaluations.

PATIENT OR PUBLIC CONTRIBUTION

No patient or public contribution.

Explaining how and why social support groups in hospice day services benefit palliative care patients, for whom, and in what circumstances

Background

Palliative care aims to provide holistic support for people with life-limiting illness, responding to psychological, social and spiritual needs, as well as to clinical and physical. In the United Kingdom, hospice day services (including day care, group interventions, group activities, and social events for palliative care outpatients) aim to provide opportunities for patients to gain social support, which is thought to improve their quality of life.

Objectives

This research explored social support within hospice day services, to explain in detail how and why social support obtained within a hospice day service could be beneficial to palliative care patients.

Design

Qualitative research using observations of hospice day services and interviews with service providers.

Methods

Data collection involved nineteen interviews with hospice service providers (n = 19) and researcher observations of hospice day services. The findings detail how patient and hospice context interact to produce mechanisms that lead to outcomes beyond the hospice day service.

Results

Practical, clinical and social aspects of the hospice day service are important for patients feeling welcome and safe in the setting. The opportunity to connect with other people and work towards personal goals can boost self-confidence for patients who have lost access to meaningful activity. New friendships between patients encourages reciprocal support and feelings of belonging. It is beneficial to have permission to speak freely about topics deemed inappropriate elsewhere, because honest communication is helpful in accepting and adapting to their circumstances.

Conclusion

Hospice day services facilitate group settings for reciprocal social support. This research proposes an initial programme theory that can be further developed and tested. It explains how and why, in some contexts, social support increases personal and practical resources to cope with illness and death, leading to changes outside of the hospice (to mood, interpersonal interactions and behaviour) that could improve quality of life.

Finding meaning: a realist-informed perspective on social risk screening and relationships as mechanisms of change

Background

Social risk screening rates in many US primary care settings remain low. This realist-informed evaluation explored the mechanisms through which a tailored coaching and technical training intervention impacted social risk screening uptake in 26 community clinics across the United States.

Methods

Evaluation data sources included the documented content of interactions between the clinics and implementation support team and electronic health record (EHR) data. Following the realist approach, analysis was composed of iterative cycles of developing, testing and refining program theories about how the intervention did-or didn't-work, for whom, under what circumstances. Normalization Process Theory was applied to the realist program theories to enhance the explanatory power and transferability of the results.

Results

Analysis identified three overarching realist program theories. First, clinic staff perceptions about the role of standardized social risk screening in person-centered care-considered "good" care and highly valued-strongly impacted receptivity to the intervention. Second, the physicality of the intervention materials facilitated collaboration and impacted clinic leaders' perception of the legitimacy of the social risk screening implementation work. Third, positive relationships between the implementation support team members, between the support team and clinic champions, and between clinic champions and staff motivated and inspired clinic staff to engage with the intervention and to tailor workflows to their settings' needs. Study clinics did not always exhibit the social risk screening patterns anticipated by the program theories due to discrepant definitions of success between clinic staff (improved ability to provide contextualized, person-centered care) and the trial (increased rates of EHR-documented social risk screening). Aligning the realist program theories with Normalization Process Theory constructs clarified that the intervention as implemented emphasized preparation over operationalization and appraisal, providing insight into why the intervention did not successfully embed sustained systematic social risk screening in participating clinics.

Conclusion

The realist program theories highlighted the effectiveness and importance of intervention components and implementation strategies that support trusting relationships as mechanisms of change. This may be particularly important in social determinants of health work, which requires commitment and humility from health care providers and vulnerability on the part of patients.

Understanding Whether and How a Digital Health Intervention Improves Transition Care for Emerging Adults Living With Type 1 Diabetes: Protocol for a Mixed Methods Realist Evaluation

BACKGROUND

Emerging adults living with type 1 diabetes (T1D) face a series of challenges with self-management and decreased health system engagement, leading to an increased risk of acute complications and hospital admissions. Effective and scalable strategies are needed to support this population to transfer seamlessly from pediatric to adult care with sufficient self-management capability. While digital health interventions for T1D self-management are a promising strategy, it remains unclear which elements work, how, and for which groups of individuals.

OBJECTIVE

This study aims to evaluate the design and implementation of a multicomponent SMS text message-based digital health intervention to support emerging adults living with T1D in real-world settings. The objectives are to identify the intervention components and associated mechanisms that support user engagement and T1D health care transition experiences and determine the individual characteristics that influence the implementation process.

METHODS

We used a realist evaluation embedded alongside a randomized controlled trial, which uses a sequential mixed methods design to analyze data from multiple sources, including intervention usage data, patient-reported outcomes, and realist interviews. In step 1, we conducted a document analysis to develop a program theory that outlines the hypothesized relationships among "individual-level contextual factors, intervention components and features, mechanisms, and outcomes," with special attention paid to user engagement. Among them, intervention components and features depict 10 core characteristics such as transition support information, problem-solving information, and real-time interactivity. The proximal outcomes of interest include user engagement, self-efficacy, and negative emotions, whereas the distal outcomes of interest include transition readiness, self-blood glucose monitoring behaviors, and blood glucose. In step 2, we plan to conduct semistructured realist interviews with the randomized controlled trial's intervention-arm participants to test the hypothesized "context-intervention-mechanism-outcome" configurations. In step 3, we plan to triangulate all sources of data using a coincidence analysis to identify the necessary combinations of factors that determine whether and how the desired outcomes are achieved and use these insights to consolidate the program theory.

RESULTS

For step 1 analysis, we have developed the initial program theory and the corresponding data collection plan. For step 2 analysis, participant enrollment for the randomized controlled trial started in January 2023. Participant enrollment for this realist evaluation was anticipated to start in July 2023 and continue until we reached thematic saturation or achieved informational power.

CONCLUSIONS

Beyond contributing to knowledge on the multiple pathways that lead to successful engagement with a digital health intervention as well as target outcomes in T1D care transitions, embedding the realist evaluation alongside the trial may inform real-time intervention refinement to improve user engagement and transition experiences. The knowledge gained from this study may inform the design, implementation, and evaluation of future digital health interventions that aim to improve transition experiences.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/46115.

Longitudinal realist evaluation of the Dementia PersonAlised Care Team (D-PACT) intervention: protocol

BACKGROUND

Different dementia support roles exist but evidence is lacking on which aspects are best, for whom, and in what circumstances, and on their associated costs and benefits. Phase 1 of the Dementia PersonAlised Care Team programme (D-PACT) developed a post-diagnostic primary care-based intervention for people with dementia and their carers and assessed the feasibility of a trial.

AIM

Phase 2 of the programme aims to 1) refine the programme theory on how, when, and for whom the intervention works; and 2) evaluate its value and impact.

DESIGN & SETTING

A realist longitudinal mixed-methods evaluation will be conducted in urban, rural, and coastal areas across South West and North West England where low-income or ethnic minority populations (for example, South Asian) are represented. Design was informed by patient, public, and professional stakeholder input and phase 1 findings.

METHOD

High-volume qualitative and quantitative data will be collected longitudinally from people with dementia, carers, and practitioners. Analyses will comprise the following: 1) realist longitudinal case studies; 2) conversation analysis of recorded interactions; 3) statistical analyses of outcome and experience questionnaires; 4a) health economic analysis examining costs of delivery; and 4b) realist economic analysis of high-cost events and 'near misses'. All findings will be synthesised using a joint display table, evidence appraisal tool, triangulation, and stakeholder co-analysis.

CONCLUSION

The realist evaluation will describe how, why, and for whom the intervention does or does not lead to change over time. It will also demonstrate how a non-randomised design can be more appropriate for complex interventions with similar questions or populations.

Trusting parent-professional relationships in interprofessional interventions for expectant and new parents in vulnerable positions: A realist evaluation

Interprofessional collaboration and trusting parent-professional relationships can be key to delivering interprofessional care to meet the needs of expectant and new parents in vulnerable positions. This, however, presents challenges. This study aimed to gain deeper understanding of how and under what circumstances trusting parent-professional relationships develop and work within interprofessional team-based care for this group, from the professionals' perspectives. Realist evaluation was undertaken based on 14 semi-structured, realist interviews with midwives and health visitors and 11 observations. Multiple interrelated mechanisms were identified including patient/family-centered care, timely and relevant interprofessional involvement in care, gentle interprofessional bridging, transparency of intervention roles and purposes, and relational continuity. Good interprofessional collaboration was a primary condition for these mechanisms. Developed, trusting relationships supported parents' engagements with interprofessional care and constituted a supportive safety net that promoted parenting skills and coping abilities. We identified harmful mechanisms: distanced encounters, uncertainty of interprofessional involvement, and compromising the safe space. These mechanisms caused distrust and disengagement. Ensuring trusting parent-professional relationships within interprofessional team-based care demands each professional involved competently engages in relational work and interprofessional collaboration. Uncontrollability is thus influenced regarding interpersonal connection and potentially gives an explanation when trust-building efforts fail.

The importance of training transfer of non-technical skills safety training of construction workers

Safety training of migrant workers in construction has focused on technical skills with limited attention to non-technical skills, which support safety training transfer to the worksite, both immediately after training and in the long term. Using realist evaluation as our theoretical framework, this study explores the transfer of two key non-technical skills to construction sites: communication and decision-making. Trained workers completed questionnaires post-training and after six months. A moderated mediation model found an indirect link through training transfer between communication and decision-making skills immediately post-training and six months later. The results also revealed that high levels of safety self-efficacy moderated the relationship between communication, but not decision-making, safety skills post-training and the extent to which trained workers reported transferring these skills. The study has important practical implications, showing the significance of training transfer of non-technical skills, such as communication and decision-making, to the worksite.

A Realist Evaluation of Case Management Models for People with Complex Health Conditions Using Novel Methods and Tools-What Works, for Whom, and under What Circumstances?

Case management developed from a generalist model to a person-centred model aligned with the evidence-informed evolution of best practice people-centred integrated care. Case management is a multidimensional and collaborative integrated care strategy where the case manager performs a set of interventions/actions to support the person with a complex health condition to progress in their recovery pathway and participate in life roles. It is currently unknown what case management model works in real life for whom and under what circumstances. The purpose of this study was to answer these questions. The study methods used realistic evaluation framework, examined the patterns and associations between case manager actions (mechanisms), the person's characteristics and environment (context), and recovery (outcomes) over 10 years post severe injury. There was mixed methods secondary analysis of data extracted via in-depth retrospective file reviews (n = 107). We used international frameworks and a novel approach with multi-layered analysis including machine learning and expert guidance for pattern identification. The study results confirm that when provided, a person-centred case management model contributes to and enhances the person's recovery and progress towards participation in life roles and maintaining well-being after severe injury.Furthermore, the intensity of case management for people with traumatic brain injury, and the person-centred actions of advising, emotional and motivational support, and proactive coordination contribute to the person achieving their goals. The results provide learnings for case management services on the case management models, for quality appraisal, service planning, and informs further research on case management.

A realist evaluation of a support and training program for family caregivers in the municipality of Envigado: A research protocol

AIMS

In Envigado, Colombia, the Secretariat of Health has implemented, since 2011, an interprofessional program involving nurses to support and train relatives to improve the quality of life of people with a loss of autonomy and their family caregivers. The objectives of this study are to assess the outcomes of this program and to explore the contextual elements and mechanisms that can explain these outcomes.

DESIGN

This article presents the research protocol for a realist evaluation that will be carried out to gather the perspectives of various local stakeholders involved.

METHODS

Four outcomes on family caregivers will be measured quantitatively using self-administered questionnaires and numerical scales. Contextual elements and mechanisms will then be explored qualitatively through focus groups and individual interviews. An iterative analysis will enable the refinement of a program theory.

RESULTS

The results will inform a program theory that underlies the outcomes of the family caregiver support and training program.

PATIENT OR PUBLIC CONTRIBUTION

Community stakeholders, family caregivers, people with a loss of autonomy and their relatives will be involved in data collection and/or in the validation of the program theory.

School Gardening and Health and Well-Being of School-Aged Children: A Realist Synthesis

School environments can create healthy settings to foster children's health and well-being. School gardening is gaining popularity as an intervention for healthier eating and increased physical activity. We used a systematic realist approach to investigate how school gardens improve health and well-being outcomes for school-aged children, why, and in what circumstances. The context and mechanisms of the specific school gardening interventions (n = 24) leading to positive health and well-being outcomes for school-aged children were assessed. The impetus of many interventions was to increase fruit and vegetable intake and address the prevention of childhood obesity. Most interventions were conducted at primary schools with participating children in Grades 2 through 6. Types of positive outcomes included increased fruit and vegetable consumption, dietary fiber and vitamins A and C, improved body mass index, and improved well-being of children. Key mechanisms included embedding nutrition-based and garden-based education in the curriculum; experiential learning opportunities; family engagement and participation; authority figure engagement; cultural context; use of multi-prong approaches; and reinforcement of activities during implementation. This review shows that a combination of mechanisms works mutually through school gardening programs leading to improved health and well-being outcomes for school-aged children.

Collaboration for Developing and Sustaining Community Dementia-Friendly Initiatives: A Realist Evaluation

BACKGROUND

Dementia-friendly communities (DFCs) are seen as key to the inclusion and participation of people with dementia and carers. Dementia-friendly initiatives (DFIs) are important building blocks for the growth of DFCs. The collaboration between different stakeholders is a central aspect in developing and sustaining DFIs.

AIM

This study tests and refines an initial theory about collaborating for DFIs with special attention for the involvement of people with dementia and their carers during the collaboration for DFIs. The realist approach is used for deepening contextual aspects, mechanisms, outcomes, and its explanatory power.

METHODS

A participatory case study design using qualitative data (focus groups, observations, reflections, minutes from meetings, and exit interviews) was executed in four Dutch municipalities that have ambitions to become dementia- friendly communities.

RESULTS

The refined theory on the collaboration for DFIs incorporates contextual aspects such as diversity, shared insights, and clarity. It draws attention to the importance of mechanisms such as the recognition of efforts and progress, informal distributed leadership, interdependency, belonging, significance, and commitment. These mechanisms resonate with feeling useful and feeling collectively powerful in the collaboration. The outcomes of collaboration were activation, getting new ideas, and fun. Our findings address how stakeholders' routines and perspectives impact the involvement of people with dementia and their carers during collaboration.

CONCLUSION

This study provides detailed information about collaboration for DFIs. The collaboration for DFIs is largely influenced by feeling useful and collectively powerful. Further research is needed to understand how these mechanisms can be triggered with the involvement of people with dementia and their carers in the heart of the collaboration.

Corrigendum: Human trafficking: Results of a 5-year theory-based evaluation of interventions to prevent trafficking of women from South Asia

[This corrects the article DOI: 10.3389/fpubh.2021.645059.].

Understanding and Improving Older People's Well-Being through Social Prescribing Involving the Cultural Sector: Interviews from a Realist Evaluation

Social prescribing is a non-clinical approach to addressing social, environmental, and economic factors affecting how people feel physical and/or emotionally. It involves connecting people to "community assets" (e.g., local groups, organizations, and charities) that can contribute to positive well-being. We sought to explain in what ways, for whom, and why the cultural sector can support social prescribing with older people. We conducted semi-structured interviews with 28 older people (aged 60+) and 25 cultural sector staff. The following nine concepts, developed from interview data, progressed the understanding of tailoring cultural offers, which came from our previous realist review-immersion, buddying, café culture, capacity, emotional involvement, perseverance, autonomy, elitism, and virtual cultural offers. Through tailoring, we propose that older people might experience one or more of the following benefits from engaging with a cultural offer as part of social prescribing-being immersed, psychological holding, connecting, and transforming through self-growth.

The Impact of Ex-Post Legislative Evaluations in Healthcare: A Mixed Methods Realist Evaluation Study Protocol for Conducting Case Studies

Background

Recent studies on the impact of ex-post legislative evaluations show that there are different types of impact and different factors that can influence it. These include the context of a legislative evaluation, research quality, and interactions between researchers and other actors within the evaluation process. However, thorough empirical research in this area is lacking. This warrants empirical research into the factors that influence the impact of ex-post legislative evaluations, so these insights can be used to increase the likelihood of ex-post legislative evaluations having an impact.

Methods and analysis

In this protocol, we report on the realist evaluation methodology that will be used to evaluate the impact of three ex-post legislative evaluations in the Dutch healthcare sector. The mixed methods realist evaluation approach will facilitate this theory-driven, qualitative research. The study will consist of the following three steps: (1) Initial programme theory development, (2) theory validation, and (3) theory refinement. Knowledge from two scoping reviews conducted previously, and two subsequent expert meetings will form the basis for developing the initial programme theory. During this study, three case studies will be conducted, in which three individual ex-post legislative evaluations will be examined. Specificmethods for data collection will include: documentary review, observation, structured questionnaires and focus group discussions with purposefully identified key stakeholders. Using the framework approach, the data will be analysed thematically in a within-case analysis followed by a cross-case analysis.

Discussion

This protocol provides insight into how the study will be conducted. As this study uses multiple qualitative researchmethods to answer one question, this protocol supports refining data collection procedures. Careful consideration of the approach beforehand can minimise pitfalls, reduce publication bias and improve reproducibility. The protocol therefore specifies how the research question will be answered in detail, and this provides solid guidance for the research process.

General Practice Patients' Experiences and Perceptions of the WiserAD Structured Web-Based Support Tool for Antidepressant Deprescribing: Protocol for a Mixed Methods Case Study With Realist Evaluation

BACKGROUND

Research suggests that the rapid increase in worldwide antidepressant use is mainly due to a rise in long-term and potentially inappropriate use. It has been suggested that 1 in 3 antidepressant users among general practice patients are no longer experiencing clinical benefits from their medication and should commence deprescribing. However there are many barriers to antidepressant deprescribing for both patients and clinicians, which adds to the complex nature of reducing or ceasing the medication. As such, antidepressant deprescribing does not routinely occur in clinical practice. Evidence-based supports and interventions for safe and successful antidepressant deprescribing are needed to assist patients and their doctors. Interventions should also include an understanding of how an intervention works, why it works, and whom it is for.

OBJECTIVE

This study aims to evaluate how the WiserAD approach to antidepressant deprescribing works, whom it is for, and the underlying circumstances by (1) examining the experiences and perceptions of WiserAD among antidepressant users, (2) identifying the underlying mechanisms of the WiserAD approach to antidepressant deprescribing, and (3) describing in what contexts and to what extent the underlying mechanisms of WiserAD are suited for antidepressant users.

METHODS

A mixed methods case study with realist evaluation will be conducted among participants in the WiserAD randomized controlled trial for antidepressant deprescribing. Quantitative data will be obtained from up to 12 participants from the intervention and control arms at baseline and 3-month follow-up. Baseline data will be used to characterize the sample using descriptive statistics. Paired samples t tests will also be performed to compare responses between baseline and 3-month follow-up for participant self-management, skills, confidence and knowledge, beliefs about medicines, current emotional health, and well-being symptoms. Qualitative data from the same participants will be collected via narrative interview at 3-month follow-up. Quantitative and qualitative data will be converged to form a "case," and analysis will be conducted within each case with comparisons made across multiple cases.

RESULTS

Recruitment of participants commenced in October 2022 and will be completed by March 2023. Analysis will be completed by June 2023.

CONCLUSIONS

To our knowledge, this will be the first realist evaluation of an antidepressant deprescribing intervention in general practice. Findings from this evaluation may assist in the implementation of the WiserAD approach to antidepressant deprescribing in routine clinical practice.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/42526.

How, for whom, and in what contexts will artificial intelligence be adopted in pathology? A realist interview study

OBJECTIVE

There is increasing interest in using artificial intelligence (AI) in pathology to improve accuracy and efficiency. Studies of clinicians' perceptions of AI have found only moderate acceptability, suggesting further research is needed regarding integration into clinical practice. This study aimed to explore stakeholders' theories concerning how and in what contexts AI is likely to become integrated into pathology.

MATERIALS AND METHODS

A literature review provided tentative theories that were revised through a realist interview study with 20 pathologists and 5 pathology trainees. Questions sought to elicit whether, and in what ways, the tentative theories fitted with interviewees' perceptions and experiences. Analysis focused on identifying the contextual factors that may support or constrain uptake of AI in pathology.

RESULTS

Interviews highlighted the importance of trust in AI, with interviewees emphasizing evaluation and the opportunity for pathologists to become familiar with AI as means for establishing trust. Interviewees expressed a desire to be involved in design and implementation of AI tools, to ensure such tools address pressing needs, but needs vary by subspecialty. Workflow integration is desired but whether AI tools should work automatically will vary according to the task and the context.

CONCLUSIONS

It must not be assumed that AI tools that provide benefit in one subspecialty will provide benefit in others. Pathologists should be involved in the decision to introduce AI, with opportunity to assess strengths and weaknesses. Further research is needed concerning the evidence required to satisfy pathologists regarding the benefits of AI.

A realist evaluation of a multifactorial falls prevention programme in care homes

BACKGROUND

falls in care homes are common, costly and hard to prevent.Multifactorial falls programmes demonstrate clinical and cost-effectiveness, but the heterogeneity of the care home sector is a barrier to their implementation. A fuller appreciation of the relationship between care home context and falls programme delivery will guide development and support implementation.

METHODS

this is a multi-method process evaluation informed by a realist approach.Data include fidelity observations, stakeholder interviews, focus groups, documentary review and falls-rate data. Thematic analysis of qualitative data and descriptive statistics are synthesised to generate care home case studies.

RESULTS

data were collected in six care homes where a falls programme was trialled. Forty-four interviews and 11 focus groups complemented observations and document review.The impact of the programme varied. Five factors were identified: (i) prior practice and (ii) training may inhibit new ways of working; (iii) some staff may be reluctant to take responsibility for falls; (iv) some may feel that residents living with dementia cannot be prevented from falling; and, (v) changes to management may disturb local innovation.In some care homes, training and improved awareness generated a reduction in falls without formal assessments being carried out.

CONCLUSIONS

different aspects of the falls programme sparked different mechanisms in different settings, with differing impact upon falls.The evaluation has shown that elements of a multifactorial falls programme can work independently of each other and that it is the local context (and local challenges faced), which should shape how a falls programme is implemented.

Realist evaluation in times of decolonising global health

Realist evaluation (RE) is a theory-driven evaluation approach inspired by scientific realism. It has become increasingly popular in the field of global health where it is often applied in low- and middle-income countries. This makes it timely to discuss RE's relationship to the emerging decolonisation of global health movement. In this short perspective, we argue that the principles and practices that underpin RE have great potential to contribute to the decolonisation endeavour. Both the focus on the inclusion of local stakeholders and the openness to the rival theories these stakeholders bring to the fore, are promising. However, in practice, we see that a lack of acknowledgement of power imbalances and different ontologies and an overreliance on Western-based theories thwart this potential. We therefore suggest that realist evaluations performed by external researchers, especially in the field of global health, should actively engage with issues of (power) inequities. This is not only the just thing to do, but will also contribute to a better understanding of the intervention and may facilitate the emancipation of the disenfranchised. One way of doing this is through the adoption of participatory (action) research methods, currently underused in realist evaluations. We finally give a short example of an evaluation that combines emancipatory and participatory practice development with a realist approach. The Afya-Tek project in Tanzania has an innovative bottom-up approach throughout the full evaluation cycle and shows the possible strength of the proposed combination to create better interventions, more empowered stakeholders, and more illuminating programme theories.

Matching of woman and doula, and the generation of trust in an Australian volunteer doula program: Findings from a realist evaluation

How women are cared for while pregnant and having a new baby can have profound and lasting effects on their health and well-being. While mainstream maternity care systems aspire to provide care that is woman-centred, women with fewest social and economic resources often have reduced access. Community-based doula support programs offer complementary care for these women and are known to, on average, have positive outcomes. Less understood is how, when and why these programs work. A realist evaluation of an Australian volunteer doula program provided for women experiencing socioeconomic adversity explored these questions. The program provides free non-medical, social, emotional, and practical support by trained doulas during pregnancy, birth and new parenting. This paper reports the testing and refinement of one program theory from the larger study. The theory, previously developed from key informant interviews and rapid realist review of literature, hypothesised that the cultural matching of woman (client) and doula led to best outcomes. This was tested in realist interviews with women and focus groups with doulas, in January-February 2020. Seven English speaking, and six Arabic speaking clients were interviewed. Two focus groups were conducted with a total of eight doulas from diverse cultural and professional backgrounds. Data were analysed in NVivo. The study found cultural matching to be valued by some but not all women, and only when the doula was also genuinely interested, kind, timely and reliable. These approaches (with or without cultural matching) generate trust between the doula and woman. Trust theory, reflexivity theory and social relations theory supported explanatory understanding of the causal contribution of a doula knowing what it takes to build trust, to a woman deciding to trust her doula.

Conducting focus groups in realist evaluation

Focus groups are valuable tools for evaluators to help stakeholders to clarify programme theories. In 1987, R.K. Merton, often attributed with the birth of focus groups, wrote about how these were 'being mercilessly misused'. In the 1940s, his team had conceived focus groups as tools for developing middle-range theory, but through their astonishing success focus groups have metamorphosed and are often an 'unchallenged' choice in many evaluation approaches, while their practice seems to provide a philosophically diverse picture. This article examines what knowledge focus group data generate, and how they support theory development. It starts with an overview of the history of focus groups, establishing a relationship between their emergence as a data collection method and the evaluation profession. Practical lessons for conducting groups in realist evaluation are suggested, while exploring how qualitative data can support programme and middle-range theory development using the example of realist evaluation.

A conceptual model of the impact of including carers in museum programmes for people with dementia

BACKGROUND

Research has highlighted a need for more theoretical work in arts interventions, including the role of the dyad. This study aimed to test theories from a literature review on the impact of including carers in museum programmes for people with dementia, and develop a model which can be used in other programmes to consider the impact of including carers more broadly.

METHODS

Using a realist evaluation approach, theory was developed through interviews and participant-observation at six museums in the UK.

FINDINGS

The impacts of including carers fell into seven broad areas - caring responsibility, session function, controlling access, preventing engagement, comparisons and losses, long-term impact of in-the-moment activities, and reducing social isolation and opening up the museum.

CONCLUSIONS

Including carers may have both unanticipated benefits and negative consequences, and greater attention is needed on how both carers and people with dementia can be supported in shared sessions. Carers should be viewed as participants of programmes, and can even be the main beneficiaries, even where the programme is ostensibly 'for' the person with dementia - it's not simply that carers are the enablers of, or barriers to, the impacts on the person with dementia.

A Tale of Two Cities: Unpacking the Success and Failure of School Street Interventions in Two Canadian Cities

One innovative strategy to support child-friendly cities is street-based interventions that provide safe, vehicle-free spaces for children to play and move about freely. School streets are one such innovation involving closing streets around elementary schools to vehicular traffic to improve children's safety as they come and go from school while providing opportunities for children to play and socialize on the street. Launching these initiatives in communities dominated by automobiles is enormously challenging and little is known about why these interventions are successfully launched in some places but not others. As part of a larger research project called Levelling the Playing Fields, two School Street initiatives were planned for the 2021-2022 school year; one initiative was successfully launched in Kingston, ON, while the second initiative failed to launch in Montreal, QC. Using a critical realist evaluation methodology, this paper documents the contextual elements and key mechanisms that enabled and constrained the launch of these School Streets in these cities, through document analysis and key informant interviews. Our results suggest that municipal and school support for the initiative are both imperative to establishing legitimacy and collaborative governance, both of which were necessary for a successful launch.

'Really there because they care': The importance of service users' interpretations of staff motivations at a crisis intervention service in New Zealand

Crisis intervention services for people experiencing psychological distress and suicidal ideation are frequently described by the people accessing them as failing to meet their needs. This paper reports a prominent finding from a realist evaluation of Taranaki Retreat-a charitable, non-clinical organisation in New Zealand, which offers free respite for people experiencing acute distress. Using qualitative methods, the study aimed to move beyond vague notions regarding the helpfulness of respite, to a deeper understanding of the contextual factors and mechanisms which generate outcomes for such an intervention. Participant observation, focus groups with staff, semi-structured interviews with service users, and analysis of service users' case notes were conducted over a six-month period in 2018. The most prominent finding from the study related to 'genuine care'-care which is interpreted by the recipient as being motivated by a genuine desire to help. We present this finding as to the central mechanism in a wider programme theory developed through the realist evaluation study. We also present five key features of the care participants were offered at Taranaki Retreat which contributed to their common interpretation regarding the motivations behind this care. Upon considering the centrality of this mechanism we conclude that, in designing crisis interventions, greater consideration should be given to how the intervention can demonstrate genuine care. Having highlighted the ways in which the structure of charitable organisations appears conducive for interpretations of genuine care, we further conclude that the provision of comprehensive crisis intervention by charitable organisations should be further explored and supported.

Balancing the effectiveness and cost of online education: A preliminary realist economic evaluation

PURPOSE

While online learning for faculty development has grown substantially over recent decades, it has been further accelerated in the face of the worldwide pandemic. The effectiveness of online learning has been repeatedly established through systematic reviews and meta-analyses, yet questions remain about its cost-effectiveness. This study evaluates how synchronous online supervision training workshops and their cost-effectiveness might work, and in what contexts.

METHODS

We conducted preliminary realist economic evaluation including qualitative (13 realist interviews), and quantitative approaches (cost Ingredients method). We developed a cost-optimised model based on identified costs and cost-sensitive mechanisms.

RESULTS

We identified 14 recurring patterns (so-called demi-regularities) illustrating multiple online workshop outcomes (e.g. satisfaction, engagement, knowledge), generated by various mechanisms (e.g. online technology, mixed pedagogies involving didactic and active/experiential learning, peer learning), and triggered by two contexts (supervisor experience levels, and workplace location). Each workshop cost $302.92 per learner, but the optimised model including senior facilitators cost $305.70.

CONCLUSIONS

Our initial realist program theories were largely supported and refined. Although findings were largely concordant with previous literature, we illustrate how online workshop costs compare favourably with face-to-face alternatives. We encourage program developers to consider synchronous online learning for faculty development especially for remote learners, and in resource-constrained environments.

Exploring the impacts of a fresh produce market program: a realist economic evaluation

OBJECTIVE

To determine the contexts under which a fresh food market program is cost-effective in improving dignified access to nutritious food for food-insecure individuals.

METHODS

A realist economic evaluation was employed. Purported cost related theories about how the program may function, known as context-mechanism-outcome configurations were developed. In-depth interviews with key stakeholders (program developers, funder, local food relief agencies, volunteers) involved in the program (n=19) as well Photovoice with focus groups with market attendees (n=8) were conducted and coded for contexts, mechanisms and outcomes. A cost-effectiveness analysis of the program was calculated whereby the cost inputs associated with operating the program were compared to the quantity and value of produce distributed. Alternative cost scenarios were evaluated in a sensitivity analysis. The cost-effectiveness analysis was used together with qualitative data to refine theory.

RESULTS

Food insecure individuals attending a partnership fresh food market with a small fee, experienced improved, yet infrequent access to nutritious food through community connections and support a more dignified, viable access to fresh nutritious food.

CONCLUSIONS

Food relief should consider alternative models.

IMPLICATIONS FOR PUBLIC HEALTH

More dignified food relief programs that support local connections may be part of the solution to addressing food insecurity.

The person-based development and realist evaluation of a pre-consultation form for GP consultations

Background

Use of telephone, video and e-consultations is increasing. These can make consultations more transactional, potentially missing patients' concerns. This study aimed to develop a complex intervention to address patients' concerns more comprehensively in general practice and test the feasibility of this in a cluster-randomised framework.The complex intervention used two technologies: a patient-completed pre-consultation form used at consultation opening and a doctor-provided summary report provided at consultation closure. This paper reports on the development and realist evaluation of the pre-consultation questionnaire.

Methods

A person-based approach was used to develop the pre-consultation form. An online questionnaire system was designed to allow patient self-completion of a form which could be shared with GPs. This was tested with 45 patients in three rounds, with iterative adjustments made based on feedback after each round.Subsequently, an intervention incorporating the pre-consultation form with the summary report was then tested in a cluster-randomised framework with 30 patients per practice in six practices: four randomised to intervention, and two to control. An embedded realist evaluation was carried out. The main feasibility study results are reported elsewhere.

Results

Intervention Development: 15 patients were recruited per practice. Twelve patients, six GPs and three administrators were interviewed and 32 changes were made iteratively in three rounds. Recruitment rates (proportion of patients responding to the text) increased from 15% in round one to 50% in round three.Realist evaluation: The pre-consultation form was most useful for people comfortable with technology and with hidden concerns or anxiety about the consultation. It resulted in more issues being discussed and support provided, more effective use of time and greater patient satisfaction.

Conclusions

The person-based approach was successful. The pre-consultation form uncovers more depth and improves satisfaction in certain consultations and patients. Technological improvements are required before this could be rolled out more widely.

Nursing staff perspectives of continuous remote vital signs monitoring on surgical wards: Theory elicitation for a realist evaluation

RATIONALE, AIMS AND OBJECTIVES

Continuous remote monitoring (CRM) provides a novel solution to the challenges of monitoring patients' vital signs in hospital, but the results of quantitative studies have been mixed. Acceptance by staff is a crucial determinant of the success of healthcare technologies and may explain these discrepancies. Drawing on the approach of realist evaluation, this paper aims to identify theories about how, why and in what conditions nursing staff perceptions vary regarding the CRM of patients' vital signs.

METHODS

Multiple methods were used to elicit theories about factors likely to facilitate or impede the successful implementation of continuous remote vital signs monitoring. This included a literature review, consultation with patients and observational work conducted during a randomized controlled trial (RCT) of CRM. In addition, a priori theories developed through informal interactions with patients and ward staff during the day-to-day set-up of the trial were included.

RESULTS

The findings suggest that the perceptions of nursing staff regarding remote monitoring can be influenced by the type of patients under their care and their previous experience of telemetry. Factors which may undermine the engagement of staff are perceived staff burden, which can be dependent on contextual factors such as staffing levels, time of day and senior staff attitudes. Staff attitudes are also likely to be influenced by patient perspectives and the utility of the devices associated with remote monitoring. The successful implementation of CRM may be dependent on staff training, research staff input and hospital culture.

CONCLUSIONS

Theories regarding nursing staff engagement with remote monitoring are numerous, varied and contradictory. The theories elicited in this initial phase will be refined during interviews with the nursing staff involved with the RCT.

A realist evaluation of weighted modalities as an alternative to pro re nata medication for mental health inpatients

Implementing psychosocial approaches into mental health inpatient settings continues to be challenging. This is despite mental health policies prioritizing trauma-informed and recovery-orientated care approaches. This study reports on an interdisciplinary project that implemented and examined the uptake of weighted modalities in a psychiatric inpatient rehabilitation setting. While over-reliance on pro re nata (PRN) medication in these settings has been reported, weighted modalities are a sensory approach that can be self-initiated by consumers as an alternative to use of PRN to manage distress and agitation. A mixed-methods realist approach was used to determine what works, for whom, and in what circumstances weighted modalities were offered by clinicians and used by consumers. Additionally, we were interested in determining whether there was an associated change in PRN use during the trial. Data included in-depth interviews with consumers (n = 12) and clinicians (n = 11), and extraction of PRN medication use preceding, throughout, and following implementation. Statistical analysis revealed a significant increase in PRN rates in the first 3 months of implementation (mean difference from baseline = 29.5, P = 0.047) but a significant decrease in the second three-month period following implementation (mean difference from baseline = -30.7, P = 0.036). Qualitative findings highlighted the key components of successful implementation as being environmental restructuring (availability/accessibility and visibility of weighted modalities) and social influences. Strategies to enhance these components are discussed.

Personal Health Records an Approach to Answer: What Works for Whom in What Circumstances?

National Health Service (NHS) policy suggests that increasing usage of electronic personal health records (PHR) by patients will result in cost savings and improved public health, especially for people with long-term conditions. PHR design features are inevitably important, since a good PHR design should make the users achieve their health goals effortlessly, which is understandable and usable. Three original theoretical models were developed using realist evaluation, one per long-term condition cohort, describing the interaction between the PHR design features and the patient and disease specific factors, to help determine what works for whom in what circumstances.

Recruiting residents from disadvantaged neighbourhoods for community-based health promotion and disease prevention services in Denmark-How, why and under what circumstances does an active door-to-door recruitment approach work?

Residents living in disadvantaged neighbourhoods have higher mortality and morbidity and participate less in health services than people living in more prosperous areas. Studies on how and why different approaches work for recruiting residents from disadvantaged neighbourhoods to health services are needed. Conducting face-to-face meetings with obliging health professionals (HPs) as part of an active recruitment strategy increases interest in service use, particularly among the most vulnerable residents. The services offered must be free, flexible and easily accessible for residents from disadvantaged neighbourhoods to consider their use. Residents immediately consider and 'translate' the relevance and acceptability of health services presented to them in light of their current situation, which affects their response. Individual contextual circumstances such as current health issues or difficult life circumstances, either promote or dampen their interest in using the services. This study is a realist evaluation of active face-to-face recruitment where HPs go from door-to-door to visit residents in disadvantaged neighbourhoods and offer them community-based health promotion and disease prevention services. A programme theory explicating the assumptions behind this recruitment approach was tested empirically in the present study. Eight direct observations of the active recruitment process in three different neighbourhoods and 17 realist interviews with residents were conducted. Sampling of residents was purposeful and opportunistic. Data were collected between March and August 2018. The findings support the use of active face-to-face recruitment in disadvantaged neighbourhoods to increase participation in health interventions. A broad range of health services should be presented to residents to accommodate their different needs and interests. Refusal to have services presented during active recruitment was rare, but more knowledge about these reasons for declining services is needed.

A realist evaluation case study of the implementation of advanced nurse practitioner roles in primary care in Scotland

Aim

To evaluate Advanced Nurse Practitioner (ANP) role implementation in primary care across Scotland in contributing to primary care transformation, and establish what works, for whom, why and in what context.

Design

A realist evaluation using multiple case studies.

Methods

Two phases, conducted March 2017 to May 2018: (1) multiple case studies of ANP implementation in 15 health boards across Scotland, deductive thematic analysis of interviews, documentary analysis; (2) in‐depth case studies of five health boards, framework analysis of interviews and focus groups.

Results

Sixty‐eight informants were interviewed, and 72 documents were reviewed across both phases. ANP roles involved substitution for elements of the GP role for minor illness and injuries, across all ages. In rural areas ANPs undertook multiple nursing roles, were more autonomous and managed greater complexity. Mechanisms that facilitated implementation included: the national ANP definition; GP, primary care team and public engagement; funding for ANP education; and experienced GP supervisors. Contexts that affected mechanisms were national and local leadership; remote, rural and island communities; and workload challenges. Small‐scale evaluations indicated that ANPs: make appropriate decisions; improve patient access and experience.

Conclusions

At the time of the evaluation, the implementation of ANP roles in primary care in Scotland was in early stages. Capacity to train ANPs in a service already under pressure was challenging. Shifting elements of GPs workload to ANPs freed up GPs but did little to transform primary care. Local evaluations provided some evidence that ANPs were delivering high‐quality primary care services and enhanced primary care services to nursing homes or home visits.

Impact

ANP roles can be implemented with greater success and have more potential to transform primary care when the mechanisms include leadership at all levels, ANP roles that value advanced nursing knowledge, and appropriate education programmes delivered in the context of multidisciplinary collaboration.

Looking Back and Looking Around: How Athletes, Parents and Coaches See Psychosocial Development in Adolescent Performance Sport

Sport has the potential to support psychosocial development in young people. However, extant studies have tended to evaluate purpose-built interventions, leaving regular organised sport relatively overlooked. Moreover, previous work has tended to concentrated on a narrow range of outcomes. To address these gaps, we conducted a season-long ethnography of a youth performance sport club based on a novel Realist Evaluation approach. We construed the club as a social intervention within a complex system of agents and structures. The results are published in this special issue as a two-part series. In this first paper, we detail the perceptions of former and current club parents, players and coaches, using them to build a set of programme theories. The resulting network of outcomes (i.e., self, emotional, social, moral and cognitive) and generative mechanisms (i.e., the attention factory, the greenhouse for growth, the personal boost and the real-life simulator), spanning across multiple contextual layers, provides a nuanced understanding of stakeholders' views and experiences. This textured perspective of the multi-faceted process of development provides new insights for administrators, coaches and parents to maximise the developmental properties of youth sport, and signposts new avenues for research in this area.

Understanding How and Why Alcohol Interventions Prevent and Reduce Problematic Alcohol Consumption among Older Adults: A Systematic Review

Problematic alcohol use has been increasing in older adults (55+) in recent decades. Many of the effective interventions that are available to prevent or reduce the negative effects of alcohol consumption are aimed at adults in general. It is unclear whether these interventions also work for older adults. The objective of this review was to understand how (i.e., which elements), in which context, and why (which mechanisms) interventions are successful in preventing or reducing (problematic) alcohol consumption among older adults. A systematic review of articles published between 2000 and 2022 was performed using PubMed, PsycINFO, Web of Science and CHINAHL. Realist evaluation was used to analyze the data. We found 61 studies on interventions aimed at preventing or reducing problematic alcohol use. Most of the interventions were not specifically designed for older adults but also included older adults. The findings of the current study highlight three major effective elements of interventions: (1) providing information on the consequences of alcohol consumption; (2) being in contact with others and communicating with them about (alcohol) problems; and (3) personalized feedback about drinking behavior. Two of these elements were also used in the interventions especially designed for older adults. Being in contact with others and communicating with them about (alcohol) problems is an important element to pay attention to for developers of alcohol interventions for older adults because loneliness is a problem for this age group and there is a relationship between the use of alcohol and loneliness.

First Things First: How to Elicit the Initial Program Theory for a Realist Evaluation of Complex Integrated Care Programs

Policy Points Realist evaluation (RE) is an emerging and promising research approach for evaluating integrated care, addressing what works, how, for whom, and in what circumstances. The rich philosophical foundation of RE, critical realism, can help to systematically unravel an integrated care program's initial theory prior to implementation, as a first step within RE. RE can be considered a robust methodological asset in integrated care research by facilitating a deeper level of insight into program functioning than traditional forms of evaluation do and by shaping a realist-informed monitoring and evaluation process.

CONTEXT

The complexity of integrated care and the need for transferable evaluation insights ask for a suitable evaluation paradigm. Realist evaluation (RE), underpinned by the philosophy of critical realism, is a theory-driven approach that addresses what works, how, for whom, and in what circumstances. The current study illustrates the process needed for RE's first step: eliciting the initial program theory (IPT). The TARGET program, a Dutch primary care initiative to facilitate more integrated care for chronically ill patients, i.e., care that is efficient, tailored, and holistic,  was taken as a real-world case.

METHODS

An RE approach informed the phased IPT elicitation: (1) identifying an abstract theory framework; (2) formulating the preliminary IPT, building on the abstract theory and informed by previous scientific studies that underpin TARGET; and (3) refining the preliminary IPT, informed by RE expert interviews (n = 7). An RE heuristic tool, specifying the interplay between intervention-context-actors-mechanisms-outcomes (ICAMO) and retroductive reasoning, was applied to synthesize the underlying theory of individual TARGET components into TARGET's IPT.

FINDINGS

Separate but related IPTs were identified for the two main types of actors involved in TARGET: primary care professionals (PCPs) and patients. For both actors, two sorts of mechanisms are assumed to be activated by TARGET, which-via instrumental outcomes-contribute to long-term quadruple aim targets. The first is confidence to enhance PCPs' person-centered conversational skills and to increase patients' active engagement in TARGET. The second is mutual trust, between PCPs and patients and between PCPs and their network partners. A supportive context is assumed crucial for activating these mechanisms-for example, sufficient resources to invest in integrated care.

CONCLUSIONS

Although the IPT elicitation process is time intensive and requires a mind shift, it facilitates a deeper insight into program functioning than accommodated by the prevailing experimental designs in integrated care. Furthermore, the design of a realist-informed evaluation process can be informed by the IPT.

Understanding Trustful Relationships between Community Health Workers and Vulnerable Citizens during the COVID-19 Pandemic: A Realist Evaluation

(1) Background: Community health workers (CHWs) are an essential public health workforce defined by their trustful relationships with vulnerable citizens. However, how trustful relationships are built remains unclear. This study aimed to understand how and under which circumstances CHWs are likely to build trust with their vulnerable clients during the COVID-19 pandemic. (2) Methods: We developed a program theory using a realist research design. Data were collected through focus groups and in-depth interviews with CHWs and their clients. Using a grounded theory approach, we aimed to unravel mechanisms and contextual factors that determine the trust in a CHW program offering psychosocial support to vulnerable citizens during the COVID-19 pandemic. (3) Results: The trustful relationship between CHWs and their clients is rooted in three mental models: recognition, equality, and reciprocity. Five contextual factors (adopting a client-centered attitude, coordination, temporariness, and link with primary care practice (PCP)) enable the program mechanisms to work. (4) Conclusions: CHWs are a crucial public health outreach strategy for PCP and complement and enhance trust-building by primary care professionals. In the process of building trustful relationships between CHWs and clients, different mechanisms and contextual factors play a role in the trustful relationship between primary care professionals and patients. Future research should assess whether these findings also apply to a non-covid context, to the involvement of CHWs in other facets of primary healthcare (e.g., prevention campaigns, etc.), and to a low- and middle-income country (LMIC) setting. Furthermore, implementation research should elaborate on the integration of CHWs in PCP to support CHWs in developing the mental models leading to build trust with vulnerable citizens and to establish the required conditions.

Developing New Methods for Person-Centred Approaches to Adjudicate Context-Mechanism-Outcome Configurations in Realist Evaluation

Realist evaluation provides a general method of evaluating the application of interventions including policy, legislation, projects, and new processes in social settings such as law enforcement, healthcare and education. Realist evaluation focuses on what about interventions works, for whom, and in what circumstances, and there is a growing body of work using realist evaluation to analyse interventions in healthcare organizations, including those using Lean Six Sigma improvement methodologies. Whilst realist evaluation facilitates the analysis of interventions using both qualitative and quantitative research, there is little guidance given on methods of data collection and analysis. The purpose of this study is to address this lack of guidance through detailing the use of innovative person-centred methods of data collection and analysis in a realist evaluation that enabled us to understand the contribution of Lean Six Sigma to person-centred care and cultures. This use of person-centred principles in the adjudication of identified program theories has informed novel methods of collecting and analysing data in realist evaluation that facilitate a person-centred approach to working with research participants and a way of making the implicit explicit when adjudicating program theory.

Village health worker motivation for better performance in a maternal and child health programme in Nigeria: A realist evaluation

Background

Community health workers play an important role in linking communities with formal health service providers, thereby improving access to and utilization of health care. A novel cadre of community health workers known as village health workers (VHWs) were recruited to create demand for maternal health services in the Nigerian Subsidy Reinvestment Programme (SURE-P/MCH). In this study, we investigated the role of contextual factors and underlying mechanisms motivating VHWs.

Methods

We used realist evaluation to understand the impact of a multi-intervention maternal and child health programme on VHW motivation using Anambra State as a case study. Initial working theories and logic maps were developed through literature review and stakeholder engagement; programme theories were developed and tested using focus group discussions and in-depth interviews with various stakeholder groups. Interview transcripts were analysed through an integrated approach of Context, Mechanism and Outcomes (CMO) categorisation and connecting, and matching of patterns of CMO configurations. Motivation theories were used to explain factors that influence VHW motivation. Explanatory configurations are reported in line with RAMESES reporting standards.

Results

The performance of VHWs in the SURE-P maternal and child health programme was linked to four main mechanisms of motivation: feelings of confidence, sense of identity or feeling of acceptance, feeling of happiness and hopefulness/expectation of valued outcome. These mechanisms were triggered by interactions of programme-specific contexts and resources such as training and supervision of VHWs by skilled health workers, provision of first aid kits and uniforms, and payments of a monthly stipend. The monthly payment was considered to be the most important motivational factor by VHWs. VHWs used a combination of innovative approaches to create demand for maternity services among pregnant women, and their performance was influenced by health system factors such as organisational capacity and culture, and societal factors such as relationship with the community and community support.

Conclusion

This paper highlights important contextual factors and mechanisms for VHW motivation that can be applied to other interventions that seek to strengthen community engagement and demand creation in primary health care. Future research on how to sustain VHW motivation is also required.

'They need to ask me first'. Community engagement with low-income citizens. A realist qualitative case-study

Background

Community engagement is seen as key to citizen‐centred and sustainable healthcare systems as involving citizens in the designing, implementation and improvement of services and policies is thought to tailor these more closely to communities’ own needs and experiences. Organizations have struggled to reach out to and involve disadvantaged citizens. This paper examines how if, why, and when low‐income citizens wish to be involved.

Methods

For this qualitative realist case‐study, 19 interviews (one dyad) were held with (20) low‐income citizens in two Dutch municipalities. Additionally, the results were discussed with a reference panel consisting of professionals and citizens to enrich the results and to ensure the results had face validity.

Results

The results showed four different ways in which low‐income citizens wished to be involved: (a) in a practical/volunteer way; (b) as a buddy; (c) as a lay expert; (d) not involved at all. The factors affecting citizens’ interest and capacity to participate include citizens’ own experiences of the services they access and their personal situations, e.g. their mental or physical health, extent of financial crisis, family situation, home environment. None of the interviewees was currently involved, but all had ideas for improving health(care) services and policies. Citizens’ experiences of the services they accessed acted as a motivator for some to be involved as they wanted to ensure others would not have the same struggles, while for others their own needs and an apathetic system remained too high a barrier. To enable involvement, citizens need continued support for their own health(care) and financial situation, better communication and accessibility from services, practical support (e.g., training and bus passes) and recognition for their input (e.g., monetary compensation).

Conclusion

The study shows that citizens’ experiences of the services they accessed influenced if and how they wanted to be involved with health and care services. Despite the fact that all participants had shared solid ideas for improving services and policies, they were hindered by a bureaucratic, impersonal and inaccessible system. Organizations seem to underestimate the required investments to reach out to low‐income citizens and the support required to ensure their involvement.

Patient and Public Involvement (PPI) in Study

Citizens as well as PPI organizations were members of the reference panel who helped formulate the research questions and recruitment strategy. The local reference panel also helped to interpret and refine the initial findings.

Nurse-led projects for people experiencing homelessness and other inclusion health groups: a realist evaluation

Nursing service development or innovation projects, even small-scale ones, can be difficult to deliver and evaluate, due to a lack of resources and support. Results can also be difficult to disseminate, limiting transfer of learning. This paper presents findings from a realist evaluation of 10 small projects supported by the Queen's Nursing Institute Homeless and Inclusion Health Programme to deliver innovation in health care for people experiencing homelessness and other marginalised groups. These nurse-led projects were funded by the Queen's Nursing Institute and the Oak Foundation, and were largely successful in achieving outcomes to support the improved health of people experiencing homelessness and other marginalised groups. This realist evaluation explores the factors that contributed to the delivery of positive outcomes. All were impacted by the context and the response (mechanisms) of people experiencing homelessness and staff within these settings. It is hoped that the lessons learned will enable better support for nurse innovation projects in the future.

Realist Evaluation of the "Abiye" Safe Motherhood Initiative in Nigeria: Unveiling the Black-Box of Program Implementation and Health System Strengthening

Introduction

Realist evaluation studies have spanned different aspects of medicine, especially in the field of public health. However, very few of these studies explicitly detailed how program implementation triggered outcomes that could strengthen understanding of its effect on Health System Strengthening in specific settings. In low- and middle-income countries, like Nigeria, there is a paucity of realist evaluation studies, despite the implementation of multiple intervention programs and projects in these countries. This article is aimed at unveiling the black-box of program implementation and Health System Strengthening of the "Abiye" Safe Motherhood Program in Ondo State, Nigeria. Specifically, it identified the role of contextual factors in the "Abiye" program in Ondo State, determined the mechanisms that facilitated or constrained outcomes of the "Abiye" program, and developed a Context Mechanism Outcome (CMO) Configuration from which a Middle Range Theory (MRT) can be framed.

Methodology

This was qualitative research structured along with the realist domains (Context, Mechanism, and Outcome). The Initial Program Theory was validated by the qualitative study, after which a new MRT was developed. The study population comprised key stakeholders, secondary stakeholders, and primary stakeholders in the Abiye safe motherhood program. Data was collected through 10 key informant interviews, 28 in-depth interviews, and six focus group discussions sessions. Thematic analysis was used to analyze all the qualitative data collected, and seven themes with 19 subthemes emerged in the study.

Results

We identified 13 contextual factors under five principal areas, with most of the factors playing enabling roles, some playing inhibitory roles, while very few played both roles. We elicited eight mechanisms, and some of these facilitated the outcomes, while some constrained the outcomes of the program. Health system strengthening was a key feature of the outcome of the program. We developed a middle-range theory based on the 6 CMO configurations we elicited from the study.

Conclusion and Policy Implications

Realist evaluation is an iterative process that looks beyond the surface to generate evidence. By applying the realist approach, we generated pieces of evidence that can be adapted for policymaking in public health interventions in LMIC.

The role of contextual factors on participation in the life area of work and employment after rehabilitation: A qualitative study on the views of persons with chronic pain

BACKGROUND:

A deeper understanding of how contextual factors affect the ability to participate in the life area of work and employment despite chronic musculoskeletal pain is needed as a basis for interprofessional rehabilitation programs.

OBJECTIVE:

To investigate which contextual factors influence rehabilitation program clients’ ability to participate in the life area of work and employment, and how they do this.

METHODS:

Nested case study using a realist evaluation framework of interprofessional interventions. Qualitative content analysis of problem-centered interviews to identify influential context-mechanism-outcome configurations.

RESULTS:

We identified several important context-mechanism-outcome configurations. In the pre-interventional phase, socioeconomic and environmental factors affected two mechanisms, “exhaustion” and “discrimination”. In the intra-interventional phase, the social skills of health professionals and opportunities for discussion with peers affected the ability of program participants to engage with program content. In the post-intervention phase, volitional competences of the social system affected the sustainable application of program content in everyday life.

CONCLUSION:

The identified context-mechanism-outcome configurations shows that the ability to participate in the life area of work is interdependent with the ability to participate in other areas of life. In practice and research, assessment and treatment should be carried out based on this understanding.

A Realist Evaluation of a 72-Hour Readmission Audit and Feedback (A&F) Intervention in Emergency Medicine

Introduction

Audit and feedback (A&F) interventions are intended to increase accountability and improve the quality of care; however, their impact can vary significantly. As performance feedback is implemented in healthcare, there is a growing need to determine how users interact with the data and how systems can achieve more consistent performance outcomes. This study aimed to understand the contexts, mechanisms, and outcomes of an emergency department 72-hour readmission A&F intervention.

Methods

Semi-structured interviews with key stakeholders were conducted and analyzed using thematic and template analysis techniques specifically aimed at identifying context, mechanism, and outcome configurations.

Results

Seventeen (17) physician interviews were conducted. We identified five outcomes of the intervention and the contexts and mechanisms contributing to them. Importantly, we identified that this A&F strategy could potentially have positive (improved follow-up of cases, improved discharge communication) and negative impacts (increased physician anxiety, potentially increased resource use) on physicians and departmental efficiency.

Conclusion

The 72-hour readmission alert A&F intervention generates a number of distinct outcome patterns that result from a variety of mechanisms acting in different contexts. Knowledge of these context-mechanism-outcome relationships may help implementers design and tailor performance feedback strategies.

How does English national end-of-life care policy impact on the experience of older people at the end of life? Findings from a realist evaluation

Aim:

To explore the extent to which national policy in end-of-life care (EOLC) in England influences and guides local practice, helping to ensure that care for older people at the EOL is of a consistently good quality.

Background:

Whilst policy is recognised as an important component in determining the effectiveness of EOLC, there is scant literature which attempts to interrogate how this happens or to hypothesise the mechanisms linking policy to better outcomes.

Method:

This article reports on the second phase of a realist evaluation comprising three case studies of clinical commissioning groups, including 98 in-depth interviews with stakeholders, meeting observation and documentary analysis.

Findings:

This study reveals the key contextual factors which need to be in place at micro, meso and macro levels if good quality EOLC for older people is to be achieved. The findings provide insight into rising local inequalities and reveal areas of dissonance between stakeholder priorities. Whilst patients privilege the importance of receiving care and compassion in familiar surroundings at EOL, there remains a clear tension between this and the medical drive to cure disease and extend life. The apparent devaluing of social care and subsequent lack of resource has impacted significantly on the way in which dying is experienced.

Patient experience at EOL, shaped by the care received both formally and informally, is driven by a fragmented health and social care system. Whilst the importance of system integration appears to have been recognised, significant challenges remain in terms of shaping policy to adequately reflect this. This study highlights the priority attached by patients and their families to the social and relational aspect of death and dying and shines a light on the stark disparities between the health and social care systems which became even more evident at the height of the Covid-19 pandemic.

Realist evaluation of a pilot intervention implementing interprofessional and interinstitutional processes for transitional care

In 2016, in Switzerland, we implemented transitional interprofessional and interinstitutional shared decision-making processes (IIPs) between a short-stay inpatient care unit (SSU) and primary care professionals. Between 2018 and 2019, we evaluated this intervention using a realist design to answer the following questions: for whom, with whom, in which context and how have IIPs been implemented? Our initial theory was tested via interviews with patients, primary care professionals and staff from the SSU. Results showed that a patient's stay at the SSU, with actors committed to facilitating IIPs, reinforced the perceived appropriateness and implementation of those IIPs. However, this appropriateness varied according to different contextual elements, such as the complexity of needs, preexisting collaborative practices and the purpose of the inpatient stay. Since IIPs occurred in a context of fragmented practices, proactive and sustained efforts are required of the actors implementing them and the organizations supporting them.

A Realist Inquiry to Identify the Contribution of Lean Six Sigma to Person-Centred Care and Cultures

A lack of fidelity to Lean Six Sigma's (LSS) philosophical roots can create division between person-centred approaches to transforming care experiences and services, and system wide quality improvement methods focused solely on efficiency and clinical outcomes. There is little research into, and a poor understanding of, the mechanisms and processes through which LSS education influences healthcare staffs' person-centred practice. This realist inquiry asks 'whether, to what extent and in what ways, LSS in healthcare contributes to person-centred care and cultures'. Realist review identified three potential Context, Mechanism, Outcome configurations (CMOcs) explaining how LSS influenced practice, relating to staff, patients, and organisational influences. Realist evaluation was used to explore the CMOc relating to staff, showing how they interacted with a LSS education Programme (the intervention) with CMOc adjudication by the research team and study participants to determine whether, to what extent, and in what ways it influenced person-centred cultures. Three more focused CMOcs emerged from the adjudication of the CMOc relating to staff, and these were aligned to previously identified synergies and divergences between participants' LSS practice and person-centred cultures. This enabled us to understand the contribution of LSS to person-centred care and cultures that contribute to the evidence base on the study of quality improvement beyond intervention effectiveness alone.