Grand rounds in methodology: when are realist reviews useful, and what does a 'good' realist review look like?

A realist evaluation of how, why and when objective structured clinical exams (OSCEs) are experienced as an authentic assessment of clinical preparedness

INTRODUCTION

Whilst rarely researched, the authenticity with which Objective Structured Clinical Exams (OSCEs) simulate practice is arguably critical to making valid judgements about candidates' preparedness to progress in their training. We studied how and why an OSCE gave rise to different experiences of authenticity for different participants under different circumstances.

METHODS

We used Realist evaluation, collecting data through interviews/focus groups from participants across four UK medical schools who participated in an OSCE which aimed to enhance authenticity.

RESULTS

Several features of OSCE stations (realistic, complex, complete cases, sufficient time, autonomy, props, guidelines, limited examiner interaction etc) combined to enable students to project into their future roles, judge and integrate information, consider their actions and act naturally. When this occurred, their performances felt like an authentic representation of their clinical practice. This didn't work all the time: focusing on unavoidable differences with practice, incongruous features, anxiety and preoccupation with examiners' expectations sometimes disrupted immersion, producing inauthenticity.

CONCLUSIONS

The perception of authenticity in OSCEs appears to originate from an interaction of station design with individual preferences and contextual expectations. Whilst tentatively suggesting ways to promote authenticity, more understanding is needed of candidates' interaction with simulation and scenario immersion in summative assessment.

Communication partner training for student health and social care professionals engaging with people with stroke acquired communication difficulties: A protocol for a realist review

Background

Stroke acquired communication impairments impede effective communication. Consequently, in stroke care, communicative interactions can be challenging for both patients and staff and can predispose patients to increased risk of preventable adverse events. Communication partner training (CPT) can mitigate such negative outcomes by optimising communicative interactions. Providing CPT to student health and social care professionals (SH&SCPs) has the potential to enhance their clinical expertise and experiences and enhance the future clinical care of patients with stroke acquired communication impairments. This research aims to expand our understanding of how CPT is operationalised for SH&SCPs in higher education institutions and determine: what works; for whom; in what contexts; how and why?

Methods

This review is Phase 1 of a research project employing a realist approach with public and patient involvement (PPI). It incorporates five iterative steps: 1.) Clarifying the scope; 2.) Searching for evidence; 3.) Selecting and appraising evidence; 4.) Data extraction; 5.) Synthesising data and developing a middle range theory explaining how CPT is expected to work for SH&SCPs. An advisory panel, including PPI advisors, content advisors, student advisors, realist advisors and educationalist advisor has been set up to consult throughout the review and collaboratively agree the middle range theory.

Discussion

While there is an evolving evidence base for CPT, including stroke specific CPT for SH&SCPs, it is acknowledged that there are challenges to its implementation in complex real-world settings. In combining empirical evidence with theoretical understanding, realist review permits synthesis of data from diverse sources and goes beyond determining efficacy to explore generative causation and solutions for real world practice. A middle range realist programme theory that coherently explains how CPT is expected to work when teaching SH&SCPs to communicate with people with stroke acquired communication impairments will provide educators with new insights into CPT development and implementation in their higher education institutions.

Mechanisms behind gender transformative approaches targeting adolescent pregnancy in low- and middle-income countries: a realist synthesis protocol

INTRODUCTION

Adolescent pregnancy is defined as pregnancy at the age of 19 or below. Pregnancy and childbirth complications are the most significant cause of death among 15-19-year-old girls. Several studies have indicated that inequitable gender norms can increase the vulnerability of adolescent girls, including violence exposure, early marriage, and adolescent pregnancy. To address these disparities, gender transformative approaches aim to challenge and transform restrictive gender norms, roles, and relations through targeted interventions, promoting progressive changes. This realist review aims to synthesise existing evidence from a broad range of data sources to understand how, why, for whom, and in what contexts gender transformative approaches succeed in reducing adolescent pregnancy in low- and middle-income countries.

METHOD AND ANALYSIS

We employ a five-step realist synthesis approach: (1) clarify the scope of review and assessment of published literature, (2) development of initial programme theories, (3) systematic search for evidence, (4) development of refined programme theories, and (5) expert feedback and dissemination of results. This protocol presents the results of the first three steps and provides details of the next steps. We extracted data from 18 studies and outlined eight initial programme theories on how gender transformative approaches targeting adolescent pregnancy work in the first three steps. These steps were guided by experts in the field of sexual and reproductive health, implementation science, and realist methodology. As a next step, we will systematically search evidence from electronic databases and grey literature to identify additional studies eligible to refine the initial programme theories. Finally, we will propose refined programme theories that explain how gender transformative approaches work, why, for whom, and under which circumstances.

ETHICS AND DISSEMINATION

Ethics approval is not required because the included studies are published articles and other policy and intervention reports. Key results will be shared with the broader audience via academic papers in open-access journals, conferences, and policy recommendations. The protocol for this realist review is registered in PROSPERO (CRD42023398293).

Long COVID in long-term care: a rapid realist review

OBJECTIVES

The goals of this rapid realist review were to ask: (a) what are the key mechanisms that drive successful interventions for long COVID in long-term care (LTC) and (b) what are the critical contexts that determine whether the mechanisms produce the intended outcomes?

DESIGN

Rapid realist review.

DATA SOURCES

Medline, CINAHL, Embase, PsycINFO and Web of Science for peer-reviewed literature and Google for grey literature were searched up to 23 February 2023.

ELIGIBILITY CRITERIA

We included sources focused on interventions, persons in LTC, long COVID or post-acute phase at least 4 weeks following initial COVID-19 infection and ones that had a connection with source materials.

DATA EXTRACTION AND SYNTHESIS

Three independent reviewers searched, screened and coded studies. Two independent moderators resolved conflicts. A data extraction tool organised relevant data into context-mechanism-outcome configurations using realist methodology. Twenty-one sources provided 51 intervention data excerpts used to develop our programme theory. Synthesised findings were presented to a reference group and expert panel for confirmatory purposes.

RESULTS

Fifteen peer-reviewed articles and six grey literature sources were eligible for inclusion. Eleven context-mechanism-outcome configurations identify those contextual factors and underlying mechanisms associated with desired outcomes, such as clinical care processes and policies that ensure timely access to requisite resources for quality care delivery, and resident-centred assessments and care planning to address resident preferences and needs. The underlying mechanisms associated with enhanced outcomes for LTC long COVID survivors were: awareness, accountability, vigilance and empathetic listening.

CONCLUSIONS

Although the LTC sector struggles with organisational capacity issues, they should be aware that comprehensively assessing and monitoring COVID-19 survivors and providing timely interventions to those with long COVID is imperative. This is due to the greater care needs of residents with long COVID, and coordinated efficient care is required to optimise their quality of life.

Integrating palliative care and heart failure: a systematic realist synthesis (PalliatHeartSynthesis)

OBJECTIVES

(1) Develop a programme theory of why, for whom and in what contexts integrated palliative care (PC) and heart failure (HF) services work/do not work; (2) use the programme theory to co-produce with stakeholders, intervention strategies to inform best practice and future research.

METHODS

A systematic review of all published articles and grey literature using a realist logic of analysis. The search strategy combined terms significant to the review questions: HF, PC and end of life. Documents were included if they were in English and provided data relevant to integration of PC and HF services. Searches were conducted in November 2021 in EMBASE, MEDLINE, PsycINFO, AMED, HMIC and CINAHL. Further relevant documents were identified via monthly alerts (up until April 2023) and the project stakeholder group (patient/carers, content experts and multidisciplinary practitioners).

RESULTS

130 documents were included (86 research, 22 literature reviews, 22 grey literature). The programme theory identified intervention strategies most likely to support integration of PC and HF services. These included protected time for evidence-based PC and HF education from undergraduate/postgraduate level and continuing professional practice; choice of educational setting (eg, online, face-to-face or hybrid); increased awareness and seeing benefits of PC for HF management; conveying the emotive and intellectual need for integrating PC and HF via credible champions; and prioritising PC and HF guidelines in practice.

CONCLUSIONS

The review findings outline the required steps to take to increase the likelihood that all key players have the capacity, opportunity and motivation to integrate PC into HF management.

PROSPERO REGISTRATION NUMBER

CRD42021240185.

A realist synthesis of educational outreach visiting and integrated academic detailing to influence prescribing in ambulatory care: why relationships and dialogue matter

BACKGROUND

Many quality improvement initiatives in healthcare employ educational outreach visits, integrating academic detailing to bridge evidence-practice gaps and accelerate knowledge translation. Replicability of their outcomes in different contexts varies, and what makes some visiting programmes more successful than others is unclear.

OBJECTIVE

We conducted a realist synthesis to develop theories of what makes educational outreach visiting integrating academic detailing work, for whom, under which circumstances and why, focusing on the clinician-visitor interaction when influencing prescribing of medicines in ambulatory care settings.

METHODS

The realist review was performed in accordance with RAMESES standards. An initial programme theory was generated, academic databases and grey literature were screened for documents with detail on contexts, intervention and outcomes. Using realist logic of analysis, data from 43 documents were synthesised in the generation of a refined programme theory, supported by additional theoretical frameworks of learning and communication.

RESULTS

Twenty-seven interdependent context-mechanism-outcome configurations explain how clinicians engage with educational outreach visits integrating academic detailing through programme design, what matters in programme design and the educational visitor-clinician interaction and how influence extends beyond the visit. They suggest that in addition to relevance, credibility and trustworthiness of a visit's contents, communication and clinical skills of educational visitors, the relationship between the educational visitor and clinician, built on a dialogue of learning from and sense-making with each other, creates conditions of critical thinking which are conducive to facilitating prescribing practice change when necessary.

CONCLUSION

This realist synthesis elucidates that the quality of clinician-educational visitor interactions is pivotal to educational outreach visiting programmes. Building and sustaining relationships, and establishing an open dialogue are important; neglecting these undermines the impact of visits. Educational visitors can facilitate clinicians' reflection on practice and influence their prescribing. Clinicians value the discussion of individualised, tailored information and advice they can translate into their practice.

PROSPERO REGISTRATION NUMBER

CRD42021258199.

A realist review of medication optimisation of community dwelling service users with serious mental illness

BACKGROUND

Severe mental illness (SMI) incorporates schizophrenia, bipolar disorder, non-organic psychosis, personality disorder or any other severe and enduring mental health illness. Medication, particularly antipsychotics and mood stabilisers are the main treatment options. Medication optimisation is a hallmark of medication safety, characterised by the use of collaborative, person-centred approaches. There is very little published research describing medication optimisation with people living with SMI.

OBJECTIVE

Published literature and two stakeholder groups were employed to answer: What works for whom and in what circumstances to optimise medication use with people living with SMI in the community?

METHODS

A five-stage realist review was co-conducted with a lived experience group of individuals living with SMI and a practitioner group caring for individuals with SMI. An initial programme theory was developed. A formal literature search was conducted across eight bibliographic databases, and literature were screened for relevance to programme theory refinement. In total 60 papers contributed to the review. 42 papers were from the original database search with 18 papers identified from additional database searches and citation searches conducted based on stakeholder recommendations.

RESULTS

Our programme theory represents a continuum from a service user's initial diagnosis of SMI to therapeutic alliance development with practitioners, followed by mutual exchange of information, shared decision-making and medication optimisation. Accompanying the programme theory are 11 context-mechanism-outcome configurations that propose evidence-informed contextual factors and mechanisms that either facilitate or impede medication optimisation. Two mid-range theories highlighted in this review are supported decision-making and trust formation.

CONCLUSIONS

Supported decision-making and trust are foundational to overcoming stigma and establishing 'safety' and comfort between service users and practitioners. Avenues for future research include the influence of stigma and equity across cultural and ethnic groups with individuals with SMI; and use of trained supports, such as peer support workers.

PROSPERO REGISTRATION NUMBER

CRD42021280980.

Peer support interventions in maternal and child healthcare delivery in sub-Saharan Africa: protocol for a realist review

BACKGROUND

Peer support has been proposed as a promising policy intervention for addressing adverse maternal and child healthcare (MCH) outcomes in sub-Saharan Africa (SSA). Existing reviews on peer support largely draw on evidence from high-income countries or focus on single services like breastfeeding, nutrition or postnatal care. In contrast, this review aims to provide a comprehensive overview of the empirical literature on peer support interventions across various MCH services in sub-Saharan Africa. Specifically, we aim to understand how, why, for whom, and in what circumstances different forms of MCH peer support interventions contribute to improving healthcare outcomes in sub-Saharan Africa.

METHODS

This review follows five iterative steps for undertaking realist reviews (1) defining the review scope; (2) developing initial programme theories; (3) searching for evidence; (4) selecting and appraising evidence; and (5) extracting, analysing and synthesising evidence. Four databases-Cochrane Library, PubMed, CINAHL, and EMBASE-were repeatedly searched between March and June 2021. From a large volume of records retrieved from the database and citation search, 61 papers have been selected for review. We will conduct a second search of the same database covering June 2021 to the present before the final extraction and synthesis. The final list of selected papers will be imported into NVivo 12 software and organised, extracted, analysed and synthesised iteratively to examine and illustrate the causal links between contexts, mechanisms and outcomes of MCH peer support interventions in SSA. We have drawn on the existing literature on peer support in healthcare generally to develop initial programme theories. We will then use the empirical literature on MCH peer support interventions in SSA, inputs from a stakeholders' workshop in Ghana and a conference presentation to refine the initial programme theory.

DISCUSSION

The review will develop an explicit theory of peer support intervention in healthcare delivery and provide insights for developing evidence-informed policy on the intervention. Drawing lessons from the different national contexts and diverse areas of MCH in SSA, the review will provide an analytically generalizable programme theory that can guide intervention design and implementation. While focusing on MCH peer support interventions in SSA, the review contributes to evolving conversations on the use of theory for health policy planning and complex intervention design and implementation globally.

TRIAL REGISTRATION

PROSPERO registration ID: CRD42023427751 .

Improving access to general practice for people with severe and multiple disadvantage: a realist review protocol (The Connection Study)

Introduction: Despite having high unmet health need, people with severe and multiple disadvantage (SMD, including combinations of homelessness, substance misuse, poor mental health and domestic violence and abuse) have poor access to general practice. This realist review will examine the existing evidence on interventions or aspects of routine care in general practice that are likely to increase or decrease access to general practice for people with SMD. Methods and analysis: he aim of this review is to identify how these interventions or aspects of routine care increase or decrease access to general practice for people with SMD, in which contexts and for which patients. This review will involve a process comprising five sequential phases: (1) identifying established theories, (2) conducting an extensive search for proof, (3) selecting appropriate articles, (4) gathering and organising relevant data, and (5) utilising a realist analytical approach to synthesise evidence and make conclusions. Local implementation documents, in addition to published research studies, will be incorporated to enrich the analysis. We will collaborate with a stakeholder group consisting of people with lived experience of SMD and those who support them to advise us throughout. Ethics and dissemination: Ethical approval is not required. Our findings will be disseminated through peer-reviewed publications, conference presentations and lay summaries and will be used to develop a complex intervention for improving access to general practice for and with people with severe and multiple disadvantage. PROSPERO registration number: CRD42023390495.

Improving the mental health and mental health support available to adolescents in out-of-home care via Adolescent-Focused Low-Intensity Life Story Work: a realist review

OBJECTIVES

Life Story Work (LSW) is used to promote the mental health and well-being of children and adolescents living in out-of-home care. LSW should be offered to all but is conventionally delivered in high-intensity ways. Low-intensity approaches are more accessible but there is significant variation and little guidance for supporting adolescents. We aimed to create guidance for Adolescent-Focused Low-Intensity LSW.

DESIGN

Realist review.

DATA SOURCES

MEDLINE, Embase, PsycINFO, Sociology Collection (ProQuest), CINAHL, CDAS, Web of Science (SCIE, SSCI), Social Care Online and grey literature sources. Searches were performed between December 2021 and March 2022.

ELIGIBILITY CRITERIA

Documents on children and adolescents in care, LSW and/or low-intensity interventions to improve mental health were included. Documents focusing on parenting style and contact with birth family were excluded.

ANALYSIS

Documents were analysed using a realist logic of analysis. In consultation with Content Expert Groups (comprising professionals and care leavers), we developed an initial programme theory. Data relating to and challenging the initial programme theory were extracted and context-mechanism-outcome-configurations developed, critiqued and refined in an iterative fashion. Interpretations were drawn from context-mechanism-outcome-configurations to enhance the programme theory.

RESULTS

75 documents contributed to the analysis. Generally, studies were small-scale and lacked in-depth methods and evaluation descriptions. Findings indicated important factors contribute to the development of high-quality Adolescent-Focused Low-Intensity LSW. Adolescent-Focused Low-Intensity LSW should be person-centred, begin in the now, involve co-construction, record everyday positive life events and be supported by trained carer(s). Context-mechanism-outcome-configurations relating to these themes are reported.

CONCLUSIONS

Using this knowledge we developed initial practice guidance to support social care to deliver better quality Adolescent-Focused Low-Intensity LSW more consistently. To address gaps in our knowledge about the impact of Adolescent-Focused Low-Intensity LSW, further primary research is needed to strengthen understandings of how this intervention works (or not) in different contexts.

PROSPERO REGISTRATION NUMBER

CRD42021279816.

Developing an understanding of networks with a focus on LMIC health systems: How and why clinical and programmatic networks form and function to be able to change practices: A realist review

Networks are an increasingly employed approach to improve quality of care, service delivery, and health systems performance, particularly in low-and-middle income country (LMIC) health systems. The literature shows that networks can improve the provision and quality of services and health system functioning but there is limited evidence explaining how and why networks are established and work to achieve their reported results. We undertook a realist review to explore this. The objective of this realist review was to develop a programme theory outlining the underlying mechanisms and interactions of contexts that explain how and why a network's set-up and function enable high-quality care and services and improved clinical outcomes in LMIC health systems. We followed Pawson's five steps for realist reviews. The search strategy was based on a previously published scoping review with additional searches. Literature was selected based on its relevance to the programme theory and rigour. Context-mechanism-outcome configurations were developed from the extracted data to refine the initial programme theory with causal explanations. Theories on social movements and organisations supported the identification of mechanism and brought additional explanatory power to the programme theory. The programme theory explains how networks are initiated, formed, and function in a way that sets them up for network leadership and committed, engaged, and motivated network members to emerge and to change practices, which may lead to improved quality of care, service delivery, and clinical outcomes through the following phases: identify a problem, developing a collective vision, taking action to solve the problem, forming purposeful relationships, linkages, and partnerships, building a network identity and culture, and the creation of a psychological safe space. This deeper understanding of networks formation and functioning can lead to a more considered planning and implementation of networks, thereby improving health system functioning and performance.

Health visiting in the UK in light of the COVID-19 pandemic experience (RReHOPE): a realist review protocol

INTRODUCTION

Health visiting services, providing support to under 5s and their families, are organised and delivered in very different ways in different parts of the UK. While there has been attention to the key components of health visiting practice and what works well and how, there is little research on how health visiting services are organised and delivered and how that affects their ability to meet their objectives. The COVID-19 pandemic rapidly disrupted service delivery from March 2020. This realist review aims to synthesise the evidence on changes during the pandemic to identify the potential for improving health visiting services and their delivery.

METHODS AND ANALYSIS

This review will follow the RAMESES (Realist And Meta-narrative Evidence Syntheses: Evolving Standards) quality standards and Pawson's five iterative stages to locate existing theories, search for evidence, select literature, extract data, synthesise evidence and draw conclusions. It will be guided by stakeholder engagement with practitioners, commissioners, policymakers, policy advocates and people with lived experience. This approach will consider the emerging strategies and evolving contexts in which the services are delivered, and the varied outcomes for different groups. A realist logic of analysis will be used to make sense of what was happening to health visiting services during and following the pandemic response through the identification and testing of programme theories. Our refined programme theory will then be used to develop recommendations for improving the organisation, delivery and ongoing postpandemic recovery of health visiting services.

ETHICS AND DISSEMINATION

General University Ethics Panel approval has been obtained from University of Stirling (reference 7662). Dissemination will build on links to policymakers, commissioners, providers, policy advocates and the public. A range of audiences will be targeted using outputs tailored to each. A final stakeholder event focused on knowledge mobilisation will aid development of recommendations.

PROSPERO REGISTRATION NUMBER

CRD42022343117.