How beneficiaries see complex health interventions: a practice review of the Most Significant Change in ten countries

Traditional Midwifery Contribution to Safe Birth in Cultural Safety: Narrative Evaluation of an Intervention in Guerrero, Mexico

A 2017 randomised controlled trial in Guerrero State, Mexico, showed supporting Indigenous traditional midwives on their own terms improved traditional childbirths without inferior maternal health outcomes. This narrative evaluation complements the trial to document participant experience of safer birth in cultural safety, transformative dynamics and implementation issues of the intervention. Stories came from 26 traditional midwives, 28 apprentices, 12 intercultural brokers and 20 Indigenous women who experienced the intervention. Their accounts indicate the intervention revitalised traditional midwifery and consolidated local skills through traditional midwife apprentices and intercultural brokers to support safe birth. According to the stories, communities reintroduced traditional perinatal care and reported positive health impacts for mothers, children, and other adults, which contributed to early collaboration with official health services. Challenges included remuneration and disinterest of younger apprentices and brokers. The intervention seems to have improved interaction between traditional and Western services, setting the stage for further intercultural dialogue.

Fuzzy cognitive mapping in participatory research and decision making: a practice review

BACKGROUND

Fuzzy cognitive mapping (FCM) is a graphic technique to describe causal understanding in a wide range of applications. This practice review summarises the experience of a group of participatory research specialists and trainees who used FCM to include stakeholder views in addressing health challenges. From a meeting of the research group, this practice review reports 25 experiences with FCM in nine countries between 2016 and 2023.

RESULTS

The methods, challenges and adjustments focus on participatory research practice. FCM portrayed multiple sources of knowledge: stakeholder knowledge, systematic reviews of literature, and survey data. Methodological advances included techniques to contrast and combine maps from different sources using Bayesian procedures, protocols to enhance the quality of data collection, and tools to facilitate analysis. Summary graphs communicating FCM findings sacrificed detail but facilitated stakeholder discussion of the most important relationships. We used maps not as predictive models but to surface and share perspectives of how change could happen and to inform dialogue. Analysis included simple manual techniques and sophisticated computer-based solutions. A wide range of experience in initiating, drawing, analysing, and communicating the maps illustrates FCM flexibility for different contexts and skill bases.

CONCLUSIONS

A strong core procedure can contribute to more robust applications of the technique while adapting FCM for different research settings. Decision-making often involves choices between plausible interventions in a context of uncertainty and multiple possible answers to the same question. FCM offers systematic and traceable ways to document, contrast and sometimes to combine perspectives, incorporating stakeholder experience and causal models to inform decision-making. Different depths of FCM analysis open opportunities for applying the technique in skill-limited settings.

Combining realist evaluation and transformative evaluation to advance research in palliative care: The case of end of life companionship

BACKGROUND

Palliative care requires innovative methods to understand what works, for whom, in what circumstances and why. Realist evaluation has become one prominent approach due to its preoccupation with building, and testing, causal theories to explain the influence of contextual factors on outcomes. Undertaking realist evaluation is not without challenges and may amplify issues of underrepresentation, disempower those working in palliative care, and produce results with poor ecological validity. Complementary approaches are needed which mitigate these challenges, whilst producing credible findings that advances knowledge.

PURPOSE

In this article it is outlined how realist evaluation provides a toolkit to advance research to explain, and empirically test, the complex contours of palliative care. Moreover, it is proposed that transformative evaluation can provide a catalyst to engage and empower those within palliative care, create the opportunity for care transformation, and produce more informed and authentic theories.

DISCUSSION

Contemporary issues in palliative care pertain to the complexity of palliative care, the insufficiency of experimental designs alone, and the challenges of achieving inclusive research participation. In this article it is argued that theory led, participatory, opportunistic and naturalistic approaches can provide an antidote to the issues in the literature. The combination also mitigates many methodological critiques of the individual approaches, by increasing the transformative potential of realist evaluation, and explanatory potential of transformative evaluation.

Ghanaian Female Adolescents Perceived Changes in Nutritional Behaviors and Social Environment After Creating Participatory Videos: A Most Significant Change Evaluation

Background

Understanding the influence of participatory video-making on the nutrition-related behavior of video creators may help shape nutrition education interventions.

Objectives

This study assessed the perceived value and influence of a participatory video intervention among participants and stakeholders.

Methods

A 2018-2019 cluster randomized controlled trial (registered at clinicaltrials.gov as NCT03704649) selected 20 schools (10 intervention, n = 181; 10 control, n = 170) in 1 Ghanaian rural district, enrolled adolescent girls aged 13-16 y, and provided a nutrition curriculum. Each intervention school also participated in 2 series of activities designed to help adolescents plan, film, and screen 2 nutrition-related videos. The Most Significant Change method involved intervention participants and local stakeholders to assess the value and influence of the intervention - a secondary outcome of the trial. Project staff collected 116 stories of change from the adolescents. Stories described shifts in 4 domains: participant, peer, and family behavior, and structural changes in the school. The project team used a selection rubric to identify 14 stories that reflected heightened nutrition literacy. Staff conducted interviews with the 14 adolescents whose stories were selected to elaborate on details and perceived resonance. Finally, local stakeholders assessed the stories to identify the 4 most significant changes of the intervention - 1 per domain. A separate thematic analysis identified emerging patterns of motivation and action across the 14 interviews.

Results

The chosen Most Significant Change stories revealed how adolescents found creative solutions to acquire iron-rich foods, encouraged neighbors to eat iron-rich foods, taught their family new agricultural practices, and promoted change in their school canteen. Local stakeholders valued stories that addressed common community nutrition issues in a creative and sustainable way, whereas adolescents prioritized stories that showed a change in health outcomes.

Conclusion

Stories of change revealed that the intervention promoted a transformative influence; participants modified their eating habits, lifestyle, and their environment.

Evaluating a peer-led wellbeing programme for doctors-in-training during the COVID-19 pandemic in Victoria, Australia, using the Most Significant Change technique

This article discusses the use of the Most Significant Change (MSC) technique in a mixed-methods evaluation of a pilot wellbeing programme for obstetrics and gynaecology doctors-in-training introduced at a large public hospital during Melbourne, Australia's second coronavirus (COVID-19) lockdown, which occurred from 7 July to 26 October 2020. The evaluation was conducted remotely using videoconferencing technology, to conform with pandemic restrictions. MSC complemented the program's participatory principles and was chosen because it seeks to learn about participants' perceptions of programme impacts by evaluating their stories of significant change. Stakeholders select one story exemplifying the most significant change resulting from the evaluated program. Inductive thematic analysis of all stories is combined with reasons for making the selection, to inform learnings (Dart & Davies, 2003; Tonkin et al., 2021). Nine stories of change were included in the selection. The most significant change was a more supportive workplace culture brought about by enabling basic needs to be met and breaking down hierarchical barriers. This was linked to five interconnected themes - connection, caring, communication, confidence and cooperation. The evaluation learnings are explored and reflections on remotely conducting MSC evaluation are shared.

Impact-Oriented Dialogue for Culturally Safe Adolescent Sexual and Reproductive Health in Bauchi State, Nigeria: Protocol for a Codesigned Pragmatic Cluster Randomized Controlled Trial

Background

Adolescents (10-19 years) are a big segment of the Nigerian population, and they face serious risks to their health and well-being. Maternal mortality is very high in Nigeria, and rates of pregnancy and maternal deaths are high among female adolescents. Rates of HIV infection are rising among adolescents, gender violence and sexual abuse are common, and knowledge about sexual and reproductive health risks is low. Adolescent sexual and reproductive health (ASRH) indicators are worse in the north of the country.

Objective

In Bauchi State, northern Nigeria, the project will document the nature and extent of ASRH outcomes and risks, discuss the findings and codesign solutions with local stakeholders, and measure the short-term impact of the discussions and proposed solutions.

Methods

The participatory research project is a sequential mixed-methods codesign of a pragmatic cluster randomized controlled trial. Focus groups of local stakeholders (female and male adolescents, parents, traditional and religious leaders, service providers, and planners) will identify local priority ASRH concerns. The same stakeholder groups will map their knowledge of factors causing these concerns using the fuzzy cognitive mapping (FCM) technique. Findings from the maps and a scoping review will inform the contextualization of survey instruments to collect information about ASRH from female and male adolescents and parents in households and from local service providers. The survey will take place in 60 Bauchi communities. Adolescents will cocreate materials to share the findings from the maps and survey. In 30 communities, randomly allocated, the project will engage adolescents and other stakeholders in households, communities, and services to discuss the evidence and to design and implement culturally acceptable actions to improve ASRH. A follow-up survey in communities with and without the intervention will measure the short-term impact of these discussions and actions. We will also evaluate the intervention process and use narrative techniques to assess its impact qualitatively.

Results

Focus groups to explore ASRH concerns of stakeholders began in October 2021. Baseline data collection in the household survey is expected to take place in mid-2022. The study was approved by the Bauchi State Health Research Ethics Committee, approval number NREC/03/11/19B/2021/03 (March 1, 2021), and by the Faculty of Medicine and Health Sciences Institutional Review Board McGill University (September 13, 2021).

Conclusions

Evidence about factors related to ASRH outcomes in Nigeria and implementation and testing of a dialogic intervention to improve these outcomes will fill a gap in the literature. The project will document and test the effectiveness of a participatory approach to ASRH intervention research.

Trial Registration

ISRCTN Registry ISRCTN18295275; https://www.isrctn.com/ISRCTN18295275

International Registered Report Identifier (IRRID)

DERR1-10.2196/36060

Lessons Learned From the Use of the Most Significant Change Technique for Adaptive Management of Complex Health Interventions

INTRODUCTION

The Most Significant Change (MSC) technique is a complex-aware monitoring and evaluation tool, widely recognized for various adaptive management purposes. The documentation of practical examples using the MSC technique for an ongoing monitoring purpose is limited. We aim to fill the current gap by documenting and sharing the experience and lessons learned of The Challenge Initiative (TCI), which is scaling up evidence-based family planning (FP) and adolescent and youth sexual and reproductive health (AYSRH) interventions in 11 countries in Asia and sub-Saharan Africa.

METHODS

The qualitative assessment took place in early 2021 to document TCI's use and adaptation of MSC and determine its added value in adaptive management, routine monitoring, and cross-learning efforts. Focus group discussions and key informant interviews were conducted virtually with staff members involved in collecting and selecting MSC stories.

RESULTS

TCI has had a positive experience with using MSC to facilitate adaptive management in multiple countries. The use of MSC has created learning opportunities that have helped diffuse evidence-based FP and AYSRH interventions both within and across countries. The responsive feedback step in the MSC process was viewed as indispensable to learning and collaboration. There are several necessary inputs to successful use of the method, including buy-in about the benefits, training on good interviewing techniques and qualitative research, and dedicated staff to manage the process.

CONCLUSION

Our assessment results suggest that the MSC technique is an effective qualitative data collection tool to strengthen routine monitoring and adaptive management efforts that allows for flexibility in how project stakeholders implement the process. The MSC technique could be an important tool for global health practitioners, policy makers, and researchers working on complex interventions because they continually need to understand stakeholders' needs and priorities, learn from lessons and evidence-based practices, and be agile about addressing potential challenges.

"I had to change my attitude": narratives of most significant change explore the experience of universal home visits to pregnant women and their spouses in Bauchi State, Nigeria

BACKGROUND

Universal home visits to pregnant women and their spouses in Bauchi State, northern Nigeria, discussed local evidence about maternal and child health risks actionable by households. The expected results chain for improved health behaviours resulting from the visits was based on the CASCADA model, which includes Conscious knowledge, Attitudes, Subjective norms, intention to Change, Agency to change, Discussion of options, and Action to change. Previous quantitative analysis confirmed the impact of the visits on maternal and child outcomes. To explore the mechanisms of the quantitative improvements, we analysed participants' narratives of changes in their lives they attributed to the visits.

METHODS

Local researchers collected stories of change from 23 women and 21 men in households who had received home visits, from eight male and eight female home visitors, and from four government officers attached to the home visits program. We used a deductive thematic analysis based on the CASCADA results chain to analyze stories from women and men in households, and an inductive thematic approach to analyze stories from home visitors and government officials.

RESULTS

The stories from the visited women and men illustrated all steps in the CASCADA results chain. Almost all stories described increases in knowledge. Stories also described marked changes in attitudes and positive deviations from harmful subjective norms. Most stories recounted a change in behaviour attributed to the home visits, and many went on to mention a beneficial outcome of the behaviour change. Men, as well as women, described significant changes. The home visitors' stories described increases in knowledge, increased self-confidence and status in the community, and, among women, financial empowerment.

CONCLUSIONS

The narratives of change gave insights into likely mechanisms of impact of the home visits, at least in the Bauchi setting. The compatibility of our findings with the CASCADA results chain supports the use of this model in designing and analysing similar interventions in other settings. The indication that the home visits changed male engagement has broader relevance and contributes to the ongoing debate about how to increase male involvement in reproductive health.