Exploring the decision-making process of female genital cosmetic procedures in Iranian women and constructing and validating a results-based logic model for a healthy public policy: a study protocol

BACKGROUND

Female genital cosmetic procedures have grown rapidly in most parts of the world. Professional organizations have issued warnings about the complications and long-term consequences of these practices. To be able to adopt the right health policies, it is necessary to know why women decide to perform these procedures. Therefore, the present study will be aim to discover the decision-making process involved in performing female genital cosmetic procedures for Iranian women and construct and validate a results-based logic model for healthy public policy.

METHODS

The present study was conducted in three phases. In the initial phase, a qualitative study will be conducted with the Corbin and Strauss ground theory approach. The participants in the study will be healthy women who desire or have undergone female genital cosmetic procedures without medical indications. In this phase, purposive and theoretical sampling will guide recruitment and data collection. The data will be collected via semi-structured interviews, field notes and observations of individual interactions. The data will be analysed using the approach of Corbin and Strauss (2015). MAXQDA 2007 software was used for managing the process of data analysis. In the second phase, the development of a results-based logic model for a healthy public policy is performed based on the findings of the first phase of the study, interviews with key informants and a review of the results of the literature in this field. Finally, validation of the designed program will be performed by the nominal group technique with the presence of a group of experts in the third phase.

DISCUSSION

The findings of this study, by identifying women's main concerns related to the studied phenomenon, the existing context, participants' reactions and the consequences of the adopted reactions, can be very important in designing a program that fits Iran's cultural characteristics. In this research, a program using a logical model will be presented that is suitable for policymakers, planners and healthcare service providers to be implemented in the social-cultural context of the study.

An Evidence-Based Framework for Medical Student Research Training

Background

As research becomes an increasingly important component of medical education, there is greater emphasis on incorporating programmatic enhancements to the research experience. This study builds a logic model to summarize research program inputs, outputs, and outcomes from research-oriented medical schools across the country, providing a framework that institutions can use to design and improve their medical student research training programs.

Methods

Between November 2021 and February 2022, we administered a survey assessing institutional characteristics, research offerings, curriculum, funding, and student scholarly products to the medical schools ranked 1-50 in research in 2021 by US News and World Report. Results were compiled in the form of a logic model.

Results

Thirty-seven institutions (72.5%) responded. Common program inputs included personnel such as at least one funded program director (97.3%), while funding for medical student research activities was highly variable (8-72%). There was much less funding for faculty research mentors (2.7%), advisors (18.9%), and teaching faculty (29.7%). Common outputs included a medical student research office or program (97.3%), formal research curricula (83.8%), and services and programs such as research day (91.9%). The most common outcomes tracked were publications (48.6%), presentations/posters (43.2%), student participation (29.7%), and completion of a research requirement (29.7%).

Conclusions

Common themes in medical student research training programs may be conceptualized with a logic model that schools can use to develop, evaluate, and iteratively improve their programs. Institutions should consider their desired program outcomes prior to designing inputs (e.g., funding, personnel) and outputs (e.g., curriculum, training).

Supplementary Information

The online version contains supplementary material available at 10.1007/s40670-024-02001-3.

Tracking activities and adaptations in a multi-site stepped wedge pragmatic trial of a cancer symptom management intervention

Background

Pragmatic trials may need to adapt interventions to enhance local fit, and adaptation tracking is critical to evaluation. This study describes the tracking approach for a multisite, stepped-wedge hybrid pragmatic trial testing implementation and effectiveness of a cancer symptom management intervention.

Methods

Study activities were documented in a spreadsheet by date and category. Intervention adaptations were tracked across multiple workgroups in a database structured around the Framework for Reporting Adaptations and Modifications-Expanded (FRAME) domains, e.g., reasons for change. Implementation strategies were tracked longitudinally and by cluster in a database using the Longitudinal Implementation Strategy Tracking System (LISTS) method. A logic model was created at the end of the study to describe core intervention components and implementation strategies with dates of adaptations.

Results

Between January 2019 and January 2023, 187 study activities were documented. Most intervention activities took place early, but there were important intervention refinements during the course of the trial, including the expansion of interventionist roles to add two new disciplines. Eleven intervention adaptations were documented. Most were unplanned and aimed at improving fit or increasing engagement. Thirty-three implementation strategies were documented, the largest number of which were related to educating stakeholders. Most (but not all) component and strategy additions were consistent with the mechanisms of change as hypothesized at trial launch.

Conclusions

A multifaceted approach to adaptation tracking, combined with a logic model, supported identification of meaningful changes for use in evaluation, but further work is needed to minimize burden and ensure robust and practical systems that inform both evaluation and timely decision-making.

Trial

Registration: ClinicalTrials.gov, NCT03892967. Registered on March 25, 2019. https://www.clinicaltrials.gov/.

Mixed method evaluation of a learning from excellence programme for community health workers in Neno, Malawi

BACKGROUND

Community Health Workers (CHWs) play an essential role in linking communities to facility-based healthcare. However, CHW programmes have often been hampered by low levels of staff motivation, and new tools aimed at improving staff motivation and work environment are needed. One such intervention is the "Learning from Excellence" (LfE) programme. We aimed to assess feasibility, outputs, and impact of a co-designed LfE programme on CHW motivation, in Neno District.

METHODS

We conducted a convergent mixed-method evaluation of the LfE programme. Co-design of the programme and forms took place between October 2019 and January 2020. LfE forms submitted between September and November 2020 were analysed using descriptive statistics and memos summarising answers to the open-ended question. To investigate experiences with LfE we conducted in-depth semi-structured interviews with key stakeholders, CHWs, and site supervisors, which were analysed thematically. A pre-post intervention questionnaire was developed to assess the impact of the co-designed LfE intervention on CHW motivation and perceived supervision. Outcomes were triangulated into a logic model.

RESULTS

In total 555 LfE forms were submitted, with 34.4% of CHWs in Neno District submitting at least one LfE report. Four themes were identified in the interviews: LfE implementation processes, experience, consequences, and recommendations. A total of 50 CHWs participated in the questionnaire in January 2020 and 46 of them completed the questionnaire in December 2020. No statistically significant differences were identified between pre-and post-LfE measurements for both motivation (Site F: p = 0.86; Site G: p = 0.31) and perceived supervision (Site F: p = 0.95; Site G: p = 0.45). A logic model, explaining how the LfE programme could impact CHWs was developed.

CONCLUSIONS

Many CHWs participated in the LfE intervention between September 2020 and November 2020. LfE was welcomed by CHWs and stakeholders as it allowed them to appreciate excellent work in absence of other opportunities to do so. However, no statistically significant differences in CHW motivation and perceived supervision were identified. While the intervention was feasible in Neno District, we identified several barriers and facilitators for implementation. We developed a logic model to explain contextual factors, and mechanisms that could lead to LfE outcomes for CHWs in Neno District. The developed logic model can be used by those designing and implementing interventions like LfE for health workers.

Supported self-management for all with musculoskeletal pain: an inclusive approach to intervention development: the EASIER study

BACKGROUND

Supported self-management interventions for patients with musculoskeletal (MSK) conditions may not adequately support those with limited health literacy, leading to inequalities in care and variable outcomes. The aim of this study was to develop a model for inclusive supported self-management intervention(s) for MSK pain that take account of health literacy.

METHODS

A mixed methods study with four work-packages was conducted: work package 1: secondary analysis of existing data to identify potential targets for intervention; work package 2: evidence synthesis to assess effective components of self-management interventions taking into account health literacy; work package 3: views of community members and healthcare professionals (HCPs) on essential components; work package 4: triangulation of findings and an online modified Delphi approach to reach consensus on key components of a logic model.

FINDINGS

Findings identified targets for intervention as self-efficacy, illness perceptions, and pain catastrophizing. A range of intervention components were identified (e.g. information in diverse formats offered at specific times, action planning and visual demonstrations of exercise). Support should be multi-professional using a combination of delivery modes (e.g. remote, face-to-face).

CONCLUSIONS

This research has developed a patient-centred model for a multi-disciplinary, multi-modal approach to supported self-management for patients with MSK pain and varying levels of health literacy. The model is evidence-based and acceptable to both patients and HCPs, with potential for significant impact on the management of MSK pain and for improving patient health outcomes. Further work is needed to establish its efficacy.

Facility-based stillbirth review processes used in different countries across the world: a systematic review

Background

Facility-based stillbirth review provides opportunities to estimate incidence, evaluate causes and risk factors for stillbirths, and identify any issues related to the quality of pregnancy and childbirth care which require improvement. Our aim was to systematically review all types and methods of facility-based stillbirth review processes used in different countries across the world, to examine how stillbirth reviews in facility settings are being conducted worldwide and to identify the outcomes of implementing the reviews. Moreover, to identify facilitators and barriers influencing the implementation of the identified facility-based stillbirth reviews processes by conducting subgroup analyses.

Methods

A systematic review of published literature was conducted by searching MEDLINE (OvidSP) [1946-present], EMBASE (OvidSP) [1974-present], WHO Global Index Medicus (globalindexmedicus.net), Global Health (OvidSP) [1973-2022 Week 8] and CINAHL (EBSCOHost) [1982-present] from their inception until 11 January, 2023. For unpublished or grey literature, the WHO databases, Google Scholar and ProQuest Dissertations & Theses Global were searched, as well as hand searching the reference lists of included studies. MESH terms encompassing "∗Clinical Audit", "∗Perinatal Mortality", "Pregnancy Complications", and "Stillbirth" were used with Boolean operators. Studies that used a facility-based review process or any approach to evaluate care prior to stillbirth, and explained the methods used were included. Reviews and editorials were excluded. Three authors (YYB, UGA, and DBT) independently screened and extracted data, and assessed the risk of bias using an adapted JBI's Checklist for Case Series. A logic model was used to inform the narrative synthesis. The review protocol was registered with PROSPERO, CRD42022304239.

Findings

A total of 68 studies from 17 high-income (HICs) and 22 low-and-middle-income countries (LMICs) met the inclusion criteria from a total of 7258 identified records. These were stillbirth reviews conducted at different levels: district, state, national, and international. Three types were identified: audit, review, and confidential enquiry, but not all desired components were included in most processes, which led to a mismatch between the description of the type and the actual method used. Routine data from hospital records was the most common data source for identifying stillbirths, and case assessment was based on stillbirth definition in 48 out of 68 studies. Hospital notes were the most common source of information about care received and causes/risk factors for stillbirth. Short-term and medium-term outcomes were reported in 14 studies, but impact of the review process on reducing stillbirth, which is more difficult to establish, was not reported in any study. Facilitators and barriers in implementing a successful stillbirth review process identified from 14 studies focused on three main themes: resources, expertise, and commitment.

Interpretation

This systematic review's findings identified that there is a need for clear guidelines on how to measure the impact of implementation of changes based on outputs of stillbirth reviews and methods to enable effective dissemination of learning points in the future and promoting them through training platforms. In addition, there is a need to develop and adopt a universal definition of stillbirth to facilitate meaningful comparison of stillbirth rates between regions. The key limitation of this review is that while using a logic model for narrative synthesis was deemed most appropriate for this study, sequence of implementing a stillbirth review in the real world is not linear, and assumptions are often not met. Therefore, the logic model proposed in this study should be interpreted with flexibility when designing a stillbirth review process. The generated learnings from the stillbirth review processes inform the action plans and allow facilities to consider where the changes should happen to improve the quality of care in the facilities, enabling positive short-term and medium-term outcomes.

Funding

Kellogg College, University of Oxford, Clarendon Fund, University of Oxford, Nuffield Department of Population Health, University of Oxford and Medical Research Council (MRC).

Theory of Change in Complex Research for Development Programmes: Challenges and Solutions from the Global Challenges Research Fund

The United Kingdom Research and Innovation (UKRI) Global Challenges Research Fund (GCRF) aimed to address global challenges to achieve the United Nations (UN) Sustainable Development Goals through 12 interdisciplinary research hubs. This research documents key lessons learned around working with Theory of Change (ToC) to guide Monitoring, Evaluation and Learning (MEL) within these complex research for development hubs. Interviews and document reviews were conducted in ten of the research hubs. The results revealed that only one hub invested in an explicit visual system mapping approach, and that funder timelines, budget constraints and issues with capacity and expertise limited the application of these approaches across all hubs. In contrast, many hubs attempted to deal with visual complexity by means of ether constructing multiple, nested ToCs, or a conscious simplification of complexity through reducing their ToC towards a straightforward and uncomplicated chain model or spherical model. While the former approach had some value, most hubs struggled to find capacity to support the full articulation of nested ToCs. In contrast, the latter approach resulted in ToCs which lacked detail or mechanism articulation, but which nevertheless were often 'fit for purpose' in ensuring effective communication and coherence across diverse stakeholders and sub-projects. We conclude that in instances where the reporting, funding and management cycles of complex research for development programmes cannot be adapted to properly support learning-based approaches to ToC development, imposing simplicity in the ToC might be fit for purpose. This might also be preferable to more complex visual approaches that are only partially realised.

Logic model for opioid safety in chronic non-malignant pain management, an in-depth qualitative study

BACKGROUND

Opioids are commonly used for the management of chronic non-malignant pain in Pakistan; but there is a lack of literature around precursors or motivators in the use of opioids.

AIM

The study holistically explored factors contributing towards the unsafe use of opioids and identifies strategies to overcome them.

METHOD

Exploratory qualitative methods using interviews, focus groups and non-participant observational case studies were used. Interviews and focus groups were carried out face-to-face as well as virtually and observations were conducted in community pharmacies in Islamabad and Khyber Pukhtoon Khuwa province, Pakistan. Data were collected from 4 stakeholder groups; pharmacy policy makers (n = 11), people with chronic non-malignant pain (n = 14), doctors (n = 31) and community pharmacists (n = 36) by purposive critical case sampling method. Data were analysed inductively using reflexive thematic analysis and then deductively mapped to a social ecological framework. Non-participant observations were analysed using a cross case synthesis using explanation building technique. Data from all three methods were triangulated to develop a logic model.

RESULTS

Identified factors at macro (regulation), meso (social perceptions of pain and opioids) and micro levels (uncontrolled pain, self-medication, health literacy) and strategies are presented holistically and were used to develop a logic model for the prevention and mitigation of factors currently causing unsafe use of opioids.

CONCLUSION

The study provides an in-depth view of factors contributing towards diversion of pharmaceutical opioids and can help guide national and international policy makers in their future initiatives to promote safe use of opioids in the management of chronic non-malignant pain in Pakistan.

Process evaluation of a medical improvisation program for healthcare communication training

BACKGROUND

Medical improvisation is an innovative approach to train healthcare professionals in effective communication. The success of this type of training depends on the active engagement of participants.

METHOD

A total of 136 interprofessional healthcare workers completed surveys before and after a two-hour medical improvisation communication training session in October 2020. We investigated individual- and program-level contributors to participation outputs (e.g. engagement and experience).

RESULTS

97% of healthcare professionals in the participating department took part in the training. 82% described the training in positive terms or as a learning experience. Younger participants, medical doctors, and those who had difficulty tolerating uncertainty were less excited than others about training. Their engagement was associated with their excitement and group size. Uncertainty tolerance predicted engagement via a full mediation of excitement.

CONCLUSIONS

Building excitement, reducing uncertainty about training, and capping group size are likely to improve participant's engagement and experience and will inform dissemination and implementation efforts.

Synthesizing frameworks and tools to develop a plan for evaluating an online data utilization curriculum for teachers

This paper presents an overview of how a plan was developed for evaluating Develop Your Data Mindset: Essentials of Educational Data Use, an online curriculum intended to improve in-service and pre-service educators' knowledge and skills relevant to data utilization. Five categories of assessment, a program evaluation framework, logic model, theory of change, project plan, metrics spreadsheet, and the A+ Inquiry model are synthesized to demonstrate how they supported the development of an evaluation plan. Types of evidence are described to provide an overview of the need for improved instructional supports relevant to educator data utilization, how the activities to develop and implement an online curriculum are appropriate to meet the need, how well the activities are implemented as planned, and the extent to which intended outcomes of the curriculum are achieved. Applying the methods outlined in this paper yielded a plan to help ensure the curriculum would be assessed through a comprehensive evaluation lens. Limitations are acknowledged. Methods described in this paper could be adapted to support program evaluation planning and implementation beyond the scope of this project.

Development of a national health policy logic model to accelerate the integration of oncology and palliative care: a nationwide Delphi survey in Japan

BACKGROUND

Despite recommendations to deliver palliative care to cancer patients and their caregivers, their distress has not been alleviated satisfactorily. National health policies play a pivotal role in achieving a comprehensive range of quality palliative care delivery for the public. However, there is no standardised logic model to appraise the efficacy of these policies. This study aimed to develop a logic model of a national health policy to deliver cancer palliative care and to reach consensus towards specific policy proposals.

METHODS

A draft version of the logic model and specific policy proposals were formulated by the research team and the internal expert panel, and the independent external expert panel evaluated the policy proposals based on the Delphi survey to reach consensus.

RESULTS

The logic model was divided into three major conceptual categories: 'care-delivery at cancer hospitals', 'community care coordination', and 'social awareness of palliative care'. There were 18 and 45 major and minor policy proposals, which were categorised into four groups: requirement of government-designated cancer hospitals; financial support; Basic Plan to Promote Cancer Control Programs; and others. These policy proposals were independently evaluated by 64 external experts and the first to third Delphi round response rates were 96.9-98.4%. Finally, 47 policy proposals reached consensus. The priority of each proposal was evaluated within the four policy groups.

CONCLUSIONS

A national health policy logic model was developed to accelerate the provision of cancer palliative care. Further research is warranted to verify the study design to investigate the efficacy of the logic model.

Process evaluation of a multi-disciplinary complex intervention to improve care for older patients with chronic conditions in rural areas (the HandinHand Study): study protocol

BACKGROUND

To improve health care in rural areas, especially for increasing numbers of people with chronic diseases, academically qualified nurses could take over expanded roles to meet the challenges of an ageing society and a decreasing number of General Practitioners (GPs). In the project "HandinHand" (HiH), qualified nurses (Expert nurses, ENs) will carry out home visits to older people with chronic diseases over a period of six months. ENs will prepare a care plan in cooperation with GPs to stabilise the care situation and avoid unplanned hospital admissions and GP visits. The process evaluation aims to provide an in-depth analysis of the implementation process and gather important information on barriers and facilitators to the implementation of ENs as a complementary health care structure in primary care, taking into account several context factors.

METHODS

Based on the Medical Research Council (MRC) Framework for complex interventions, a logic model was developed and applied as the basis for data collection. Qualitative and quantitative data will be collected during the study. A mixed methods approach should allow to gain important insights from participants (e.g. ENs, GPs, patients) involved in the study as well as relevant stakeholders. Semi-structured interviews and surveys will be conducted. Data analysis will be based on the logical model, combining qualitative and quantitative data. Qualitative data will be analysed inductively-deductively using qualitative thematic framework analysis.

DISCUSSION

The process evaluation will provide guidance and conclusions on further development and transferability. Of particular interest is the expanded role of ENs in primary care, which has barely been implemented in Germany and can be seen as a precursor to the development of an Advanced Practice Nursing (APN) role in primary care.

Contextual factors influencing advance care planning in home care: process evaluation of the cluster-randomised controlled trial STADPLAN

Background

The STADPLAN study is a cluster-randomised controlled trial including 27 home care services in Germany. It assesses the effect of an advance care planning (ACP) intervention delivered by trained nurses to older care-dependent patients. Patients received two ACP conversations and an information brochure. Nurses were educated through a two-day programme and topic guides structuring the conversations.

Objectives of the process evaluation were to determine: [1] whether the intervention was implemented as planned, [2] which change mechanisms were observed, [3] whether targeted process outcomes were achieved and [4] in which way contextual factors influenced the implementation process.

Methods

The process evaluation is based on a mixed methods approach following the recommendations of the UK-MRC framework for the development and evaluation of complex interventions. Qualitative and quantitative assessments were developed and analysed guided by a logic model comprising intervention, participants, mechanisms of change and context factors. The results of the main trial will be published elsewhere.

Results

Educational programme and topic guides were mostly implemented as planned and resulted in motivation, knowledge, and perceived competencies to facilitate ACP conversations in nurses. Deviances in the performance of ACP conversations indicated patients’ varied individual needs, but also obstacles like reluctance of patients and caregivers to participate actively and time constraints of nurse facilitators. Patients and caregivers reported increased awareness of ACP, planning and other activities indicating that targeted process outcomes could be achieved. The relevance of multifaceted contextual factors acting as barriers or facilitators for the engagement in ACP interventions on the individual, organisational and macro level was evident.

Conclusions

The process evaluation elicits obstacles and achievements of the ACP intervention. The logic model organised a plethora of mixed methods data into a holistic picture of multifaceted results. Nurses as ACP facilitators in home care can fulfil a crucial initiating role based on a trusting relationship with their patients. To support older care-dependent people’s ACP engagement, access should be simplified.

Furthermore, education for nurse facilitators and sufficient resources for service provision are needed. Independent of monetary reimbursement, healthcare providers must respect patients’ choice for or against any ACP intervention.

Ethics and trial registration

Approved by the Ethics Committees of Martin Luther University Halle-Wittenberg (Ref.-No. 2019–045), Carl von Ossietzky University Oldenburg (Ref.-No. 2019–024), and University of Lübeck (Ref.-No. 19–080).

German Clinical Trials Register: DRKS00016886. Registered retrospectively 04/06/2019, first participant included 29/05/2019.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12877-022-03026-2.

The My Strengths Training for Life™ program: Rationale, logic model, and description of a strengths-based intervention for young people experiencing homelessness

Traditionally, UK housing services have focused on providing temporary accommodation, identifying risk factors, and preventing negative outcomes to young people experiencing homelessness. However, deficit approaches may lead young people to becoming dependent on services and face greater marginalization and stigmatization. Meeting long-standing calls to focus more on young people's positive attributes and abilities, the My Strengths Training for Life™ (MST4Life™) program was developed as a community partnership with a large housing service. This paper describes the rationale, logic model, and content of the MST4Life™ program using the TIDieR (Template for Intervention Description and Replication) checklist. MST4Life™ is a strengths-based and experiential psychoeducation intervention for young people aged 16-24 years who are homeless or at risk. Grounded in positive youth development and basic psychological needs theory, its aim is to provide meaningful opportunities for participants to recognize, use, and further develop their mental skills and strengths. In turn, enhancing intentional self-regulation is expected to improve physical, mental, and social health and wellbeing, and support positive transitions to independent living. The potential long-term impacts include a reduction in the number of young people returning as homeless, lower rates of mental illness and mortality, and a cost saving to the public purse.

Repairing boundaries along pathways to tuberculosis case detection: a qualitative synthesis of intervention designs

BACKGROUND

Tuberculosis case-finding interventions often involve several activities to enhance patient pathways, and it is unclear which activity defines the type of case-finding intervention. When conducting studies to identify the most effective case-finding intervention it is important to have a clear understanding of these interventions for meaningful comparisons. This review aimed to construct a systems-based logic model of all pathways to tuberculosis case detection through a synthesis of intervention designs.

METHODS

We identified an existing systematic review on the effectiveness of interventions to increase tuberculosis case detection and updated the search from December 2016 to October 2020. We included randomized controlled trials, as these designs encourage detailed description of interventions. Taking each study in turn, intervention descriptions were read in detail. The texts were analysed qualitatively by constantly comparing emerging codes to construct patient journeys, visualized as logical chains. Actions taken as part of interventions were positioned along patient journeys to theorize the sequence of outcomes. Patient journeys formed the basis of the model, which was refined through discussion.

RESULTS

Based on intervention descriptions from 17 randomized controlled trials, our model distinguishes two care-seeking pathways and four screening pathways. An open invitation to people with tuberculosis symptoms creates care-seeking pathways. On care-seeking pathways, systematic screening can be conducted at general health services, but not at specific TB care services. People invited to tuberculosis services regardless of symptoms follow tuberculosis screening pathways and may be identified with presumptive tuberculosis even if they do not seek care for tuberculosis symptoms. Tuberculosis screening pathways include screening offered to all people accessing care at general health services, screening at a mobile clinic or health facility with open invitation to a whole population or tuberculosis contacts, screening personally offered to a whole population or tuberculosis contacts at home, work or school, and screening offered to people receiving care for human immunodeficiency virus or other clinical risk-group care.

CONCLUSION

This systems-based logic model of tuberculosis case-finding pathways may support standardized terminology, consistency, transparency and improved communication among researchers, policy-makers, health workers and community members when implementing and evaluating interventions to improve tuberculosis case detection.

"We are not stray leaves blowing about in the wind": exploring the impact of Family Wellbeing empowerment research, 1998-2021

BACKGROUND

An Aboriginal-developed empowerment and social and emotional wellbeing program, known as Family Wellbeing (FWB), has been found to strengthen the protective factors that help Indigenous Australians to deal with the legacy of colonisation and intergenerational trauma. This article reviews the research that has accompanied the implementation of the program, over a 23 year period. The aim is to assess the long-term impact of FWB research and identify the key enablers of research impact and the limitations of the impact assessment exercise. This will inform more comprehensive monitoring of research impact into the future.

METHODS

To assess impact, the study took an implementation science approach, incorporating theory of change and service utilisation frameworks, to create a logic model underpinned by Indigenous research principles. A research impact narrative was developed based on mixed methods analysis of publicly available data on: 1) FWB program participation; 2) research program funding; 3) program outcome evaluation (nine studies); and 4) accounts of research utilisation (seven studies).

RESULTS

Starting from a need for research on empowerment identified by research users, an investment of $2.3 million in research activities over 23 years produced a range of research outputs that evidenced social and emotional wellbeing benefits arising from participation in the FWB program. Accounts of research utilisation confirmed the role of research outputs in educating participants about the program, and thus, facilitating more demand (and funding acquisition) for FWB. Overall research contributed to 5,405 recorded participants accessing the intervention. The key enablers of research impact were; 1) the research was user- and community-driven; 2) a long-term mutually beneficial partnership between research users and researchers; 3) the creation of a body of knowledge that demonstrated the impact of the FWB intervention via different research methods; 4) the universality of the FWB approach which led to widespread application.

CONCLUSIONS

The FWB research impact exercise reinforced the view that assessing research impact is best approached as a "wicked problem" for which there are no easy fixes. It requires flexible, open-ended, collaborative learning-by-doing approaches to build the evidence base over time. Steps and approaches that research groups might take to build the research impact knowledge base within their disciplines are discussed.

Longitudinal assessment: A strategy to improve continuing professional certification

Healthcare certification organizations carefully balance a commitment to bring value to their membership through programs that support lifelong learning and professional growth, while protecting the public by ensuring competent certified practitioners. These certifying bodies are challenged with remaining current with their maintenance of certification programs while keeping pace with the growing breadth of knowledge, industry standards and guidelines, innovative advances, and rapid technological gains in testing and assessment. Within the context of process innovation, the National Board of Certification and Recertification for Nurse Anesthetists (NBCRNA) evaluated the current landscape of Longitudinal Assessment (LA) as a potential strategy for the assessment of core knowledge as part of their Continued Professional Certification Program for Certified Registered Nurse Anesthetists. This manuscript details the evaluation of LA using a Logic Model as the tool to scaffold inquiry, a review of LA literature, an environmental scan of current LA programs with identification of LA program elements available, and the results of a LA feasibility study. The findings substantiate that continued professional certification which incorporates a LA strategy can augment lifelong learning, but is not an assessment strategy that can be implemented without thoughtful planning, customization and continuous maintenance.

Supporting evidence-informed policy making in rehabilitation: A logic framework for continuous improvement of rehabilitation programs

Because of important health and population trends, rehabilitation is a key health strategy of the 21^st century, and efforts are being made towards a broader integration of rehabilitation into health systems worldwide. In the scope of the call for action Rehabilitation 2030, WHO recommends a logic model approach for monitoring and evaluating national efforts at the systems level. In this paper we argue that as strengthening rehabilitation in health systems will require countries to implement a range of new programs, sound logic models designed to monitor and evaluate rehabilitation programs at service level are needed as well. We therefore propose in this paper an overarching logic model for monitoring and evaluating rehabilitation programs, including feasible indicators, options for their operationalization and considerations about data analyses. Our overarching logic model for monitoring and evaluation at service level is suitable to structure national clinical quality management for continuous improvement, to inform policy making and programming at the national level, and to serve as a reference framework for research. The model will serve as a starting point for developing other logic models that fit specific features of rehabilitation programs in diverse settings and countries.

Understanding the effectiveness and underlying mechanisms of lifestyle modification interventions in adults with learning disabilities: protocol for a mixed-methods systematic review

BACKGROUND

Adults with learning disabilities have an increased disposition to unhealthy lifestyle behaviours which often occur simultaneously. Existing studies focus on complex interventions targeting unhealthy diet, physical inactivity, sedentary behaviour, smoking, and alcohol use to reduce health risks experienced. It is essential to understand how well these interventions work, what works, for whom, in what context and why. This study aims to investigate the effectiveness and underlying mechanisms of lifestyle modification interventions for adults with learning disabilities.

METHODS

This is a mixed-methods systematic review consisting of a network meta-analysis (NMA) and realist synthesis. Electronic databases (ASSIA, CINAHL, EMBASE, MEDLINE, and PsycINFO) will be searched from inception to 14 January 2021 with no language restriction. Additionally, trial registries, grey literature databases and references lists will be searched. Studies related to lifestyle modification interventions on the adult population (>18 years) with learning disabilities will be eligible for inclusion. Two independent researchers will screen studies, extract data and assess its quality and risk of bias using the Cochrane Collaboration's Risk of Bias Assessment Tool (RoB Version 2) and ROBINS-I. The strength of the body of evidence will be assessed based on the GRADE approach. The NMA will incorporate results from RCTs and quasi-experimental studies to estimate the effectiveness of various lifestyle interventions. Where appropriate, a component NMA (CNMA) will be used to estimate effectiveness. The realist synthesis will complement and explain the findings of NMA and CNMA by including additional qualitative and mixed-methods studies. Studies will be included based on their relevance to the programme theory and the rigour of their methods, as determined by quality appraisal tools appropriate to the study design. Results from both syntheses will be incorporated into a logic model.

DISCUSSION

The paucity of population-specific lifestyle interventions contributes to the challenges of behaviour change in adults with learning disabilities. This study will provide an evidence-base from which various stakeholders can develop effective interventions for adults with learning disabilities. The evidence will also help prioritise and inform research recommendations for future primary research so that people with learning disabilities live happier, healthier and longer lives.

TRIAL REGISTRATION

PROSPERO CRD 42020223290.

Evaluation plan of the 6for6 research skills program for rural and remote physicians

Overwhelming issues and barriers often prevent rural and remote physicians (RRPs) from pursuing the many socially accountable research questions they encounter on a daily basis. Although research training programs can empower RRPs to rise to these challenges, there is a lack of evidence on how they should be developed and refined. At Memorial University, a faculty development program (FDP) called 6for6 has been helping RRPs surmount their research quagmires and engage in scholarship since 2014. After an initial three-year (2014-17) pilot, we prepared a detailed plan to evaluate the 6for6 research FDP for RRPs and inform future years of delivery. Using a modified Delphi method and participatory action model a group of program team members, stakeholders and evaluation experts developed an evaluation plan including a logic model and an evaluation matrix addressing five key themes. To our knowledge, this is the first evaluation plan for a research-focused FDP targeting RRPs. While this plan was developed specifically for the 6for6 FDP, our approach to its development may be useful to any institution interested in evaluating an FDP with limited resources.

Documenting HIV research-utilization activities, outputs and outcomes: examples and lessons learned from Project SOAR

The importance of using research findings to inform policy and program decisions is well recognized, but the literature on measuring research utilization activities is scarce. As funding to support some areas of research wanes or remains stagnant, the need to document the value of investing in research by its’ effect on improved programs and policies becomes increasingly necessary. We present the experience of Project SOAR, a six-year USAID-funded project focusing on HIV/AIDS-related implementation research, to demonstrate measurement of research utilization. We follow the project’s research-utilization logic model, including inputs, activities, outputs, and outcomes. We present tools the project developed and examples from project studies and discuss what works, remaining challenges and how to overcome them, and lessons learned. We then make recommendations for incorporating research-utilization activities and measurement in implementation-research studies.

The theoretical basis of a nationally implemented type 2 diabetes prevention programme: how is the programme expected to produce changes in behaviour?

BACKGROUND

It is considered best practice to provide clear theoretical descriptions of how behaviour change interventions should produce changes in behaviour. Commissioners of the National Health Service Diabetes Prevention Programme (NHS-DPP) specified that the four independent provider organisations must explicitly describe the behaviour change theory underpinning their interventions. The nationally implemented programme, launched in 2016, aims to prevent progression to Type 2 diabetes in high-risk adults through changing diet and physical activity behaviours. This study aimed to: (a) develop a logic model describing how the NHS-DPP is expected to work, and (b) document the behaviour change theories underpinning providers' NHS-DPP interventions.

METHODS

A logic model detailing how the programme should work in changing diet and activity behaviours was extracted from information in three specification documents underpinning the NHS-DPP. To establish how each of the four providers expected their interventions to produce behavioural changes, information was extracted from their programme plans, staff training materials, and audio-recorded observations of mandatory staff training courses attended in 2018. All materials were coded using Michie and Prestwich's Theory Coding Scheme.

RESULTS

The NHS-DPP logic model included information provision to lead to behaviour change intentions, followed by a self-regulatory cycle including action planning and monitoring behaviour. None of the providers described an explicit logic model of how their programme will produce behavioural changes. Two providers stated their programmes were informed by the COM-B (Capability Opportunity Motivation - Behaviour) framework, the other two described targeting factors from multiple theories such as Self-Regulation Theory and Self-Determination Theory. All providers cited examples of proposed links between some theoretical constructs and behaviour change techniques (BCTs), but none linked all BCTs to specified constructs. Some discrepancies were noted between the theory described in providers' programme plans and theory described in staff training.

CONCLUSIONS

A variety of behaviour change theories were used by each provider. This may explain the variation between providers in BCTs selected in intervention design, and the mismatch between theory described in providers' programme plans and staff training. Without a logic model describing how they expect their interventions to work, justification for intervention contents in providers' programmes is not clear.

A Contextual Analysis and Logic Model for Integrated Care for Frail Older Adults Living at Home: The INSPIRE Project

INTRODUCTION

Implementation science methods and a theory-driven approach can enhance the understanding of whether, how, and why integrated care for frail older adults is successful in practice. In this study, we aimed to perform a contextual analysis, develop a logic model, and select preliminary implementation strategies for an integrated care model in newly created information and advice centers for older adults in Canton Basel-Landschaft, Switzerland.

METHODS

We conducted a contextual analysis to determine factors which may influence the integrated care model and implementation strategies needed. A logic model depicting the overall program theory, including inputs, core components, outputs and outcomes, was designed using a deductive approach, and included stakeholders' feedback and preliminary implementation strategies.

RESULTS

Contextual factors were identified (e.g., lack of integrated care regulations, existing community services, and a care pathway needed). Core components of the care model include screening, referral, assessment, care plan creation and coordination, and follow-up. Outcomes included person-centred coordinated care experiences, hospitalization rate and symptom burden, among others. Implementation strategies (e.g., nurse training and co-developing educational materials) were proposed to facilitate care model adoption.

CONCLUSION

Contextual understanding and a clear logic model should enhance the potential for successful implementation of the integrated care model.

Using a logic model to develop an intervention for improving miscarriage care in the emergency department

Miscarriage is the most common pregnancy-related complication and is frequently treated at the emergency department (ED). Parents have expressed dissatisfaction with the care they received at the ED and few interventions have been developed for improving miscarriage care. When planning an intervention, it is crucial to develop a program theory specifying what must be done for an intervention to achieve its objectives. The purpose of this paper is to describe the logic model process for developing an intervention intended to improve parents' miscarriage experience at the ED. The six steps of W. K. Kellogg Foundation (2004) theory logic model were used to 1) describe the problem; 2) conduct a needs assessment; and to identify 3) expected results, 4) influential factors, 5) intervention strategies, and 6) assumptions related to change strategies. A community-based participatory approach was used. It included two planning groups: parents who had visited the ED for a miscarriage (N = 9) and health professionals (N = 8). The theory logic model provided a rigorous framework for intervention development based on theories, scientific evidence, and the experiences of parents and health professionals. Detailed description of the intervention should facilitate its implementation, evaluation, and replication for other health problems.

Co-creating physical activity interventions: a mixed methods evaluation approach

BACKGROUND

Co-creation strategies, such as cooperative planning, are promising as a means to ensure that physical activity interventions address real-world problems and are tailored to the target group. This has already been validated in diverse settings. However, questions targeting the transferability of cooperative planning to new settings and the key factors influencing its success or failure remain unclear. At the same time, co-creation processes are complex, and evaluation can be challenging. Following calls for detailed reporting, this paper describes the programme activities, the underlying logic, and methodological design of a study that aims to evaluate the transfer of cooperative planning to new settings and to explore the associated key determinants.

METHODS

Cooperative planning was utilized as a strategy to target physical activity promotion in three real-world German settings in the nursing care and automotive mechatronics sectors. This involved researchers working alongside stakeholders from practice and policy to conjointly develop new interventions to promote physical activity in physically demanding jobs. A pragmatic approach is used to evaluate both the transferability and key determinants of this strategy. We developed a logic model for this co-creation process that describes the underlying assumptions and guides the evaluation. The evaluation outcomes of this study include planning meetings, newly developed interventions, and the determinants that are likely to affect cooperative planning. Quantitative and qualitative data will be collected using questionnaires, documents, and interviews. The quantitative data will be analysed descriptively, while the qualitative data will mainly be analysed using qualitative content analysis, split by settings. Subsequently, data triangulation will be used to integrate the quantitative and qualitative findings, which will then be compared across all three settings.

DISCUSSION

The study findings will contribute to a better understanding of co-creation strategies, their transferability, and key determinants. The practical implications can include a checklist for assessing key determinants and a guideline for transferring cooperative planning into new settings to benefit more people. Ultimately, this study will help to advance co-creation strategies and may be relevant for researchers, practitioners, and policy-makers targeting physical activity promotion in various contexts.

TRIAL REGISTRATION

Open Science Framework: https://osf.io/r6xnt/ (retrospectively registered).

Visualizing the drivers of an effective health workforce: a detailed, interactive logic model

BACKGROUND

A strong health workforce is a key building block of a well-functioning health system. To achieve health systems goals, policymakers need information on what works to improve and sustain health workforce performance. Most frameworks on health workforce planning and policymaking are high-level and conceptual, and do not provide a structure for synthesizing the growing body of empirical literature on the effectiveness of strategies to strengthen human resources for health (HRH). Our aim is to create a detailed, interactive logic model to map HRH evidence and inform policy development and decision-making.

METHODS

We reviewed existing conceptual frameworks and models on health workforce planning and policymaking. We included frameworks that were: (1) visual, (2) comprehensive (not concentrated on specific outcomes or strategies), and (3) designed to support decision-making. We compared and synthesized the frameworks to develop a detailed logic model and interactive evidence visualization tool.

RESULTS

Ten frameworks met our inclusion criteria. The resulting logic model, available at hrhvisualizer.org , allows for visualization of high-level linkages as well as a detailed understanding of the factors that affect health workforce outcomes. HRH data and governance systems interact with the context to affect how human resource policies are formulated and implemented. These policies affect HRH processes and strategies that influence health workforce outcomes and contribute to the overarching health systems goals of clinical quality, responsiveness, efficiency, and coverage. Unlike existing conceptual frameworks, this logic model has been operationalized in a highly visual, interactive platform that can be used to map the research informing policies and illuminating their underlying mechanisms.

CONCLUSIONS

The interactive logic model presented in this paper will allow for comprehensive mapping of literature around effective strategies to strengthen HRH. It can aid researchers in communicating with policymakers about the evidence behind policy questions, thus supporting the translation of evidence to policy.

Qualitative process evaluation of the Fostering Changes program for foster carers as part of the Confidence in Care randomized controlled trial

BACKGROUND

Fostering Changes is an in-service training program for foster carers designed to enhance carer skills, coping strategies and carer-child relationships. The training program has been evaluated in a randomised controlled trial comparing Fostering Changes to usual care.

OBJECTIVE

To conduct a qualitative process evaluation drawing on stakeholder perspectives to describe the logic model of Fostering Changes, identify potential mechanisms of impact of the program and enhance understanding of the trial results.

PARTICIPANTS AND SETTING

Participants were stakeholders in the Fostering Changes program delivered in Wales, UK including foster carers invited to attend the program (18 attendees, eight non-attendees), two program developers, five trainers, 12 social workers who attended or recruited to the program.

METHODS

Total population sampling with qualitative data collection methods. Qualitative data were subject to thematic analysis.

RESULTS

A logic model summarising the program resources, activities and anticipated outcomes was generated. Implementation themes were quality of training, setting and group composition. Mechanisms of impact were identified with themes falling into two categories, group process and skills development. Potential barriers to effectiveness included a poor fit between the carer needs and the program in relation to levels of challenge being faced, age-appropriate content and responsiveness. Contextual factors were also relevant, including the existing relationship between foster carers and the agency and the perceived value of training.

CONCLUSIONS

Although the group aspects of the program were well received, the program itself did not help foster carers deal with more complex challenges and needed to be more targeted in terms of carers needs and circumstances.

An innovative teaching model in an academic-practice partnership for a Doctor of Nursing Practice program

Teaching innovations in Doctor of Nursing Practice (DNP) programs are essential for preparing practitioners for role responsibilities. The purpose of this paper is to describe a teaching model implemented in a public-private academic-practice partnership in which DNP-prepared healthcare organization nursing leaders joined with college of nursing faculty to teach didactic courses in the DNP program. The conceptual framework for this model is organized around Boyer's (1990) principles of the scholarship of teaching, integration, and application, and the American Association of Colleges of Nursing definition of scholarship (2018). A logic model evaluation plan of the teaching model includes a description of inputs, activities, outputs, and outcomes. Outcomes include the attainment of short term goals related to student satisfaction and DNP program completion; faculty and peer satisfaction; and administrator perspectives. Future evaluation will include assessment of long-term impact of the teaching model. The teaching model can be replicated in preparing cohorts of students for advanced practice nursing and cultivating the scholarship of teaching, integration and application.

When complexity matters: a step-by-step guide to incorporating a complexity perspective in guideline development for public health and health system interventions

BACKGROUND

Guidelines on public health and health system interventions often involve considerations beyond effectiveness and safety to account for the impact that these interventions have on the wider systems in which they are implemented. This paper describes how a complexity perspective may be adopted in guideline development to facilitate a more nuanced consideration of a range of factors pertinent to decisions regarding public health and health system interventions. These factors include acceptability and feasibility, and societal, economic, and equity and equality implications of interventions.

MAIN MESSAGE

A 5-step process describes how to incorporate a complexity perspective in guideline development with examples to illustrate each step. The steps include: (i) guideline scoping, (ii) formulating questions, (iii) retrieving and synthesising evidence, (iv) assessing the evidence, and (v) developing recommendations. Guideline scoping using stakeholder consultations, complexity features, evidence mapping, logic modelling, and explicit decision criteria is emphasised as a key step that informs all subsequent steps.

CONCLUSIONS

Through explicit consideration of a range of factors and enhanced understanding of the specific circumstances in which interventions work, a complexity perspective can yield guidelines with better informed recommendations and facilitate local adaptation and implementation. Further work will need to look into the methods of collecting and assessing different types of evidence beyond effectiveness and develop procedural guidance for prioritising across a range of decision criteria.

Process evaluation of a complex intervention to promote advance care planning in community-dwelling older persons (the STADPLAN study)-study protocol

BACKGROUND

Process evaluation addresses the implementation, mechanisms of impact, and context of participants in complex interventions. The STADPLAN study assesses the effects of conversations on advance care planning (ACP) led by trained nurse facilitators. The complex intervention consists of several components that may lead to various changes in attitude and behavior regarding personal ACP activities. With the process evaluation, we aim to assess how changes were achieved in the STADPLAN intervention.

METHODS

The planned process evaluation study will be conducted alongside a cluster-randomized controlled trial on ACP in home care services (HCS). Trained nurse facilitators will deliver the ACP intervention consisting of an information brochure and two ACP conversations. A logic model depicts the assumed change processes of the intervention: the educational program enables nurses to conduct ACP conversations with patients and their caregivers. Patients gain knowledge and reflect upon and engage in their own ACP. Caregivers better understand patients' wishes and feel reassured in their role as surrogates. Designation of a surrogate and communication on ACP are facilitated. We will assess the effects of the educational program with questionnaires and a focus group including all participating nurses. We will measure ACP engagement, and prevalence of advance directives in patients, and ask for their experiences with the intervention. We will conduct semi-structured interviews with caregivers about their expectations and experiences regarding ACP in general and the intervention. We will address context factors, e.g., basic characteristics of the HCS (such as ownership, number of clients, staff and qualification). Analysis will be based upon the logic model, integrating qualitative and quantitative data.

DISCUSSION

The comprehensive process evaluation will provide essential information on the feasibility of implementation strategies and the clinical relevance of a nurse-led ACP intervention in home care recipients and its generalizability and transferability to other settings.

TRIAL REGISTRATION

German Clinical Trials Register: DRKS00016886 . Registered retrospectively on June 4, 2019, first participant included on May 29, 2019.

A randomised controlled trial of hearing and vision support in dementia: Protocol for a process evaluation in the SENSE-Cog trial

BACKGROUND

Optimising hearing and vision function may be important in improving a range of outcomes for people living with dementia (PwD) and their companions. The SENSE-Cog cross-national randomised controlled trial (RCT) is evaluating the effectiveness of a sensory intervention (SI) to improve quality of life for PwD with concurrent hearing and/or vision impairment, in five European countries. To ascertain how or why the intervention will, or will not, achieve its outcomes, we have designed a process evaluation to explore potential discrepancies between expected and observed outcomes. This will also help us to understand how context may influence the outcomes. Here we describe the protocol for this process evaluation, which is embedded within the RCT.

METHODS/DESIGN

We will use a mixed methods approach with a theoretical framework derived from the UK Medical Research Council's' guidance on process evaluations. It will include the following: (1) evaluating how key aspects of the intervention will be delivered, which will be important to scale the intervention in real world populations; (2) characterising the contextual issues, which may shape the delivery and the impact of the intervention in different countries; and (3) investigating possible causal mechanisms through analyses of potential moderators and mediators. To avoid bias, we will analyse the process data before the analysis of the main effectiveness outcomes.

DISCUSSION

This evaluation will provide insight into how the complex SENSE-Cog SI will be tailored, enacted and received across the different European contexts, all of which have unique health and social care economies. The findings will provide insight into the causal mechanisms effecting change, and will determine whether we should implement the intervention, if effective, on a wider scale for PwD and concurrent sensory impairment.

TRIAL REGISTRATION

ISRCTN, ISRCTN17056211. Registered on 19 February 2018.

Recruitment settings, delivery contexts, intervention techniques and outcomes of health promotion programmes for young adults with intellectual and developmental disabilities: A systematic review

BACKGROUND

People with intellectual and developmental disabilities (IDD) are at risk of developing long term health conditions, and a preventative health agenda research is emerging. However, little is known about the recruitment settings, delivery contexts, intervention techniques and outcomes of health promotion programmes for this population. Therefore, the aim of this review was to synthesize and evaluate these characteristics.

METHOD

A systematic review of studies identified from multiple databases on healthy lifestyle interventions for adolescents and young people with IDD was conducted. Data were synthesized and evaluated using a logic model. Quality of rigour was also assessed.

RESULTS

Sixteen geographically diverse studies were selected and evaluated. Participants were most commonly recruited from schools, with interventions typically taking place in a gym setting and involving physical activity training.

CONCLUSIONS

This review indicates that physical activity and dietary interventions in people with IDD may lead to lifestyle changes, however more robust evidence is required. Educational settings are conducive, with settings beyond schools requiring further consideration.

STructured lifestyle education for people WIth SchizophrEnia (STEPWISE): mixed methods process evaluation of a group-based lifestyle education programme to support weight loss in people with schizophrenia

BACKGROUND

STEPWISE is a theory-informed self-management education programme that was co-produced with service users, healthcare professionals and interventionists to support weight loss for people with schizophrenia. We report the process evaluation to inform understanding about the intervention and its effectiveness in a randomised controlled trial (RCT) that evaluated its efficacy.

METHODS

Following the UK Medical Research Council (MRC) Guidelines for developing and evaluating complex interventions, we explored implementation quality. We considered causal mechanisms, unanticipated consequences and contextual factors associated with variation in actual and intended outcomes, and integrated treatment fidelity, using the programme theory and a pipeline logic model. We followed a modified version of Linnan and Steckler's framework and single case design. Qualitative data from semi-structured telephone interviews with service-users (n = 24), healthcare professionals delivering the intervention (n = 20) and interventionists (n = 7) were triangulated with quantitative process and RCT outcome data and with observations by interventionists, to examine convergence within logic model components.

RESULTS

Training and course materials were available although lacked co-ordination in some trusts. Healthcare professionals gained knowledge and some contemplated changing their practice to reflect the (facilitative) 'style' of delivery. They were often responsible for administrative activities increasing the burden of delivery. Healthcare professionals recognised the need to address antipsychotic-induced weight gain and reported potential value from the intervention (subject to the RCT results). However, some doubted senior management commitment and sustainability post-trial. Service-users found the intervention highly acceptable, especially being in a group of people with similar experiences. Service-users perceived weight loss and lifestyle benefits; however, session attendance varied with 23% (n = 47) attending all group-sessions and 17% (n = 36) attending none. Service-users who lost weight wanted closer monitoring and many healthcare professionals wanted to monitor outcomes (e.g. weight) but it was outside the intervention design. No clinical or cost benefit was demonstrated from the intermediate outcomes (RCT) and any changes in RCT outcomes were not due to the intervention.

CONCLUSIONS

This process evaluation provides a greater understanding of why STEPWISE was unsuccessful in promoting weight loss during the clinical trial. Further research is required to evaluate whether different levels of contact and objective monitoring can support people with schizophrenia to lose weight.

TRIAL REGISTRATION

ISRCTN, ISRCTN19447796. Registered 20 March 2014.

Development of a logic model for a service learning, dog walking course for college students

Despite overwhelming evidence of the importance of physical activity, only 20% of US adults meet physical activity recommendations. A subsector of the population vulnerable to inactivity is college students. There is a disconnect between evidence-based physical activity programming and dissemination into college communities, which may be overcome through logic model development and evaluation. The purpose of this paper is to discuss logic model development for a service-learning, dog walking program for undergraduates. Participants were undergraduates (18-25 years). Process evaluations and observational data were used in a content analysis to determine major themes to create the logic model. A set of resources were needed to complete program activities, which led to short- and long-term outcomes. The overarching goal of the program is to improve the overall health status of college students and companion animals. This logic model may have significant public health impact as it can serve as a framework for universities wanting to develop, implement, and evaluate a service-learning, dog walking program for students.

A logic model development for an adolescent based intervention to improve benefits from Therapeutic Residential Care (TRC)

There is a significant need to comprehensively describe and illustrate via a logic model what processes work for adolescents in residential treatment facilities and how to make improvements (Bean, White, Neagle & Lake, 2005).The purpose of this article is to highlight one Adolescent TRC's journey to develop and implement a working Logic Model.

Using logic model and visualization to conduct portfolio evaluation

Portfolio evaluation is the evaluation of multiple projects with a common purpose. While logic models have been used in many ways to support evaluation, and data visualization has been used widely to present and communicate evaluation findings, adopting logic models for portfolio evaluation and using data visualization to share findings simultaneously is surprisingly limited in the literature. With the data from a sample portfolio of 209 projects which aims to improve the system of early care and education (ECE), this study illustrated how to use logic model and data visualization techniques to conduct a portfolio evaluation by answering two evaluation questions: "To what extent are the elements of a logic model (strategies, sub-strategies, activities, outcomes, and impacts) reflected in the sample portfolio?" and "Which dominant paths through the logic model were illuminated by the data visualization technique?" For the first question, the visualization technique illuminated several dominant strategies, sub-strategies, activities, and outcomes. For the second question, our visualization techniques made it convenient to identify critical paths through the logic model. Implications for both program evaluation and program planning were discussed.

Dealing with context in logic model development: Reflections from a realist evaluation of a community health worker programme in Nigeria

Logic models (LMs) have been used in programme evaluation for over four decades. Current debate questions the ability of logic modelling techniques to incorporate contextual factors into logic models. We share experience of developing a logic model within an ongoing realist evaluation which assesses the extent to which, and under what circumstances a community health workers (CHW) programme promotes access to maternity services in Nigeria. The article contributes to logic modelling debate by: i) reflecting on how other scholars captured context during LM development in theory-driven evaluations; and ii) explaining how we explored context during logic model development for realist evaluation of the CHW programme in Nigeria. Data collection methods that informed our logic model development included documents review, email discussions and teleconferences with programme stakeholders and a technical workshop with researchers to clarify programme goals and untangle relationships among programme elements. One of the most important findings is that, rather than being an end in itself, logic model development is an essential step for identifying initial hypotheses for tentative relevant contexts, mechanisms and outcomes (CMOs) and CMO configurations of how programmes produce change. The logic model also informed development of a methodology handbook that is guiding verification and consolidation of underlying programme theories.

Using an intervention mapping approach to develop prevention and rehabilitation strategies for musculoskeletal pain among surgeons

BACKGROUND

The aim of the present study was to describe the development of strategies to prevent and rehabilitate musculoskeletal pain among surgeons. Musculoskeletal pain affects surgeons' life, and evidence on interventions for effective prevention and rehabilitation is lacking for this occupational group.

METHODS

An Intervention Mapping approach was used to develop intervention strategies specifically tailored to surgeons. This approach entailed conducting a systematic scoping literature search and semi-structured interviews with six surgeons.

RESULTS

The first step was to develop a logic model of the problem of musculoskeletal pain among surgeons. Step two was to formulate health-enhancing outcomes and performance objectives for the intervention, while in step three theory-based methods and practical strategies for the intervention were identified.

CONCLUSION

The present Intervention Mapping study demonstrated that musculoskeletal pain among surgeons is a complex area that needs attention. Our findings highlight a need for individual behavioural changes as well as organisational, attitudinal, and management changes.

It's just the right thing to do: Conceptualizing a theory of change for a school food and beverage sales environment interv ention and implications for implementation evaluation

School food environments are the target of nutrition interventions and evaluations across the globe. Yet little work to-date has articulated the importance of developing a theory of change upon which to base evaluation of both implementation and outcomes. This paper undertakes an interpretive approach to develop a retrospective theory of change for an implementation evaluation of British Columbia's school food and beverage sales Guidelines. This study contributes broadly to a nuanced conceptualization of this type of public health intervention and provides a methodological contribution on how to develop a retrospective theory of change with implications for effective evaluation. Data collection strategies included document analysis, semi-structured interviews with key stakeholders, and participant observation. Developing the logic model revealed that, despite the broad population health aims of the intervention, the main focus of implementation is to change behaviors of adults who create school food environments. Derived from the analysis and interpretation of the data, the emergent program theory focuses on the assumption that if adults are responsibilized through information and education campaigns and provided implementation tools, they will be 'convinced' to implement changes to school food environments to foster broader public health goals. These findings highlight the importance of assessing individual-level implementation indicators as well as the more often evaluated measures of food and beverage availability.

An intervention to support stroke survivors and their carers in the longer term (LoTS2Care): study protocol for the process evaluation of a cluster randomised controlled feasibility trial

Background

Whilst pathways relating to the early stages of stroke care have become well established, strategies for longer-term care are less developed and longer-term outcomes remain poor for many stroke survivors. New Start, a complex intervention that includes needs identification, exploration of social networks and components of problem-solving and self-management, was designed to improve stroke survivors’ quality of life by addressing unmet needs and increasing participation. It is delivered approximately 6 months post-stroke by trained staff (facilitators). We are currently undertaking a cluster randomised feasibility trial of the New Start intervention with an embedded process evaluation, which is an important component of the design and testing of complex interventions as it provides an understanding of how interventions are delivered and function in different settings.

Methods/design

This mixed methods process evaluation will explore the degree to which New Start is implemented as intended, the impact of context on intervention delivery and the acceptability of the intervention for stroke survivors, their families and practitioners. It will include non-participant observation of facilitator training and intervention delivery. Interviews with stroke survivors, facilitators and other relevant staff (including administrators and managerial staff) will be undertaken. Qualitative data from interview transcripts, facilitator reflections and observational field notes will be analysed thematically alongside numerical data documenting intervention delivery collected as part of the trial.

Discussion

This process evaluation will identify factors that aid and impede implementation of the New Start intervention and improve understanding of this novel approach to longer-term stroke care.

Trial registration

ISRCTN Registry, ISRCTN38920246. Registered on 22 June 2016.

Systematic iteration between model and methodology: A proposed approach to evaluating unintended consequences

This article argues that evaluators could better deal with unintended consequences if they improved their methods of systematically and methodically combining empirical data collection and model building over the life cycle of an evaluation. This process would be helpful because it can increase the timespan from when the need for a change in methodology is first suspected to the time when the new element of the methodology is operational. The article begins with an explanation of why logic models are so important in evaluation, and why the utility of models is limited if they are not continually revised based on empirical evaluation data. It sets the argument within the larger context of the value and limitations of models in the scientific enterprise. Following will be a discussion of various issues that are relevant to model development and revision. What is the relevance of complex system behavior for understanding predictable and unpredictable unintended consequences, and the methods needed to deal with them? How might understanding of unintended consequences be improved with an appreciation of generic patterns of change that are independent of any particular program or change effort? What are the social and organizational dynamics that make it rational and adaptive to design programs around single-outcome solutions to multi-dimensional problems? How does cognitive bias affect our ability to identify likely program outcomes? Why is it hard to discern change as a result of programs being embedded in multi-component, continually fluctuating, settings? The last part of the paper outlines a process for actualizing systematic iteration between model and methodology, and concludes with a set of research questions that speak to how the model/data process can be made efficient and effective.

Integrated children's clinic care (ICCC) versus a self-directed care pathway for children with a chronic health condition: a multi-centre randomised controlled trial study protocol

Background

Children with chronic health conditions have better health-related outcomes when their care is managed in a personalised and coordinated way. However, increased demand on Australian ambulatory care hospital services has led to longer waitlist times to access specialists and appropriate intervention services; placing vulnerable children at increased risk of poorer short-term (e.g. social difficulties) and long-term (e.g. convictions) health and social outcomes. Traditional approaches to increasing frequency and service of delivery are expensive and can have minimal impact on caregiver burden. A community based service-integration approach, rather than self-directed care is proposed as increased service linkages are more likely to occur and improve the health outcomes of children with a chronic health condition.

Methods

An open, unblinded, multi-centre randomised controlled trial in two Australian public hospitals. 112 children (0–16 years) fulfilling the inclusion criteria will be randomised to one of two clinical pathways for management of their chronic health condition: (1) integrated children’s care clinic (ICCC) or (2) self-directed care pathway. All children and caregivers will be interviewed at 1 week, and 3, 6 and 12 month time intervals. Primary outcome measures include the Pediatric Quality of Life (PedQOL) questionnaire, Subjective Units of Distress Scale, Child Behaviour Checklist (CBCL) and Rotter’s Locus of Control Scale. Secondary outcome measures include the total number of medical appointments, school days missed and quantity of services accessed. Our main objectives are to determine if the ICCC results in better health and economics outcomes compared to the self-directed care pathway.

Discussion

The success of a health systems approach needs to be balanced against clinical, mortality and cost-effectiveness data for long-term sustainability within a publicly funded health system. A clinical pathway that is sustainable, cost-effective, provides efficient evidence-based care and improves the quality of life outcomes for children with chronic health conditions has the potential to reduce waitlist times, improve access to health services, increase consumer satisfaction; and prevent costs associated with poorly managed chronic health conditions into adulthood. This study will be the first to provide clinical and health economics data on an integrated care pathway for the management of chronic health conditions in children. On a broader scale, results from this study will help guide care coordination frameworks for children with chronic health conditions; particularly with the introduction and implementation of a National Disability Insurance Scheme (NDIS) across Australia.

Trial registration

Australia and New Zealand Clinical Trials Register (ANZCTR) ACTRN12617001188325. Registered: 14th August, 2017.

Electronic supplementary material

The online version of this article (10.1186/s12887-018-1034-x) contains supplementary material, which is available to authorized users.

Evaluating a dementia learning community: exploratory study and research implications

Background

Access times for, the costs and overload of hospital services are an increasingly salient issue for healthcare managers in many countries. Rising demand for hospital care has been attributed partly to unplanned admissions for older people, and among these partly to the increasing prevalence of dementia. The paper makes a preliminary evaluation of the logic model of a Dementia Learning Community (DLC) intended to reduce unplanned hospital admissions from care homes of people with dementia. A dementia champion in each DLC care home trained other staff in dementia awareness and change management with the aims of changing work routines, improving quality of life, and reducing demands on external services.

Methods

Controlled mixed methods realistic evaluation comparing 13 intervention homes with 10 controls in England during 2013–15. Each link in the assumed logic model was tested to find whether that link appeared to exist in the DLC sites, and if so whether its effects appeared greater there than in control sites, in terms of selected indicators of quality of life (DCM Well/Ill-Being, QUALID, end-of-life planning); and impacts on ambulance call-outs and hospital admissions.

Results

The training was implemented as planned, and triggered cycles of Plan-Do-Study-Act activity in all the intervention care homes. Residents’ well-being scores, measured by dementia care mapping, improved markedly in half of the intervention homes but not in the other half, where indeed some scores deteriorated markedly. Most other care quality indicators studied did not significantly improve during the study period. Neither did ambulance call-out or emergency hospital admission rates.

Conclusions

PDSA cycles appeared to be the more ‘active ingredient’ in this intervention. The reasons why they impacted on well-being in half of the intervention sites, and not the others, require further research. A larger, longer study would be necessary to measure definitively any impacts on unplanned hospital admissions. Our evidence suggested revising the DLC logic model to include care planning and staff familiarisation with residents’ personal histories and needs as steps towards improving residents’ quality of life.

Devil is in the details: Using logic models to investigate program process

Theory-based logic models are commonly developed as part of requirements for grant funding. As a tool to communicate complex social programs, theory based logic models are an effective visual communication. However, after initial development, theory based logic models are often abandoned and remain in their initial form despite changes in the program process. This paper examines the potential benefits of committing time and resources to revising the initial theory driven logic model and developing detailed logic models that describe key activities to accurately reflect the program and assist in effective program management. The authors use a funded special education teacher preparation program to exemplify the utility of drill down logic models. The paper concludes with lessons learned from the iterative revision process and suggests how the process can lead to more flexible and calibrated program management.

Ultrasound and the Complete Urologist Part 2: Outcomes

Outcome studies help establish the benefits of tests or procedures that can change the delivery of medical care. Through this discourse, the current status of outcomes with respect to the use of ultrasound imaging for AMH as well as its general use during a routine urologic physical examination are explored. The changes in today's health care environment focus on cost reduction and outcomes that produce value in the context of patient satisfaction. The question is whether we have to wait for outcome results (that can take a lengthy period of time to achieve) before the benefits produced by ultrasound can be applied. This report reviews the literature and the role ultrasound can play in today's urologic health care environment. Supportive evidence demonstrates the value that ultrasound use brings to the urologic examining table for AMH and to daily urologic practice to the benefit of both patient and physician.

Towards a Comprehensive Conceptual Framework of Active Travel Behavior: a Review and Synthesis of Published Frameworks

PURPOSE OF REVIEW

This paper reviews the use of conceptual frameworks in research on active travel, such as walking and cycling. Generic framework features and a wide range of contents are identified and synthesized into a comprehensive framework of active travel behavior, as part of the Physical Activity through Sustainable Transport Approaches project (PASTA). PASTA is a European multinational, interdisciplinary research project on active travel and health.

RECENT FINDINGS

Along with an exponential growth in active travel research, a growing number of conceptual frameworks has been published since the early 2000s. Earlier frameworks are simpler and emphasize the distinction of environmental vs. individual factors, while more recently several studies have integrated travel behavior theories more thoroughly. Based on the reviewed frameworks and various behavioral theories, we propose the comprehensive PASTA conceptual framework of active travel behavior. We discuss how it can guide future research, such as data collection, data analysis, and modeling of active travel behavior, and present some examples from the PASTA project.

Hierarchical Logic Models as a Tool to Evaluate Programmatic Initiatives: Practical Solutions to Identified Problems

Introduction

Large programmatic grants advance the missions of funding agencies or organizations. This article describes the programmatic impact of using “hierarchical” logic models in two Centers funded by the National Institute of Occupational Safety and Health (NIOSH) that were designed to achieve NIOSH goals. Such models are supportive of priority setting, policy implementation, and effective evaluation.

Methods

Two NIOSH Centers, an Agricultural Center and an Occupational Safety and Health Education and Research Center, used the same hierarchical logic model process to support the NIOSH programmatic goal of improving worker health and safety in their respective occupational categories. The logic model development processes were led by the same evaluator.

Results

Case studies describe the utilization of “hierarchical” logic models: in each case, NIOSH was the “grandparent”, the Center was its descendant (parent) and the cores were the children. This lineage was articulated through the Center-wide logic model and through the logic model of each of its core programmatic areas (core). The Center-wide logic model ensured that the Center’s goals, and the intended outcomes and impact of its work were linked to the mission and goals of NIOSH. Each core’s logic model articulated how its goals, activities, and outcomes were specifically linked to the Center-wide model.

Discussion

A hierarchical logic model process ensures that the objectives of the funding agency or organization are addressed, and enables stakeholders to articulate the linkages between each layer. This facilitates the process of developing, implementing and evaluating programmatic elements within the framework of strategic planning.

AN INTEGRATED PERSPECTIVE ON THE ASSESSMENT OF TECHNOLOGIES: INTEGRATE-HTA

OBJECTIVES

Current health technology assessment (HTA) is not well equipped to assess complex technologies as insufficient attention is being paid to the diversity in patient characteristics and preferences, context, and implementation. Strategies to integrate these and several other aspects, such as ethical considerations, in a comprehensive assessment are missing. The aim of the European research project INTEGRATE-HTA was to develop a model for an integrated HTA of complex technologies.

METHODS

A multi-method, four-stage approach guided the development of the INTEGRATE-HTA Model: (i) definition of the different dimensions of information to be integrated, (ii) literature review of existing methods for integration, (iii) adjustment of concepts and methods for assessing distinct aspects of complex technologies in the frame of an integrated process, and (iv) application of the model in a case study and subsequent revisions.

RESULTS

The INTEGRATE-HTA Model consists of five steps, each involving stakeholders: (i) definition of the technology and the objective of the HTA; (ii) development of a logic model to provide a structured overview of the technology and the system in which it is embedded; (iii) evidence assessment on effectiveness, economic, ethical, legal, and socio-cultural aspects, taking variability of participants, context, implementation issues, and their interactions into account; (iv) populating the logic model with the data generated in step 3; (v) structured process of decision-making.

CONCLUSIONS

The INTEGRATE-HTA Model provides a structured process for integrated HTAs of complex technologies. Stakeholder involvement in all steps is essential as a means of ensuring relevance and meaningful interpretation of the evidence.

Pathways linking car transport for young adults and the public health in Northern Ireland: a qualitative study to inform the evaluation of graduated driver licensing

Background

Novice drivers are at relatively high risk of road traffic injury. There is good evidence that Graduated Driving Licensing (GDL) schemes reduce collisions rates, by reducing exposure to risk and by extending learning periods. Legislation for a proposed scheme in Northern Ireland was passed in 2016, providing an opportunity for future evaluation of the full public health impacts of a scheme in a European context within a natural experiment. This qualitative study was designed to inform the logic model for such an evaluation, and provide baseline qualitative data on the role of private cars in health and wellbeing.

Methods

Nine group interviews with young people aged 16–23 (N = 43) and two group interviews with parents of young people (N = 8) were conducted in a range of settings in Northern Ireland in 2015. Data were analysed using thematic content analysis.

Results

Informal car-pooling within and beyond households led to routine expectations of lift provision and uptake. Experiences of risky driving situations were widespread. In rural areas, extensive use of farm vehicles for transport needs meant many learner drivers had both early driving experience and expectations that legislation may have to be locally adapted to meet social needs. Cars were used as a site for socialising, as well as essential means of transport. Alternative modes (public transport, walking and cycling) were held in low esteem, even where available. Recall of other transport-related public health messages and parents’ existing use of GDL-type restrictions suggested GDL schemes were acceptable in principle. There was growing awareness and use of in-car technologies (telematics) used by insurance companies to reward good driving.

Conclusions

Key issues to consider in evaluating the broader public health impact of GDL will include: changes in injury rates for licensed car occupants and other populations and modes; changes in exposure to risk in the licensed and general population; and impact on transport exclusion. We suggest an important pathway will be change in social norms around offering and accepting lifts and to risk-taking. The growing adoption of in-car telematics will have implications for future GDL programmes and for evaluation.

Evaluation of a cross-cultural training program for Pakistani educators: Lessons learned and implications for program planning

In this paper, we share the results of a summative evaluation of PEILI, a US-based adult professional development/training program for secondary school Pakistani teachers. The evaluation was guided by the theories of cultural competence (American Psychological Association, 2003; Bamberger, 1999; Wadsworth, 2001) and established frameworks for the evaluation of professional development/training and instructional design (Bennett, 1975; Guskey, 2002; King, 2014; Kirkpatrick, 1967). The explicit and implicit stakeholder assumptions about the connections between program resources, activities, outputs, and outcomes are described. Participant knowledge and skills were measured via scores on a pre/posttest of professional knowledge, and a standards-based performance assessment rubric. In addition to measuring short-term program outcomes, we also sought to incorporate theory-driven thinking into the evaluation design. Hence, we examined participant self-efficacy and access to social capital, two evidenced-based determinants or "levers" that theoretically explain the transformative space between an intervention and its outcomes (Chen, 2012). Data about program determinants were collected and analyzed through a pre/posttest of self-efficacy and social network analysis. Key evaluation findings include participant acquisition of new instructional skills, increased self-efficacy, and the formation of a nascent professional support network. Lessons learned and implications for the design and evaluation of cross-cultural teacher professional development programs are discussed.