Characterizing implementation strategies using a systems engineering survey and interview tool: a comparison across 10 prevention programs for drug abuse and HIV sexual risk behavior

The Application of Systems Thinking to the Prevention and Control of Sexually Transmissible Infections among Adolescents and Adults: A Scoping Review

Systems thinking is a mechanism for making sense of complex systems that challenge linear explanations of cause-and-effect. While the prevention and control of sexually transmissible infections (STIs) has been identified as an area that may benefit from systems-level analyses, no review on the subject currently exists. The aim of this study is to conduct a scoping review to identify literature in which systems thinking has been applied to the prevention and control of STIs among adolescent and adult populations. Joanna Briggs Institute guidelines for the conduct of scoping reviews were followed. Five databases were searched for English-language studies published after 2011. A total of n = 6102 studies were screened against inclusion criteria and n = 70 were included in the review. The majority of studies (n = 34) were conducted in African nations. Few studies focused on priority sub-populations, and 93% were focused on HIV (n = 65). The most commonly applied systems thinking method was system dynamics modelling (n = 28). The review highlights areas for future research, including the need for more STI systems thinking studies focused on: (1) migrant and Indigenous populations; (2) conditions such as syphilis; and (3) innovations such as pre-exposure prophylaxis and at-home testing for HIV. The need for conceptual clarity around 'systems thinking' is also highlighted.

High-Fidelity Agent-Based Modeling to Support Prevention Decision-Making: an Open Science Approach

Preventing adverse health outcomes is complex due to the multi-level contexts and social systems in which these phenomena occur. To capture both the systemic effects, local determinants, and individual-level risks and protective factors simultaneously, the prevention field has called for adoption of system science methods in general and agent-based models (ABMs) specifically. While these models can provide unique and timely insight into the potential of prevention strategies, an ABM's ability to do so depends strongly on its accuracy in capturing the phenomenon. Furthermore, for ABMs to be useful, they need to be accepted by and available to decision-makers and other stakeholders. These two attributes of accuracy and acceptability are key components of open science. To ensure the creation of high-fidelity models and reliability in their outcomes and consequent model-based decision-making, we present a set of recommendations for adopting and using this novel method. We recommend ways to include stakeholders throughout the modeling process, as well as ways to conduct model verification, validation, and replication. Examples from HIV and overdose prevention work illustrate how these recommendations can be applied.

Implementation methodology from a social systems informatics and engineering perspective applied to a parenting training program

OBJECTIVE

For implementation of an evidence-based program to be effective, efficient, and equitable across diverse populations, we propose that researchers adopt a systems approach that is often absent in efficacy studies. To this end, we describe how a computer-based monitoring system can support the delivery of the New Beginnings Program (NBP), a parent-focused evidence-based prevention program for divorcing parents.

METHOD

We present NBP from a novel systems approach that incorporates social system informatics and engineering, both necessary when utilizing feedback loops, ubiquitous in implementation research and practice. Examples of two methodological challenges are presented: how to monitor implementation, and how to provide feedback by evaluating system-level changes due to implementation.

RESULTS

We introduce and relate systems concepts to these two methodologic issues that are at the center of implementation methods. We explore how these system-level feedback loops address effectiveness, efficiency, and equity principles. These key principles are provided for designing an automated, low-burden, low-intrusive measurement system to aid fidelity monitoring and feedback that can be used in practice.

DISCUSSION

As the COVID-19 pandemic now demands fewer face-to-face delivery systems, their replacement with more virtual systems for parent training interventions requires constructing new implementation measurement systems based on social system informatics approaches. These approaches include the automatic monitoring of quality and fidelity in parent training interventions. Finally, we present parallels of producing generalizable and local knowledge bridging systems science and engineering method. This comparison improves our understanding of system-level changes, facilitates a program's implementation, and produces knowledge for the field. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Evaluation of public health interventions from a complex systems perspective: A research methods review

INTRODUCTION

Applying a complex systems perspective to public health evaluation may increase the relevance and strength of evidence to improve health and reduce health inequalities. In this review of methods, we aimed to: (i) classify and describe different complex systems methods in evaluation applied to public health; and (ii) examine the kinds of evaluative evidence generated by these different methods.

METHODS

We adapted critical review methods to identify evaluations of public health interventions that used systems methods. We conducted expert consultation, searched electronic databases (Scopus, MEDLINE, Web of Science), and followed citations of relevant systematic reviews. Evaluations were included if they self-identified as using systems- or complexity-informed methods and if they evaluated existing or hypothetical public health interventions. Case studies were selected to illustrate different types of complex systems evaluation.

FINDINGS

Seventy-four unique studies met our inclusion criteria. A framework was developed to map the included studies onto different stages of the evaluation process, which parallels the planning, delivery, assessment, and further delivery phases of the interventions they seek to inform; these stages include: 1) theorising; 2) prediction (simulation); 3) process evaluation; 4) impact evaluation; and 5) further prediction (simulation). Within this framework, we broadly categorised methodological approaches as mapping, modelling, network analysis and 'system framing' (the application of a complex systems perspective to a range of study designs). Studies frequently applied more than one type of systems method.

CONCLUSIONS

A range of complex systems methods can be utilised, adapted, or combined to produce different types of evaluative evidence. Further methodological innovation in systems evaluation may generate stronger evidence to improve health and reduce health inequalities in our complex world.

Improving the Capacity of Health System and Community for Sickle Cell Disease Screening and Management Among Tribal Population in India: Protocol of an Intervention Study

Sickle cell disease (SCD) is one of the major public health problems in the world. In India, the burden of SCD is comparatively high in socio-economically disadvantaged tribal communities. Though efficacious interventions are available to manage SCD, they are not reaching to these communities and no comprehensive programme is in place in the health care system. Therefore, the Indian Council of Medical Research has initiated a nation-wide study to develop an effective intervention model for SCD patients in tribal areas through the government health care system. This intervention includes increasing awareness and preparing the communities for accessing the government health care system for SCD care, and improving the capacity of the primary health care systems including the training of the health care providers on prevention and management of SCD. The study adopted a quasi-experimental design with pre-vs. post-intervention comparisons of outcome variables within the interventional groups and with the control group. The study will be implemented in 6 districts which are endemic for SCD, spread across different geographical zones of India. In each district, four primary health centre (PHC) areas which are predominantly inhabited by tribal population will be selected. Of these four PHC areas, two will be selected randomly for implementing the intervention and the remaining two will be the control area. Information necessary for development and implementation of the intervention will be gathered during formative research, by using both quantitative and qualitative research methods. Intervention with an inclusive partnership and community mobilization will be implemented. The major steps in the implementation of intervention are partnership building with various health and non-health partners including the community. Capacity building and strengthening is another important component to enable the primary health facilities to screen and manage SCD patients. Primarily, sub-health centres and primary healthcare centres will be equipped with appropriate SCD screening techniques. All doctors in the system will be trained in advanced treatment and management issues. To improve the community's awareness and readiness, community mobilization activities will be conducted. An impact evaluation will be carried out at the end of the intervention by comparing the improvement of SCD management in intervention PHCs to that of the control PHCs. However, the process evaluation and necessary mid-term corrections will be made throughout the intervention period. Thus, an intervention model in terms of its suitability, replicability and sustainability for the tribal population will be developed and tested. The findings of this study are more suitable to use during advocacy and to replicate the model by the state health departments. This study develops and places an appropriate referral system for SCD patients at the PHC level. Improving the community's access to health care, improving the quality of care in government health centres and raising awareness among tribal communities are crucial to achieving through innovation. Taken together, these innovations would significantly contribute to better access to health care and management of the SCD patients of underserved tribal population.

Visual process maps to support implementation efforts: a case example

BACKGROUND

Process mapping is often used in quality improvement work to examine current processes and workflow and to identify areas to intervene to improve quality. Our objective in this paper is to describe process maps as a visual means of understanding modifiable behaviors and activities, in this case example to ensure that goals of care conversations are part of admitting a veteran in long-term care settings.

METHODS

We completed site visits to 6 VA nursing homes and reviewed their current admission processes. We conducted interviews to document behaviors and activities that occur when a veteran is referred to a long-term care setting, during admission, and during mandatory VA reassessments. We created visualizations of the data using process mapping approaches. Process maps for each site were created to document the admission activities for each VA nursing home and were reviewed by the research team to identify consistencies across sites and to identify potential opportunities for implementing goals of care conversations.

RESULTS

We identified five consistent behaviors that take place when a veteran is referred and admitted in long-term care. These behaviors are assessing, discussing, decision-making, documenting, and re-assessing.

CONCLUSIONS

Based on the process maps, it seems feasible that the LST note and order template could be completed along with other routine assessment processes. However, this will require more robust multi-disciplinary collaboration among both prescribing and non-prescribing health care providers. Completing the LST template during the current admission process would increase the likelihood that the template is completed in a timely manner, potentially alleviate the perceived time burden, and help with the provision of veteran-centered care.

Qualitative process evaluation from a complex systems perspective: A systematic review and framework for public health evaluators

BACKGROUND

Public health evaluation methods have been criticized for being overly reductionist and failing to generate suitable evidence for public health decision-making. A "complex systems approach" has been advocated to account for real world complexity. Qualitative methods may be well suited to understanding change in complex social environments, but guidance on applying a complex systems approach to inform qualitative research remains limited and underdeveloped. This systematic review aims to analyze published examples of process evaluations that utilize qualitative methods that involve a complex systems perspective and proposes a framework for qualitative complex system process evaluations.

METHODS AND FINDINGS

We conducted a systematic search to identify complex system process evaluations that involve qualitative methods by searching electronic databases from January 1, 2014-September 30, 2019 (Scopus, MEDLINE, Web of Science), citation searching, and expert consultations. Process evaluations were included if they self-identified as taking a systems- or complexity-oriented approach, integrated qualitative methods, reported empirical findings, and evaluated public health interventions. Two reviewers independently assessed each study to identify concepts associated with the systems thinking and complexity science traditions. Twenty-one unique studies were identified evaluating a wide range of public health interventions in, for example, urban planning, sexual health, violence prevention, substance use, and community transformation. Evaluations were conducted in settings such as schools, workplaces, and neighborhoods in 13 different countries (9 high-income and 4 middle-income). All reported some utilization of complex systems concepts in the analysis of qualitative data. In 14 evaluations, the consideration of complex systems influenced intervention design, evaluation planning, or fieldwork. The identified studies used systems concepts to depict and describe a system at one point in time. Only 4 evaluations explicitly utilized a range of complexity concepts to assess changes within the system resulting from, or co-occurring with, intervention implementation over time. Limitations to our approach are including only English-language papers, reliance on study authors reporting their utilization of complex systems concepts, and subjective judgment from the reviewers relating to which concepts featured in each study.

CONCLUSION

This study found no consensus on what bringing a complex systems perspective to public health process evaluations with qualitative methods looks like in practice and that many studies of this nature describe static systems at a single time point. We suggest future studies use a 2-phase framework for qualitative process evaluations that seek to assess changes over time from a complex systems perspective. The first phase involves producing a description of the system and identifying hypotheses about how the system may change in response to the intervention. The second phase involves following the pathway of emergent findings in an adaptive evaluation approach.

Community REACH: An Implementation of an Evidence-Based Caregiver Program

Background and Objectives

Family caregivers (CGs) are critical to the provision of long-term services and support for older adults. Numerous intervention programs to alleviate CG distress have been developed and evaluated yet few have been implemented in community settings. This paper describes and presents outcomes from Community REACH, a community implementation of the evidence-based Resources for Enhancing Alzheimer's Caregiver Health (REACH) II program.

Research Design and Methods

Community REACH involved a partnership between REACH II investigators and United HomeCare Services (UHCS), a nonprofit home health organization that provides home health, personal care, companion, and respite services. The intervention program, an adapted version of an evidence-based program, was a 6-month multicomponent psychosocial intervention, which involved six individual face-to-face and six individual telephone sessions, and telephone support groups. One hundred and forty-six CGs who were primarily female (76%) and Latino, and providing care for an individual with Alzheimer's disease (AD) were enrolled. Program effectiveness was assessed by examining changes in perceived social support, burden, and depression, and CG self-efficacy.

Results

At 6 months, CGs reported significant decreases in depression, burden, being and bothered by the care recipient's memory problems. There was also a significant decline in the number of CGs at risk for clinical depression. These improvements were maintained at 12 months and there was an increase in feelings of social support.

Discussion and Implications

The findings indicate that evidence-based CG programs can be successfully implemented in community settings and benefit CGs of AD patients. A continued partnership between the program developers and community partners is key to implementation success.

Common and distinct components of communication partner training programmes in stroke, traumatic brain injury and dementia

BACKGROUND

Communication partner training (CPT) programmes for health and care staff working with people with the neurologically based communication disorders associated with stroke, traumatic brain injury (TBI) and dementia are efficacious in improving communication. However, current programmes are lengthy and disorder specific, and therefore may not be suitable as staff training tools in environments with people with multiple communication disorders, and services with a variety of neurological populations.

AIMS

To identify common and distinct components of CPT programmes for stroke, TBI and dementia in order to determine whether there are common delivery methods and content that can be consolidated to improve implementation of CPT in health and care services.

METHODS & PROCEDURES

A qualitative enquiry was used. Four CPT programmes targeting three disorders were identified from systematic reviews and literature searches. Programme data were recorded onto data sheets from manuals, study articles and supplementary materials, and were categorized using the Intervention Taxonomy (ITAX). Content analysis and elements of constant comparative analysis were employed to analyse the CPT programmes.

OUTCOMES & RESULTS

Delivery characteristics were similar across all four programmes. All were delivered face to face in either group or individual contexts. However, duration varied from 1 to 35 h. Six of the 12 categories of information provided were common across all programmes, including 'strategy' 'background to disorder', 'information about communication', 'information about program', 'negative behaviours to avoid' and 'purpose of program'. Programmes differed in the types of skill-building techniques used, with the most common being videos (3/4 programmes), discussions (2/4 programmes) and trainer demonstrations (2/4 programmes). While strategies provided to trained partners targeted similar domains of communication, only 3/96 individual strategies were common to all programmes.

CONCLUSIONS & IMPLICATIONS

There are both common and distinct components in content and delivery methods across CPT programmes. Further research is needed to evaluate the distinct components of the programmes, to determine which individual strategies are efficacious, and which approach to skill-building techniques is most effective. The present study has contributed a comprehensive list of programme components, which can form a basis for describing and refining CPT programmes in future.

Specifying sickle cell disease interventions: a study protocol of the Sickle Cell Disease Implementation Consortium (SCDIC)

BACKGROUND

Sickle cell disease (SCD) is an inherited blood disorder that results in a lifetime of anemia, severe pain, and end-organ damage that can lead to premature mortality. While the SCD field has made major medical advances, much needs to be done to improve the quality of care for people with SCD. This study capitalizes on the Sickle Cell Disease Implementation Consortium (SCDIC), a consortium of eight academic sites aiming to test implementation strategies that could lead to more accelerated application of the NHLBI guidelines for treating SCD. This report documents the process to support the consortium by specifying the interventions being developed.

METHODS

This study consists of three steps. The Principal Investigator of each site and two site representatives who are knowledgeable of the intervention (e.g., study coordinator or the person delivering the intervention) will answer an online survey aiming to capture components of the interventions. This survey will be completed by the site representatives three times during the study: during the development of the interventions, after one year of the interventions being implemented, and at the end of this study (after 2 years). A site visit and semi-structured interview (Step 2) in the first year of the process will capture the context of the sites. Step 3 comprises of the development of a framework with the details of the multi-component SCDIC interventions at the sites.

DISCUSSION

The outcome of this study, a framework of the SCDIC, will enable accurate replication and extension of published research, facilitating the translation of SCD studies to diverse populations and settings and allowing for theory testing of the effects of the intervention components across studies in different contexts and for different populations.

TRIAL REGISTRATION

ClinicalTrial.Gov (# NCT03380351 ). Registered December 21, 2017.

"Scaling-out" evidence-based interventions to new populations or new health care delivery systems

BACKGROUND

Implementing treatments and interventions with demonstrated effectiveness is critical for improving patient health outcomes at a reduced cost. When an evidence-based intervention (EBI) is implemented with fidelity in a setting that is very similar to the setting wherein it was previously found to be effective, it is reasonable to anticipate similar benefits of that EBI. However, one goal of implementation science is to expand the use of EBIs as broadly as is feasible and appropriate in order to foster the greatest public health impact. When implementing an EBI in a novel setting, or targeting novel populations, one must consider whether there is sufficient justification that the EBI would have similar benefits to those found in earlier trials.

DISCUSSION

In this paper, we introduce a new concept for implementation called "scaling-out" when EBIs are adapted either to new populations or new delivery systems, or both. Using existing external validity theories and multilevel mediation modeling, we provide a logical framework for determining what new empirical evidence is required for an intervention to retain its evidence-based standard in this new context. The motivating questions are whether scale-out can reasonably be expected to produce population-level effectiveness as found in previous studies, and what additional empirical evaluations would be necessary to test for this short of an entirely new effectiveness trial. We present evaluation options for assessing whether scaling-out results in the ultimate health outcome of interest.

CONCLUSION

In scaling to health or service delivery systems or population/community contexts that are different from the setting where the EBI was originally tested, there are situations where a shorter timeframe of translation is possible. We argue that implementation of an EBI in a moderately different setting or with a different population can sometimes "borrow strength" from evidence of impact in a prior effectiveness trial. The collection of additional empirical data is deemed necessary by the nature and degree of adaptations to the EBI and the context. Our argument in this paper is conceptual, and we propose formal empirical tests of mediational equivalence in a follow-up paper.