1
|
Moon DJ, Nichols CB, Zhang Y, Cruce A, Haran H, Sgourakis A, Lee H, Johnson-Motoyama M. Engagement Measures in Maltreatment Prevention Studies: A Scoping Review. TRAUMA, VIOLENCE & ABUSE 2024; 25:1551-1567. [PMID: 37626470 DOI: 10.1177/15248380231188070] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/27/2023]
Abstract
Prevention services can promote public health by building protective factors and reducing maltreatment risk. Yet, engaging caregivers in prevention services presents a unique set of challenges. Measurement studies are important first steps to increase the knowledge of caregiver engagement in prevention services. The purpose of this scoping review was to investigate how family engagement has been measured and operationalized in the studies of maltreatment prevention/positive parenting programs. The review examined quantitative and mixed methods studies conducted in the U.S., which measured multiple dimensions of client engagement, including behavioral, attitudinal, and relational domains. A total of 88 studies selected from PubMed, CINAHL, ERIC, PsycINFO, Social Work Abstracts, Academic Search Premier, and Web of Science were included in this review. Results indicated that studies examine engagement constructs in all three domains of engagement with a primary focus on behavioral engagement. The attitudinal and relational engagement was mostly assessed through general satisfaction surveys, and a limited number of studies utilized validated measures to assess those constructs. While most studies reported acceptable internal reliabilities, only two studies reported other dimensions of psychometric qualities. Only one validated measure was found, which assessed client perceptions of provider cultural competence. More measurement studies are needed to further incorporate multiple dimensions of engagement into the studies of maltreatment prevention programs, which can inform the effort to develop tailored implementation strategies to fully engage various groups of parents in maltreatment prevention programs.
Collapse
|
2
|
Smith JD, Li DH, Merle JL, Keiser B, Mustanski B, Benbow ND. Adjunctive interventions: change methods directed at recipients that support uptake and use of health innovations. Implement Sci 2024; 19:10. [PMID: 38331832 PMCID: PMC10854146 DOI: 10.1186/s13012-024-01345-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2023] [Accepted: 01/24/2024] [Indexed: 02/10/2024] Open
Abstract
BACKGROUND Implementation science groups change methods into two categories: (1) clinical, behavioral, or biomedical intervention targeting recipient's health outcomes and (2) implementation strategies targeting the delivery system. Differentiating interventions from strategies based on their intended functions is critical to accurately attributing their effects to health or implementation outcomes. However, in coordinating 200+ HIV implementation research projects and conducting systematic reviews, we identified change methods that had characteristics of both interventions and strategies that were inconsistently categorized. To alleviate confusion and improve change method specification, we propose that implementation science should adopt an extant but rarely used term-adjunctive interventions-to classify change methods that are distinct from the common intervention/strategy taxonomy. MAIN TEXT Adjunctive interventions as change methods that target recipients (e.g., patients, participants) of a health intervention but are designed to increase recipients' motivation, self-efficacy, or capacity for initiating, adhering to, complying with, or engaging with the health intervention over time. In two of our published reviews on implementation of HIV interventions, 25 out of 45 coded change methods fell into this gray area between strategy and intervention. We also noted instances in which the same change method was labelled as the intervention ("the thing"), as an adjunctive intervention, or an implementation strategy in different studies-further muddying the waters. Adjunctive interventions are distinguished from other change methods by their intended targets, desired outcomes, and theory of action and causal processes. Whereas health interventions target recipients and have a direct, causal effect on the health outcome, adjunctive interventions enhance recipients' attitudes and behaviors to engage with the intervention and have an indirect causal link to the health outcome via increasing the probability of recipients' utilization and adherence to the intervention. Adjunctive interventions are incapable of directly producing the health outcome and will themselves require implementation strategies to effectively impact sustained uptake, utilization, and adherence. Case examples, logic modeling, and considerations (e.g., relationship to consumer engagement strategies) for adjunctive intervention research are provided. CONCLUSION Conceptualizing adjunctive interventions as a separate type of change method will advance implementation research by improving tests of effectiveness, and the specification of mechanisms and outcomes.
Collapse
Affiliation(s)
- Justin D Smith
- Department of Population Health Sciences, Spencer Fox Eccles School of Medicine, University of Utah, Salt Lake City, UT, USA.
| | - Dennis H Li
- Department of Psychiatry and Behavioral Sciences and Institute for Sexual and Gender Minority Health and Wellbeing, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - James L Merle
- Department of Population Health Sciences, Spencer Fox Eccles School of Medicine, University of Utah, Salt Lake City, UT, USA
| | - Brennan Keiser
- Department of Family Medicine, University of Washington School of Medicine, Seattle, WA, USA
| | - Brian Mustanski
- Department of Medical Social Sciences, Third Coast Center for AIDS Research, Institute for Sexual and Gender Minority Health and Wellbeing, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Nanette D Benbow
- Department of Psychiatry and Behavioral Sciences and Institute for Sexual and Gender Minority Health and Wellbeing, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| |
Collapse
|
3
|
Devlin BL, Paes TM, Geer EA, Bryant LM, Zehner TM, Korucu I, Morse K, Duncan RJ, Purpura DJ, Schmitt SA. Moving beyond dosage and adherence: A protocol for capturing dimensions of active child engagement as a measure of fidelity for social-emotional learning interventions. Front Psychol 2023; 13:1014713. [PMID: 36698587 PMCID: PMC9869279 DOI: 10.3389/fpsyg.2022.1014713] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2022] [Accepted: 12/20/2022] [Indexed: 01/12/2023] Open
Abstract
Social-emotional competencies are important for school-readiness and can be supported through social-emotional learning (SEL) interventions in the preschool years. However, past research has demonstrated mixed efficacy of early SEL interventions across varied samples, highlighting a need to unpack the black box of which early interventions work, under what conditions, and for whom. In the present article we discuss the critical implementation component of active child engagement in an intervention as a potential point of disconnect between the intervention as designed and as implemented. Children who are physically present but unengaged during an intervention may lead to decreased average impacts of an intervention. Furthermore, measuring young children's active engagement with an intervention may help to guide iterative intervention development. We propose a four-step protocol for capturing the multi-dimensional and varied construct of active child engagement in a SEL intervention. To illustrate the utility of the protocol, we apply it to data from a pilot study of a researcher-implemented, semi-structured block play intervention focused on supporting the development of SEL and math skills in preschoolers. We then present future directions for the integration of active participant engagement into the measurement of implementation of SEL interventions for young children.
Collapse
Affiliation(s)
- Brianna L. Devlin
- Human Development and Family Studies, Purdue University, West Lafayette, IN, United States
| | - Tanya M. Paes
- Human Development and Family Studies, Purdue University, West Lafayette, IN, United States
| | - Elyssa A. Geer
- Human Development and Family Studies, Purdue University, West Lafayette, IN, United States
| | - Lindsey M. Bryant
- Human Development and Family Studies, Purdue University, West Lafayette, IN, United States
| | - Tracy M. Zehner
- Human Development and Family Studies, Purdue University, West Lafayette, IN, United States
| | - Irem Korucu
- Yale Center for Emotional Intelligence, School of Medicine, Yale University, New Haven, CT, United States
| | - Kathleen Morse
- Human Development and Family Studies, Purdue University, West Lafayette, IN, United States
| | - Robert J. Duncan
- Human Development and Family Studies, Purdue University, West Lafayette, IN, United States
| | - David J. Purpura
- Human Development and Family Studies, Purdue University, West Lafayette, IN, United States
| | - Sara A. Schmitt
- Human Development and Family Studies, Purdue University, West Lafayette, IN, United States
| |
Collapse
|
4
|
Berkel C, Knox DC, Flemotomos N, Martinez VR, Atkins DC, Narayanan SS, Rodriguez LA, Gallo CG, Smith JD. A machine learning approach to improve implementation monitoring of family-based preventive interventions in primary care. IMPLEMENTATION RESEARCH AND PRACTICE 2023; 4:26334895231187906. [PMID: 37790171 PMCID: PMC10375039 DOI: 10.1177/26334895231187906] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/05/2023] Open
Abstract
Background Evidence-based parenting programs effectively prevent the onset and escalation of child and adolescent behavioral health problems. When programs have been taken to scale, declines in the quality of implementation diminish intervention effects. Gold-standard methods of implementation monitoring are cost-prohibitive and impractical in resource-scarce delivery systems. Technological developments using computational linguistics and machine learning offer an opportunity to assess fidelity in a low burden, timely, and comprehensive manner. Methods In this study, we test two natural language processing (NLP) methods [i.e., Term Frequency-Inverse Document Frequency (TF-IDF) and Bidirectional Encoder Representations from Transformers (BERT)] to assess the delivery of the Family Check-Up 4 Health (FCU4Health) program in a type 2 hybrid effectiveness-implementation trial conducted in primary care settings that serve primarily Latino families. We trained and evaluated models using 116 English and 81 Spanish-language transcripts from the 113 families who initiated FCU4Health services. We evaluated the concurrent validity of the TF-IDF and BERT models using observer ratings of program sessions using the COACH measure of competent adherence. Following the Implementation Cascade model, we assessed predictive validity using multiple indicators of parent engagement, which have been demonstrated to predict improvements in parenting and child outcomes. Results Both TF-IDF and BERT ratings were significantly associated with observer ratings and engagement outcomes. Using mean squared error, results demonstrated improvement over baseline for observer ratings from a range of 0.83-1.02 to 0.62-0.76, resulting in an average improvement of 24%. Similarly, results demonstrated improvement over baseline for parent engagement indicators from a range of 0.81-27.3 to 0.62-19.50, resulting in an approximate average improvement of 18%. Conclusions These results demonstrate the potential for NLP methods to assess implementation in evidence-based parenting programs delivered at scale. Future directions are presented. Trial registration NCT03013309 ClinicalTrials.gov.
Collapse
Affiliation(s)
- Cady Berkel
- College of Health Solutions, Arizona State University, Phoenix, AZ, USA
- Ming Hsieh Department of Electrical Engineering, USC Viterbi School of Engineering, REACH Institute, Arizona State University, Tempe, AZ, USA
| | - Dillon C. Knox
- Signal Analysis and Interpretation Laboratory, University of Southern California, Los Angeles, CA, USA
| | - Nikolaos Flemotomos
- Signal Analysis and Interpretation Laboratory, University of Southern California, Los Angeles, CA, USA
| | - Victor R. Martinez
- Signal Analysis and Interpretation Laboratory, University of Southern California, Los Angeles, CA, USA
| | - David C. Atkins
- Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA, USA
| | - Shrikanth S. Narayanan
- Signal Analysis and Interpretation Laboratory, University of Southern California, Los Angeles, CA, USA
| | - Lizeth Alonso Rodriguez
- Ming Hsieh Department of Electrical Engineering, USC Viterbi School of Engineering, REACH Institute, Arizona State University, Tempe, AZ, USA
| | - Carlos G. Gallo
- Department of Psychiatry and Behavioral Sciences, Northwestern University, Chicago, IL, USA
| | - Justin D. Smith
- Department of Population Health Sciences, Spencer Fox Eccles School of Medicine, University of Utah, Salt Lake City, UT, USA
| |
Collapse
|
5
|
Oliveira S, Roberto MS, Veiga-Simão AM, Marques-Pinto A. Effects of the A+ intervention on elementary-school teachers' social and emotional competence and occupational health. Front Psychol 2022; 13:957249. [PMID: 36312103 PMCID: PMC9596939 DOI: 10.3389/fpsyg.2022.957249] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Accepted: 09/20/2022] [Indexed: 11/13/2022] Open
Abstract
Teaching is, to date, one of the most prone jobs to experiencing occupational stress and burnout. Owing to burnout's negative personal, social, organizational and economic impacts, researchers, practitioners and education policy leaders are interested in developing practices and interventions aimed at preventing/reducing its prevalence. With teachers' main professional demands to be of a social and emotional nature, interventions designed with a view to promote teachers' social and emotional competence appears to be particularly promising, positively impacting teachers' well-being and personal accomplishment and contributing to a decrease in their psychological distress, namely emotional exhaustion. However, theoretical and empirically grounded interventions with ecological validity and specifically targeting teachers are still scarce. Thus, to bridge the previously identified gaps, the present study aimed to evaluate the efficacy and the quality of the intervention's implementation of the A+, an online social and emotional learning intervention for elementary-school teachers. A quasi-experimental study was conducted with a total of 81 participants (96.3% female, MAge = 46.21, SDAge = 4.82, n = 42 assigned to the experimental group) from three different school contexts. School clusters were similar in size, organizational structure and socioeconomic level, and as regards previous attendance at social and emotional learning interventions; however, they differed with regards to perceived organizational climate. Data on the efficacy of the A+ was collected across four waves using a set of self-report questionnaires that assessed proximal variables (i.e., social and emotional skills) and distal variables (e.g., well-being, burnout symptoms), and analyzed through Robust Linear Mixed-Effects Models. Coefficient omegas suggested adequate reliability of the measures. Additionally, two trained observers completed an observation grid to evaluate the quality of the A+ implementation (e.g., participant responsiveness, fidelity), with excellent inter-rater reliability. Results suggested that, over time, the A+ had positive impacts across proximal (e.g., increased self-regulation, positive relationship, conflict management skills) and distal variables (e.g., increased emotional well-being, decreased occupational stress and emotional exhaustion symptoms) favoring the experimental group. However, results differed across the school contexts. These findings were accompanied by good implementation quality indicators, namely high fidelity in the delivery of the A+ contents and high participants' responsiveness. Despite its limitations, this study contributes to a growing body of research which reinforces the importance of investing in social and emotional learning interventions to prevent teachers' burnout and improve their occupational health. Furthermore, it highlights the importance of implementation quality research as a component of program planning with a view to enhancing programs' efficacy, as well as the need to adapt and consider context variables in research and practice.
Collapse
Affiliation(s)
- Sofia Oliveira
- Centro de Investigação em Ciência Psicológica, Faculdade de Psicologia, Universidade de Lisboa, Lisboa, Portugal
| | | | | | | |
Collapse
|
6
|
Responsiveness to mHealth Intervention for Cannabis Use in Young Adults Predicts Improved Outcomes. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2022; 23:630-635. [PMID: 35080713 DOI: 10.1007/s11121-022-01333-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/05/2022] [Indexed: 10/19/2022]
Abstract
Mobile health (mHealth) interventions have proliferated rapidly in part because of their advantages in reducing consumer and provider burden, but less attention has been paid to participant responsiveness to mHealth programs and how this may affect outcomes. This study adds to that literature by examining whether participant responsiveness to a text messaging-delivered intervention was predictive of treatment outcomes over baseline levels of the outcome. We analyzed data from a pilot-randomized controlled trial of a text messaging-intervention to treat young adults with cannabis use disorder (treatment arm, N = 47), examining three indicators of responsiveness (two behavioral: treatment completion and booster message participation; and one subjective: perceived helpfulness of treatment) on abstinence from cannabis use and use-related problems measured at 3-month follow-up. With the exception of completion, the indicators were positively correlated with each other. Each of the indicators was predictive of better treatment outcomes above and beyond baseline risk. Treatment completion and booster participation-measured via technical data captured during intervention administration-appeared to be stronger predictors of improved outcomes than self-reported perceived helpfulness. Results suggest that behavioral and subjective responsiveness measures appear to be valid indicators of treatment response to mHealth interventions for substance use. Responsiveness measured via technical data captured during intervention administration may be a stronger and more efficient strategy for monitoring continued engagement. We discuss implications of these findings for deploying mHealth interventions at scale and monitoring responsiveness.
Collapse
|
7
|
Pasalich DS, Moretti MM, Hassall A, Curcio A. Pilot randomized controlled trial of an attachment- and trauma-focused intervention for kinship caregivers. CHILD ABUSE & NEGLECT 2021; 120:105178. [PMID: 34243100 DOI: 10.1016/j.chiabu.2021.105178] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/09/2021] [Revised: 06/10/2021] [Accepted: 06/17/2021] [Indexed: 06/13/2023]
Abstract
BACKGROUND Kinship care placements are increasing in many Western countries, however families in kinship care are underserved partly due to the lack of evidence-based interventions addressing their unique needs. OBJECTIVE We conducted a pilot randomized controlled trial (RCT) to examine research feasibility and the acceptability, fidelity, and preliminary outcomes of an attachment- and trauma-focused intervention for kinship caregivers in Australia. PARTICIPANTS AND SETTING Participants included 26 kinship caregivers (Mage = 55.00 years; 54% grandparents) with youth (Mage = 10.58 years; 38% female), and 19 practitioners from child protection, out-of-home care and other services. METHOD Kinship caregivers were randomized to Connect for Kinship Parents (Connect-KP) or care-as-usual (CAU), and completed assessments at baseline, post-intervention, and 6-month follow-up. Placement changes were evaluated at 6-month follow-up. A subset of practitioners received both training and supervision in implementing Connect-KP, and their videorecorded sessions were coded for fidelity. RESULTS Practitioners rated the training highly and demonstrated excellent fidelity to program content and process. Results supported the research feasibility and all kinship caregivers reported high levels of program satisfaction and had very high attendance and completion rates. Direction of effects favored Connect-KP vs. CAU for all nine caregiver and youth outcomes, with the largest effects observed for significant reductions in caregiver strain, caregiver psychological aggression, and youth affect suppression following intervention. At 6-month follow-up, more youth from CAU (15%) vs. Connect-KP (0%) experienced an unplanned placement change. CONCLUSIONS Our findings show promise of Connect-KP as a potentially effective intervention that can be successfully implemented in child welfare services, and provide impetus for a larger-scale RCT.
Collapse
Affiliation(s)
- Dave S Pasalich
- Research School of Psychology, Australian National University, Australia.
| | | | - Alison Hassall
- Research School of Psychology, Australian National University, Australia
| | - Angela Curcio
- Community Services Directorate, ACT Government, Australia
| |
Collapse
|
8
|
O'Hara KL, Wolchik SA, Sandler IN. The Development, Evaluation, and Implementation of Parenting-focused Prevention Programs in Collaboration with Family Court. FAMILY COURT REVIEW 2021; 59:710-724. [PMID: 35832350 PMCID: PMC9272994 DOI: 10.1111/fcre.12604] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Promoting the well-being and best interests of children in separated and divorcing families is a shared value among family court professionals and prevention scientists who develop and evaluate intervention programs. This article chronicles the development, evaluation, and implementation of two programs - the New Beginnings Program (NBP), a parenting intervention for separated/divorcing parents and the Family Transitions Guide (FTG), an intervention designed to motivate high conflict separated/divorcing parents to attend the NBP. The development and evaluation of these programs was facilitated by a long-standing collaboration with Maricopa Family Court. We discuss the process of developing these programs, their underlying small theories, and the evaluation of their effects in randomized trials. We also describe our collaboration with the family court and ways that the court promoted the development and evaluation of these programs. Finally, we summarize lessons learned and discuss future directions to bolster the public health impact of evidence-based programs for separated/divorcing families.
Collapse
|
9
|
Berkel C, Mauricio AM, Rudo-Stern J, Dishion TJ, Smith JD. Motivational Interviewing and Caregiver Engagement in the Family Check-Up 4 Health. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2021; 22:737-746. [PMID: 32488687 PMCID: PMC7710530 DOI: 10.1007/s11121-020-01112-8] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
Abstract
Motivational interviewing (MI) is a therapeutic style in which a provider elicits client motivation and helps strengthen commitment to change (Miller and Rollnick 2002). The original Family Check-Up (FCU; Dishion and Stormshak 2007)-and the adapted version for improving health behaviors in primary care, the Family Check-Up 4 Health (FCU4Health; Smith et al. 2018a)-are brief, assessment-driven, and family-centered preventive interventions that use MI to improve parent engagement in services to improve parenting and prevent negative child outcomes. This study examines the role of MI in the Raising Healthy Children project, a randomized trial to test the effectiveness of the FCU4Health for the prevention of obesity in pediatric primary care, with data from the 141 families assigned to receive the FCU4Health. Families were eligible for the study if the child was between 5.5 and 12 years of age at the time of identification and had a BMI ≥ 85th percentile for age and gender at the most recent visit to their primary care provider. MI skills at the first session predicted caregiver in-session active engagement, attendance at follow-up parenting sessions, and improvements in motivation to address child health and behavior goals. Baseline characteristics of the family (i.e., child health diagnosis, caregiver baseline depression, motivation, and Spanish language preference) had differential associations with responsiveness and MI skills. This study has implications for program development, provider training, and fidelity monitoring.
Collapse
Affiliation(s)
- Cady Berkel
- Department of Psychology, REACH Institute, Arizona State University, 900 S McAllister Ave, Tempe, AZ, USA.
- Phoenix Children's Hospital, Phoenix, AZ, USA.
| | - Anne M Mauricio
- Department of Psychology, REACH Institute, Arizona State University, 900 S McAllister Ave, Tempe, AZ, USA
| | | | - Thomas J Dishion
- Department of Psychology, REACH Institute, Arizona State University, 900 S McAllister Ave, Tempe, AZ, USA
| | - Justin D Smith
- Center for Prevention Implementation Methodology, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| |
Collapse
|
10
|
Michaelson V, Pilato KA, Davison CM. Family as a health promotion setting: A scoping review of conceptual models of the health-promoting family. PLoS One 2021; 16:e0249707. [PMID: 33844692 PMCID: PMC8041208 DOI: 10.1371/journal.pone.0249707] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2020] [Accepted: 03/23/2021] [Indexed: 12/02/2022] Open
Abstract
Background The family is a key setting for health promotion. Contemporary health promoting family models can establish scaffolds for shaping health behaviors and can be useful tools for education and health promotion. Objectives The objective of this scoping review is to provide details as to how conceptual and theoretical models of the health promoting potential of the family are being used in health promotion contexts. Design Guided by PRISMA ScR guidelines, we used a three-step search strategy to find relevant papers. This included key-word searching electronic databases (Medline, PSycINFO, Embase, and CINAHL), searching the reference lists of included studies, and intentionally searching for grey literature (in textbooks, dissertations, thesis manuscripts and reports.) Results After applying inclusion and exclusion criteria, the overall search generated 113 included manuscripts/chapters with 118 unique models. Through our analysis of these models, three main themes were apparent: 1) ecological factors are central components to most models or conceptual frameworks; 2) models were attentive to cultural and other diversities, allowing room for a wide range of differences across family types, and for different and ever-expanding social norms and roles; and 3) the role of the child as a passive recipient of their health journey rather than as an active agent in promoting their own family health was highlighted as an important gap in many of the identified models. Conclusions This review contributes a synthesis of contemporary literature in this area and supports the priority of ecological frameworks and diversity of family contexts. It encourages researchers, practitioners and family stakeholders to recognize the value of the child as an active agent in shaping the health promoting potential of their family context.
Collapse
Affiliation(s)
- Valerie Michaelson
- Department of Health Sciences, Brock University, St. Catharines, Canada
- * E-mail:
| | - Kelly A. Pilato
- Department of Health Sciences, Brock University, St. Catharines, Canada
| | - Colleen M. Davison
- Department of Public Health Sciences, Queen’s University, Kingston, Canada
| |
Collapse
|
11
|
Gallo CG, Berkel C, Mauricio A, Sandler I, Wolchik S, Villamar JA, Mehrotra S, Brown CH. Implementation methodology from a social systems informatics and engineering perspective applied to a parenting training program. FAMILIES, SYSTEMS & HEALTH : THE JOURNAL OF COLLABORATIVE FAMILY HEALTHCARE 2021; 39:7-18. [PMID: 34014726 PMCID: PMC8962635 DOI: 10.1037/fsh0000590] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
OBJECTIVE For implementation of an evidence-based program to be effective, efficient, and equitable across diverse populations, we propose that researchers adopt a systems approach that is often absent in efficacy studies. To this end, we describe how a computer-based monitoring system can support the delivery of the New Beginnings Program (NBP), a parent-focused evidence-based prevention program for divorcing parents. METHOD We present NBP from a novel systems approach that incorporates social system informatics and engineering, both necessary when utilizing feedback loops, ubiquitous in implementation research and practice. Examples of two methodological challenges are presented: how to monitor implementation, and how to provide feedback by evaluating system-level changes due to implementation. RESULTS We introduce and relate systems concepts to these two methodologic issues that are at the center of implementation methods. We explore how these system-level feedback loops address effectiveness, efficiency, and equity principles. These key principles are provided for designing an automated, low-burden, low-intrusive measurement system to aid fidelity monitoring and feedback that can be used in practice. DISCUSSION As the COVID-19 pandemic now demands fewer face-to-face delivery systems, their replacement with more virtual systems for parent training interventions requires constructing new implementation measurement systems based on social system informatics approaches. These approaches include the automatic monitoring of quality and fidelity in parent training interventions. Finally, we present parallels of producing generalizable and local knowledge bridging systems science and engineering method. This comparison improves our understanding of system-level changes, facilitates a program's implementation, and produces knowledge for the field. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
Collapse
Affiliation(s)
- Carlos G Gallo
- Department of Psychiatry and Behavioral Sciences, Feinberg School of Medicine, Northwestern University
| | - Cady Berkel
- Integrated Behavior Health, College of Health Solutions, AZ State University
| | - Anne Mauricio
- REACH Institute, Department of Psychology, AZ State University
| | - Irwin Sandler
- REACH Institute, Department of Psychology, AZ State University
| | | | - Juan A Villamar
- Department of Psychiatry and Behavioral Sciences, Feinberg School of Medicine, Northwestern University
| | - Sanjay Mehrotra
- Department of Industrial Engineering and Management Sciences, Northwestern University
| | - C Hendricks Brown
- Department of Psychiatry and Behavioral Sciences, Feinberg School of Medicine, Northwestern University
| |
Collapse
|
12
|
Kim JJ, Gonzales NA, Thamrin H, Mauricio A, Kuckertz M, Camacho-Thompson D. What got in the way? Caregiver-reported challenges to home practice of assigned intervention skills. IMPLEMENTATION RESEARCH AND PRACTICE 2021; 2:26334895211055994. [PMID: 37090005 PMCID: PMC9978686 DOI: 10.1177/26334895211055994] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
Background Evidence-based intervention (EBI) effectiveness is hindered by low rates of caregiver home practice, or caregiver rehearsal of intervention skills at home. Although home practice is essential to intervention success, we know little about what makes home practice difficult to complete. We sought to identify the challenges caregivers face when attempting home practice assignments within the context of community implementation of a family-based preventative EBI. We also aimed to identify barriers, perceived need, and implementation-related factors that may contribute to frequency of home practice challenges. Methods The family-based preventative intervention Bridges was delivered to caregivers and adolescents in three public middle schools in North America within a randomized efficacy-effectiveness hybrid trial. Caregivers (N = 233) completed worksheets in which they reported challenges to home practice of intervention skills. Caregiver-endorsed challenges were coded for emergent themes and categorized into higher-order home practice challenge domains. Additional data was gathered through multiple methods. In addition to descriptive analyses, multilevel linear regressions were conducted to identify factors associated with frequency of home practice challenge domains. Results Over 85% of caregivers reported one or more home practice challenges. Challenges fell into six domains: daily stresses, cognitions/beliefs about the intervention/practice, caregiver wellness/regulation, adolescent responsiveness, relevance/acceptability of intervention or assignment, and external barriers to practice. Adolescent externalizing symptoms, academic performance, and caregiver participation in the English-language intervention group (compared to Spanish-language delivery) were associated with greater frequency of reported challenges. Conclusions Among caregivers already attending intervention sessions, specific support to overcome challenges in daily stresses and beliefs about the intervention/practice can support greater home practice compliance, which in turn may impact overall intervention effectiveness. Findings suggest that providing services in caregivers' preferred language may avert potential challenges. Further research into home practice challenges is necessary to identify appropriate home practice implementation supports. Plain Language Summary Low rates and poor quality of caregiver home practice of intervention skills hampers intervention effectiveness, but no prior studies have identified the difficulties caregivers face when attempting home practice of intervention skills and their rates of occurrence. Within a community implementation of a family-based preventative intervention, we found that difficulties were common and largely fell in the domains theorized to impact engagement, such as daily stresses and beliefs about the intervention/practice. Findings also suggest that providing services and materials in caregivers' preferred language may avert potential home practice challenges. By identifying common home practice challenges and correlates of different challenges, our findings point to a need to develop appropriate and effective home practice implementation support.
Collapse
|
13
|
Martin M, Steele B, Lachman JM, Gardner F. Measures of Facilitator Competent Adherence Used in Parenting Programs and Their Psychometric Properties: A Systematic Review. Clin Child Fam Psychol Rev 2021; 24:834-853. [PMID: 34021442 PMCID: PMC8541983 DOI: 10.1007/s10567-021-00350-8] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/28/2021] [Indexed: 01/15/2023]
Abstract
Implementation fidelity is a critical component of intervention science, which aims to understand how interventions unfold in practice to improve outcomes. A key element of fidelity is facilitator competent adherence-the extent to which a program is delivered as prescribed with the specified level of quality. We conducted a two-part systematic review examining these aspects in parenting programs aiming to reduce child behavior problems and maltreatment. Part One reviews measures of facilitator competent adherence and Part Two examines the psychometric properties of the observational measures found. Searches identified 9153 articles from electronic databases, citation tracking, and expert input. After screening using pre-specified criteria, 156 (Part One) and 41 (Part Two) articles remained. In Part One, measure, facilitator, and intervention characteristics were extracted and synthesized from 65 measures. Most measures were observational, used by facilitators and researchers, and employed Likert-scale ratings. In Part Two, evidence on the reliability (internal consistency, inter-rater, intra-rater, test-retest) and validity (content, construct, convergent/divergent, criterion) of 30 observational measures identified from Part One was synthesized and evaluated. An adapted COSMIN checklist was used to assess study and measure quality. We found most studies to be of reasonably high quality. This is the first review to summarize and critically appraise measures of facilitator competent adherence used in the parenting program literature and establish their psychometric properties. The findings underscore the need to advance research on measures of facilitator competent adherence; reliable, valid, and high-quality implementation measures allow for evidence-based decisions regarding the delivery and scale-up of parenting programs. PROSPERO Registration Number: CRD42020167872.
Collapse
Affiliation(s)
- Mackenzie Martin
- Centre for Evidence-Based Intervention, Department of Social Policy and Intervention, University of Oxford, Oxford, UK.
| | - Bridget Steele
- Centre for Evidence-Based Intervention, Department of Social Policy and Intervention, University of Oxford, Oxford, UK
| | - Jamie M. Lachman
- Centre for Evidence-Based Intervention, Department of Social Policy and Intervention, University of Oxford, Oxford, UK
| | - Frances Gardner
- Centre for Evidence-Based Intervention, Department of Social Policy and Intervention, University of Oxford, Oxford, UK
| |
Collapse
|
14
|
Berkel C, Gallo CG, Sandler IN, Mauricio AM, Smith JD, Brown CH. Redesigning Implementation Measurement for Monitoring and Quality Improvement in Community Delivery Settings. J Prim Prev 2020; 40:111-127. [PMID: 30656517 DOI: 10.1007/s10935-018-00534-z] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
Abstract
The field of prevention has established the potential to promote child adjustment across a wide array of outcomes. However, when evidence-based prevention programs have been delivered at scale in community settings, declines in implementation and outcomes have resulted. Maintaining high quality implementation is a critical challenge for the field. We describe steps towards the development of a practical system to monitor and support the high-quality implementation of evidence-based prevention programs in community settings. Research on the implementation of an evidence-based parenting program for divorcing families called the "New Beginnings Program" serves as an illustration of the promise of such a system. As a first step, we describe a multidimensional theoretical model of implementation that links aspects of program delivery with improvements in participant outcomes. We then describe research on the measurement of each of these implementation dimensions and test their relations to intended program outcomes. As a third step, we develop approaches to the assessment of these implementation constructs that are feasible to use in community settings and to establish their reliability and validity. We focus on the application of machine learning algorithms and web-based data collection systems to assess implementation and provide support for high quality delivery and positive outcomes. Examples are presented to demonstrate that valid and reliable measures can be collected using these methods. Finally, we envision how these measures can be used to develop an unobtrusive system to monitor implementation and provide feedback and support in real time to maintain high quality implementation and program outcomes.
Collapse
Affiliation(s)
- Cady Berkel
- , 900 S McAllister Ave., Tempe, AZ, 85287, USA. .,REACH Institute, Department of Psychology, Arizona State University, Tempe, AZ, USA.
| | - Carlos G Gallo
- Center for Prevention Implementation Methodology (Ce-PIM), Northwestern University, Chicago, IL, USA
| | - Irwin N Sandler
- REACH Institute, Department of Psychology, Arizona State University, Tempe, AZ, USA
| | - Anne M Mauricio
- REACH Institute, Department of Psychology, Arizona State University, Tempe, AZ, USA
| | - Justin D Smith
- Center for Prevention Implementation Methodology (Ce-PIM), Northwestern University, Chicago, IL, USA
| | - C Hendricks Brown
- Center for Prevention Implementation Methodology (Ce-PIM), Northwestern University, Chicago, IL, USA
| |
Collapse
|
15
|
Williams N, Beidas R. Annual Research Review: The state of implementation science in child psychology and psychiatry: a review and suggestions to advance the field. J Child Psychol Psychiatry 2019; 60:430-450. [PMID: 30144077 PMCID: PMC6389440 DOI: 10.1111/jcpp.12960] [Citation(s) in RCA: 121] [Impact Index Per Article: 24.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 07/18/2018] [Indexed: 02/04/2023]
Abstract
BACKGROUND Scientists have developed evidence-based interventions that improve the symptoms and functioning of youth with psychiatric disorders; however, these interventions are rarely used in community settings. Eliminating this research-to-practice gap is the purview of implementation science, the discipline devoted to the study of methods to promote the use of evidence-based practices in routine care. METHODS We review studies that have tested factors associated with implementation in child psychology and psychiatry, explore applications of social science theories to implementation, and conclude with recommendations to advance implementation science through the development and testing of novel, multilevel, causal theories. RESULTS During its brief history, implementation science in child psychology and psychiatry has documented the implementation gap in routine care, tested training approaches and found them to be insufficient for behavior change, explored the relationships between variables and implementation outcomes, and initiated randomized controlled trials to test implementation strategies. This research has identified targets related to implementation (e.g., clinician motivation, organizational culture) and demonstrated the feasibility of activating these targets through implementation strategies. However, the dominant methodological approach has been atheoretical and predictive, relying heavily on a set of variables from heuristic frameworks. CONCLUSIONS Optimizing the implementation of effective treatments in community care for youth with psychiatric disorders is a defining challenge of our time. This review proposes a new direction focused on developing and testing integrated causal theories. We recommend implementation scientists: (a) move from observational studies of implementation barriers and facilitators to trials that include causal theory; (b) identify a core set of implementation determinants; (c) conduct trials of implementation strategies with clear targets, mechanisms, and outcomes; (d) ensure that behaviors that are core to EBPs are clearly defined; and (e) agree upon standard measures. This agenda will help fulfill the promise of evidence-based practice for improving youth behavioral health.
Collapse
Affiliation(s)
| | - Rinad Beidas
- Department of Psychiatry, University of Pennsylvania Perelman School of Medicine
| |
Collapse
|
16
|
How Program Users Enhance Fidelity: Implementing the WITS Programs in Rural Canadian Elementary Schools. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2018; 19:1066-1078. [PMID: 30259288 DOI: 10.1007/s11121-018-0948-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/28/2022]
Abstract
Research on the processes that enhance implementation fidelity is needed to increase understanding of ways to advance the uptake and sustainability of evidence-based programs (Berkel et al. in Prevention Science, 12, 23-33, 2011; Berkel et al. 2017). We propose and test a theoretical model of interrelations among implementation fidelity (i.e., adherence to program components), and rarely investigated activities of program users that may enhance fidelity; namely integration of program strategies into daily activities and children's use of program strategies (CUoPS). These were assessed across the initial 2 years of the implementation of the WITS peer victimization prevention programs in 16 Canadian rural schools. WITS stands for Walk away, Ignore, Talk it out, and Seek help. All schools were implementing the program. We examined the interrelations among these implementation indicators and their effects on child outcomes targeted by the WITS Programs (i.e., social responsibility, prosocial leadership, peer victimization, emotional symptoms, and aggression). Four implementation assessments were collected from children (n = 1326), their parents, and teachers in the fall and spring of two academic years. The within-time correlations among fidelity, integration, and CUoPS were significant at each assessment. Cross-lagged models showed fidelity and integration, and CUoPS and integration were reciprocally related during each academic year. CUoPS predicted higher subsequent levels of fidelity during and across academic years. Suggestions are given for training and coaching to enhance teachers' integration of program strategies into daily life and for creating opportunities for school staff to observe children using program strategies in order to enhance implementation fidelity.
Collapse
|