Initial implementation indicators from a statewide rollout of SafeCare within a child welfare system

Mandated Parent Education: Applications, Impacts, and Future Directions

Mandated participation in parent education programs is a common practice across the United States for families who are undergoing divorce or who are involved in the child welfare or juvenile justice systems. Mandates to participate in parenting programs create substantial challenges for families, service providers, and service systems. Furthermore, the type and quality of the parenting services accessed vary widely, and their impacts need to be better understood. To address this need, an overview of the current state of the empirical literature on the impacts and outcomes of mandated parenting interventions for divorce and in child welfare and juvenile justice settings is provided, and suggestions to the field are offered to refine research related to mandated parenting programs. Given the challenges that mandated parenting programs pose, an alternative approach that views parenting through a public health lens is highlighted to build on the growing body of research on the impacts of population-wide applications of parenting support programs, and as a possible way to decrease the number of parents who are required to attend parenting programs. Opportunities to advance universal parenting support within a range of community settings, including primary care, early childhood education, and community mental health systems are offered. Gaps in knowledge regarding mechanisms of action of universal supports and impacts on the number of parents mandated to treatment are highlighted, and future directions for research in this area are suggested.

A Longitudinal Analysis of Trajectories and Predictors of Fidelity Using the SafeCare Parenting Model

Evidence-based practice (EBP) fidelity, understood as the extent to which a program is implemented as the developers intended, is a key implementation variable which likely relates to consumer outcomes. However, studies that track fidelity longitudinally and at large scale are uncommon, and finding reliable predictors of fidelity has proven to be a complex challenge. Further, attitudes toward EBP are a potentially important predictor of fidelity, but results across the literature have been mixed. The purpose of the present study is to use data from the ongoing implementation and dissemination of the SafeCare model to better understand (1) the characteristics of SafeCare implementation fidelity trajectories, and (2) whether individual level factors predict differences in fidelity and fidelity trajectories, especially provider attitudes toward EBP. The analyses reported here include 14,778 observed fidelity sessions by 868 providers in 172 agencies. We use multilevel modeling to examine fidelity, fidelity trajectories over time, and several potential individual-level predictors of fidelity, including demographics, work history, and attitudes toward EBP. We found: (1) that SafeCare fidelity begins high at baseline (93.85% on average); (2) that SafeCare fidelity displays a statistically significant trend of positive linear growth, even among those with less positive attitudes; and (3) that positive attitudes are associated with slightly higher fidelity on average and at baseline, while negative attitudes are associated with slightly lower fidelity on average and at baseline. To our knowledge, this is the largest longitudinal analysis of EBP fidelity in a child welfare program to date, and our findings support the notion that intensive coaching supports which are titrated over time can be sufficient to ensure sustained high fidelity, at least in some cases. Further, these findings indicate that robust training and coaching processes can ensure high fidelity and fidelity growth even among providers with less positive attitudes toward EBP.

Nurturing Children's Mental Health Body and Soul: Confronting American Child Psychiatry's Racist Past to Reimagine Its Antiracist Future

This paper unpacks the legacy of racism and white supremacy in American child psychiatry, connecting them to current racist inequities, to reimagine an antiracist future for the profession, and to serve all children's mental health body and soul. History reveals how child psychiatry has neglected and even perpetuated the intergenerational trauma suffered by minoritized children and families. By refusing to confront racial injustice, it has centered on white children's protection and deleted their role in white supremacist violence. An antiracist future for the profession demands a profound historical reckoning and comprehensive reimagining, a process that this paper begins to unfold.

A mixed-methods study to understand the impact of practitioner and organizational factors on fidelity of a child maltreatment prevention intervention in community-based settings

It has been well-documented that the degree to which interventions are implemented with fidelity in typical service settings has varied. Frequently, interventions are developed and tested in highly controlled or early adopter settings. Less attention has been given to what implementation looks like in usual care, and which factors promote practitioners' ability to implement with fidelity. Individuals and organizations implementing interventions in the real world receive varying levels of external supports and may apply a new intervention unaided. The purpose of this mixed-methods study was to explore factors that support implementation as intended in local community agencies. In the quantitative phase of this study, 32 case planners implementing Family Connections (FC), a child maltreatment preventive intervention, completed a survey about their perceptions of practitioner and organizational factors related to fidelity. The survey data were connected to case-level fidelity scores to understand the relationship between perceptions and fidelity. The qualitative phase of this study involved further exploration with nine case planner interviews and two separate focus groups with supervisors and agency leadership. The results of this study suggest that supervision is a key contributor to a practitioner's ability to implement an intervention in usual care. The quantitative and qualitative results suggest supervision, including supervisors’ perseverance, proactiveness, knowledge, availability, and skill reinforcement are important components of enhancing a practitioners' ability to learn and use FC. The quantitative results suggest that the level of education was positively associated with fidelity and perceptions of the intervention's limitations may be negatively related to implementation. Additional components that influence implementation for future research emerged from the qualitative phase related to system expectations and policies, individual practitioner attributes, and characteristics of the intervention.

Plain Language Abstract

This mixed-methods study sought to understand the impact of practitioner and organizational factors on fidelity of a child maltreatment prevention intervention in community-based settings. The study first asked case planners about their perceptions of practitioner and organizational factors related to fidelity through an online survey. This survey was connected to case-level fidelity scores to understand the relationship between perceptions and fidelity. The qualitative phase of this study involved further exploration with nine case planner interviews and two separate focus groups with supervisors and agency leadership. The results of this study suggest that supervision is a key contributor to a practitioner’s ability to implement a maltreatment prevention intervention. Both methods of the study suggest that various aspects of supervision, including supervisors’ perseverance, proactiveness, knowledge, availability, and skill reinforcement are important components of enhancing a practitioner's ability to learn and use the intervention. Additional components that influence the implementation for future research emerged from the qualitative phase related to system expectations and policies, individual practitioner attributes, and characteristics of the intervention.

An equivalence analysis of provider education in youth mental health care

OBJECTIVE

To test for equivalence between providers with and without advanced degrees in multiple domains related to delivery of evidence-based treatment.

DATA SOURCE

Provider and client data from an effectiveness trial of Alternatives for Families: A Cognitive Behavioral Therapy (AF-CBT) in a major metropolitan area in the United States.

STUDY DESIGN

We tested for equivalence between providers (N = 182) with and without advanced degrees in treatment-related knowledge, practices, and attitudes; job demands and stress; and training engagement and trainer-rated competence in AF-CBT. We also conducted exploratory analyses to test for equivalence in family clinical outcomes.

DATA COLLECTION

Providers completed measures prior to randomization and at 6-month follow-up, after completion of training and consultation in AF-CBT. Children and caregivers completed assessments at 0, 6, 12, and 18 months.

PRINCIPAL FINDINGS

Providers without advanced degrees were largely non-inferior to those with advanced degrees in treatment-related knowledge, practices, and attitudes, while findings for job demands and stress were mixed. Providers without advanced degrees were non-inferior to providers with advanced degrees in consultation attendance (B = -1.42; confidence interval (CI) = -3.01-0.16; margin of equivalence (Δ) = 2), number of case presentations (B = 0.64; CI = -0.49-1.76; Δ = 2), total training hours (B = -4.57; CI = -10.52-1.37; Δ = 3), and trainer-rated competence in AF-CBT (B = -0.04; CI = -3.04-2.96; Δ = 4), and they were significantly more likely to complete training (odds ratio = 0.66; CI = 0.10-0.96; Δ = 30%). Results for clinical outcomes were largely inconclusive.

CONCLUSIONS

Provider-level outcomes for those with and without advanced degrees were generally comparable. Additional research is needed to examine equivalence in clinical outcomes. Expanding evidence-based treatment training to individuals without advanced degrees may help to reduce workforce shortages and improve reach of evidence-based treatments.

The Association Between Clinician and Perceived Organizational Factors with Early Fidelity to Cognitive Processing Therapy for Posttraumatic Stress Disorder in a Randomized Controlled Implementation Trial

A common metric for determining implementation success is the measurement of clinician adherence to, and competence in, delivering a psychotherapy. The present study examined clinician and organizational factors as predictors of early adherence and competence among 78 clinicians delivering cognitive processing therapy (CPT), an evidence-based psychotherapy (EBP) for posttraumatic stress disorder, in a randomized controlled implementation trial. Results indicated that clinicians' willingness to adopt an EBP if required to do so was significantly associated with early adherence and competence in CPT delivery. Level of clinician education was significantly associated with early competence in delivering CPT. Organizational factors did not predict early adherence or competence. Implications of the findings are discussed.

Effect of the SafeCare© intervention on parenting outcomes among parents in child welfare systems: A cluster randomized trial

Child maltreatment has long-lasting negative impacts, and interventions are needed to improve caregiver's parenting skills to prevent maltreatment. This paper reports on a randomized trial comparing the SafeCare© model to services as usual (SAU) for child-welfare referred caregivers. SafeCare is an 18-session behavioral parenting program that teaches skills in positive parent-child interactions, home safety, and child health. SAU is generally unstructured and includes support, crisis management, referrals for need, and parenting education. Teams of providers at nine sites were randomized to implement SafeCare (19 teams; 119 providers) or continue SAU (17 teams; 118 providers). Two-hundred eighty eight caregivers (193 SafeCare; 95 SAU) with children aged 0-5 who were receiving services agreed to complete a baseline and 6-month assessment. Assessments measured positive parenting behaviors, parenting stress, protective factors, and neglectful behaviors using validated scales. Participants were primarily white (74.6%), female (87.0%), and low-income (68.6%), and had a mean age of 29. Latent change score models (LCSM) using a sandwich estimator consistent with the trial design were used to examine changes in 13 outcomes. Results indicated that SafeCare had small to medium effects for improving several parenting outcomes including supporting positive child behaviors (d = 0.46), proactive parenting (d = 0.25), and two aspects of parenting stress (d = 0.28 and .30). No differential change between groups was found for other indicators, including all indicators of neglect. Parenting programs such as SafeCare offer a promising mode of intervention for child welfare systems. Scale-up of parenting programs can improve parenting, improve child outcomes, and potentially reduce maltreatment. CLINICALTRIAL.GOV REGISTRATION NUMBER: NCT02549287.

"Creative Financing": Funding Evidence-Based Interventions in Human Service Systems

Implementation and sustainment of evidence-based interventions (EBIs) is influenced by outer (e.g., broader environments in which organizations operate) and inner (e.g., organizations, their administrators, and staff) contexts. One important outer-context element that shapes the inner context is funding, which is complex and unpredictable. There is a dearth of knowledge on how funding arrangements affect sustainment of EBIs in human service systems and the organizations delivering them, including child welfare and behavioral health agencies. This study uses qualitative interview and focus group data with stakeholders at the system, organizational, and provider levels from 11 human service systems in two states to examine how stakeholders strategically negotiate diverse and shifting funding arrangements over time. Study findings indicate that, while diverse funding streams may contribute to flexibility of organizations and possible transformations in the human service delivery environment, a dedicated funding source for EBIs is crucial to their successful implementation and sustainment.

AF-CBT for Families Experiencing Physical Aggression or Abuse Served by the Mental Health or Child Welfare System: An Effectiveness Trial

The Partnerships for Families project is a randomized clinical trial to evaluate the effectiveness of Alternatives for Families: A Cognitive Behavioral Therapy (AF-CBT), an evidence-based treatment (EBT) for families who are at risk of or have histories of child physical abuse. Across 10 agencies whose programs were supported by referrals from the mental health or child welfare system, individual providers were randomized to receive AF-CBT training ( n = 90) in a 6-month learning community or treatment as usual (TAU; n = 92) which provided trainings per agency routine. We recruited families served by providers in the AF-CBT ( n = 122) and TAU ( n = 73) conditions and collected multiple outcomes at up to four time points (0, 6, 12, and 18 months). Using univariate tests and growth curve models, the analyses revealed that AF-CBT (vs. TAU) showed improvements in both service systems (e.g., abuse risk, family dysfunction) or one service system (e.g., threats of force, child to parent minor assault), with some outcomes showing no improvement (e.g., parental anger). These findings are discussed in relation to AF-CBT, service system, provider, and family characteristics, and training/dissemination methods that affect the delivery of an EBT for this population in community settings.

The good coach: implementation and sustainment factors that affect coaching as evidence-based intervention fidelity support

PURPOSE

Evidence-based interventions (EBIs) for human services unfold within complicated social and organizational circumstances and are influenced by the attitudes and behaviors of diverse stakeholders situated within these environments. Coaching is commonly regarded as an effective strategy to support service providers in delivering EBIs and attaining high levels of fidelity over time. The purpose of this paper is to address a lacuna in research examining the factors influencing coaching, an important EBI support component.

METHODOLOGY

The authors use the Exploration, Preparation, Implementation, and Sustainment framework to consider inner- and outer-context factors that affect coaching over time. This case study of coaching draws from a larger qualitative data set from three iterative investigations of implementation and sustainment of a home visitation program, SafeCare®. SafeCare is an EBI designed to reduce child neglect.

FINDINGS

The authors elaborate on six major categories of findings derived from an iterative data coding and analysis process: perceptions of "good" and "bad" coaches by system sustainment status; coach as peer; in-house coaching capacity; intervention developer requirements vs other outer-context needs; outer- context support; and inner-context support.

PRACTICAL IMPLICATIONS

Coaching is considered a key component for effective implementation of EBIs in public-sector systems, yet is under-studied. Understanding inner- and outer-context factors illuminates the ways they affect the capacity of coaches to support service delivery.

ORIGINALITY

This paper demonstrates that coaching can accomplish more than provision of EBI fidelity support. Stakeholders characterized coaches as operating as boundary spanners who link inner and outer contexts to enable EBI implementation and sustainment.

A Technology-Mediated Approach to the Implementation of an Evidence-Based Child Maltreatment Prevention Program

Implementation of evidence-based parenting programs is critical for parents at-risk for child maltreatment perpetration; however, widespread use of effective programs is limited in both child welfare and prevention settings. This exploratory study sought to examine whether a technology-mediated approach to SafeCare^® delivery can feasibly assist newly trained providers in achieving successful implementation outcomes. Thirty-one providers working in child welfare or high-risk prevention settings were randomized to either SafeCare Implementation with Technology-Assistance (SC-TA) or SafeCare Implementation as Usual (SC-IU). SC-TA providers used a web-based program during session that provided video-based psychoeducation and modeling directly to parents and overall session guidance to providers. Implementation outcome data were collected from providers for six months. Data strongly supported the feasibility of SC-TA. Further, data indicated that SC-TA providers spent significantly less time on several activities in preparation, during, and in follow-up to SafeCare sessions compared to SC-IU providers. No differences were found between the groups with regard to SafeCare fidelity and certification status. Findings suggest that technology can augment implementation by reducing the time and training burden associated with implementing new evidence-based practices for at-risk families.

Organizational theory for dissemination and implementation research

Background

Even under optimal internal organizational conditions, implementation can be undermined by changes in organizations’ external environments, such as fluctuations in funding, adjustments in contracting practices, new technology, new legislation, changes in clinical practice guidelines and recommendations, or other environmental shifts. Internal organizational conditions are increasingly reflected in implementation frameworks, but nuanced explanations of how organizations’ external environments influence implementation success are lacking in implementation research. Organizational theories offer implementation researchers a host of existing, highly relevant, and heretofore largely untapped explanations of the complex interaction between organizations and their environment. In this paper, we demonstrate the utility of organizational theories for implementation research.

Discussion

We applied four well-known organizational theories (institutional theory, transaction cost economics, contingency theories, and resource dependency theory) to published descriptions of efforts to implement SafeCare, an evidence-based practice for preventing child abuse and neglect. Transaction cost economics theory explained how frequent, uncertain processes for contracting for SafeCare may have generated inefficiencies and thus compromised implementation among private child welfare organizations. Institutional theory explained how child welfare systems may have been motivated to implement SafeCare because doing so aligned with expectations of key stakeholders within child welfare systems’ professional communities. Contingency theories explained how efforts such as interagency collaborative teams promoted SafeCare implementation by facilitating adaptation to child welfare agencies’ internal and external contexts. Resource dependency theory (RDT) explained how interagency relationships, supported by contracts, memoranda of understanding, and negotiations, facilitated SafeCare implementation by balancing autonomy and dependence on funding agencies and SafeCare developers.

Summary

In addition to the retrospective application of organizational theories demonstrated above, we advocate for the proactive use of organizational theories to design implementation research. For example, implementation strategies should be selected to minimize transaction costs, promote and maintain congruence between organizations’ dynamic internal and external contexts over time, and simultaneously attend to organizations’ financial needs while preserving their autonomy. We describe implications of applying organizational theory in implementation research for implementation strategies, the evaluation of implementation efforts, measurement, research design, theory, and practice. We also offer guidance to implementation researchers for applying organizational theory.

EBT Fidelity Trajectories Across Training Cohorts Using the Interagency Collaborative Team Strategy

The Interdisciplinary Collaborative Team (ICT) strategy uses front-line providers as adaptation, training and quality control agents for multi-agency EBT implementation. This study tests whether an ICT transmits fidelity to subsequent provider cohorts. SafeCare was implemented by home visitors from multiple community-based agencies contracting with child welfare. Client-reported fidelity trajectories for 5,769 visits, 957 clients and 45 providers were compared using three-level growth models. Provider cohorts trained and live-coached by the ICT attained benchmark fidelity after 12 weeks, and this was sustained. Hispanic clients reported high cultural competency, supporting a cultural adaptation crafted by the ICT.

The what, when, and why of implementation frameworks for evidence-based practices in child welfare and child mental health service systems

It is widely recognized that children in the child welfare system are particularly vulnerable to the adverse health and mental effects associated with exposure to abuse and neglect, making it imperative to have broad-based availability of evidence-based practices (EBPs) that can prevent child maltreatment and reduce the negative mental health outcomes for youth who are victims. A variety of EBPs exist for reducing child maltreatment risk and addressing the associated negative mental health outcomes, but the reach of these practices is limited. An emerging literature documents factors that can enhance or inhibit the success of EBP implementation in community service agencies, including how the selection of a theory-driven conceptual framework, or model, might facilitate implementation planning by providing guidance for best practices during implementation phases. However, limited research is available to guide decision makers in the selection of implementation frameworks that can boost implementation success for EBPs that focus on preventing child welfare recidivism and serving the mental health needs of maltreated youth. The aims of this conceptual paper are to (1) provide an overview of existing implementation frameworks, beginning with a discussion of definitional issues and the selection criteria for frameworks included in the review; and (2) offer recommendations for practice and policy as applicable for professionals and systems serving victims of child maltreatment and their families.