Reliability, Validity, and Factor Structure of the Current Assessment Practice Evaluation-Revised (CAPER) in a National Sample

Family-Centered Care in Adolescent Intensive Outpatient Mental Health Treatment in the United States: A Case Study

BACKGROUND

Social isolation, national turmoil, and an adolescent mental health crisis in the wake of the COVID-19 pandemic have resulted in a significant uptick in inpatient admissions and re-admissions for high-risk patients. This trend persists even as the pandemic wanes. Intensive outpatient programs (IOPs) serve as a critical steppingstone between the community and inpatient mental health services, providing comprehensive psychiatric care for at-risk youth. Significant research has identified family engagement as a key element of successful collaborative care in adolescents.

OBJECTIVES

This article provides models of family-centered care in the adolescent IOP through a case study detailing the six-week course of care of an adolescent struggling with increased emotionality and distress intolerance in the context of family conflicts.

METHODS

This case highlights five family engagement components, including (1) family-centered psychiatric medication management, (2) individualized case management, parental education, and peer support, (3) Measurement Based Care (MBC) family assessment and feedback sessions, (4) Dialectical Behavior Therapy (DBT) multi-family skill groups, and (5) Compassionate Home Action Together (CHATogether) family intervention to address teen-parent relational health and communication.

RESULTS

This case showed improvement in depressive and anxiety symptoms, family conflict behaviors, self-reported suicide risk, and help-seeking attitudes towards parents/adults. The case family, along with others (n = 26), endorsed the parent peer support groups' acceptability and feasibility implemented in the adolescent IOP.

CONCLUSIONS

This article emphasizes the importance of family engagement during clinical care and provides a practical guide to implement collaborative family-centered therapeutic interventions in intensive outpatient services.

Adolescent Client and Clinician Predictors of Measurement-Based Care Fidelity in Community Mental Health Settings

OBJECTIVE

Regularly administering outcome measures, measurement-based care (MBC), informs clinical decision-making and improves youth mental health. Understanding predictors of high-fidelity MBC delivery helps ensure all youth can benefit from this evidence-based practice. Research on client and clinician predictors of MBC fidelity has mixed findings.

METHOD

Participants included 53 clinicians and 115 adolescents from a randomized controlled effectiveness trial with MBC only and MBC + Unified Protocol for the Transdiagnostic Treatment of Emotional Disorders in Adolescents (UPA) conditions. Clients and clinicians reported demographic information. Clinicians completed broad- and narrow-band attitude measures. MBC fidelity was operationalized using the implementation index, combining rates of administering and viewing questionnaires, using objective MBC data.

RESULTS AND DISCUSSION

The average MBC fidelity was 56.75% (SD = 30.36) and was significantly higher in the MBC only condition (M = 67.46%, SD = 27.63%) than in MBC + UPA condition (M = 46.58%, SD = 29.52%; p = .012). Clients identifying as White received significantly higher MBC fidelity than youth not identifying as White. Youth ethnicity, family income, age, severity, and number of sessions did not significantly predict MBC fidelity. Clinicians with less experience at their agency, less confidence, and who found manualized treatments less appealing, had significantly higher MBC fidelity than other clinicians. These results can inform future efforts to increase fidelity and equity in MBC delivery.

A psychometric assessment for evaluating spiritual care competency for geriatric nurses

This study evaluates the validity and reliability of the Chinese Version of Spiritual Care Competency (C-SCC) for geriatric nurses. Using convenience and snowball sampling, 1,094 geriatric nurses were randomly divided into two groups of 547 each for exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). EFA confirmed a four-dimensional structure, explaining 74.399 % of the total variance. Structural validity was assessed through EFA and CFA, while correlation analysis evaluated concurrent and convergent validity. Internal consistency, measured by Cronbach's α coefficient, was 0.987, with individual factors scoring 0.983, 0.964, 0.933, and 0.976. Test-retest reliability was 0.983, and concurrent validity was 0.860 (P < 0.01). Modified CFA results (RMSEA = 0.04, GFI = 0.93, AGFI = 0.93, NFI = 0.93, RFI = 0.93, PGFI = 0.62) indicated a well-fitting model with excellent psychometric properties. The C-SCC is a reliable tool for assessing spiritual care competencies among geriatric nurses.

Provider Use of, Attitudes Towards, and Self-efficacy with Key Measurement-based Care Practices in Youth Mental Health Treatment: A Multi-site Examination

Measurement-based care (MBC) is an underutilized evidence-based practice, and current implementation efforts demonstrate limited success in increasing MBC use. A better understanding of MBC implementation determinants is needed to improve these efforts, particularly from studies examining the full range of MBC practices and that span multiple samples of diverse providers using different MBC systems. This study addressed these limitations by conducting a multi-site survey examining MBC predictors and use in youth treatment. Participants were 159 clinicians and care coordinators working in youth mental health care settings across the United States. Participants were drawn from three program evaluations of MBC implementation. Providers completed measures assessing use of five MBC practices (administering measures, viewing feedback, reviewing feedback in supervision, sharing feedback with clients in session, and using feedback to plan treatment), MBC self-efficacy, and MBC attitudes. Despite expectations that MBC should be standard care for all clients, providers reported only administering measures to 40-60% of clients on average, with practices related to the use of feedback falling in the 1-39% range. Higher MBC self-efficacy and more positive views of MBC practicality predicted higher MBC use, although other attitude measures were not significant predictors. Effects of predictors were not moderated by site, suggesting consistent predictors across implementation settings. Implications of study findings for future research and for the implementation of MBC are discussed.

Consultation as an Implementation Strategy to Increase Fidelity of Measurement-Based Care Delivery in Community Mental Health Settings: An Observational Study

Regularly administering outcome measures to clients to inform clinical decision making, referred to as measurement-based care (MBC), has the potential to improve mental health treatment due to its applicability across disorders and treatment settings. Given the utility of MBC, understanding predictors of high-fidelity MBC implementation is important. Training and consultation are widely used implementation strategies to increase the fidelity of evidence-based practice delivery but have rarely been studied with MBC. The current study will examine the relationship between time clinicians spent discussing a case in consultation ("dosage") and MBC fidelity. Thirty clinicians and 56 youth were in the MBC condition of a randomized controlled trial and completed baseline questionnaires. Consultation dosage was extracted from call notes. MBC fidelity was measured using the implementation index, which combines rates of administering and viewing questionnaires, using objective data from the online MBC system. Multi-level modeling was used. Greater consultation dosage significantly predicted a higher implementation index (ß = 0.27, SE = 0.06, p < .001). For every 30 min spent discussing a case in consultation, the case's MBC fidelity increased by 8.1%. Greater consultation dosage significantly predicted higher rates of administration (ß = 0.07, SE = 0.03, p = 0.033) and feedback report viewing (ß = 0.24, SE = 0.06, p < .001). More consultation at the case level predicts greater MBC fidelity using objective measures. The greatest impact was increasing the clinician feedback report viewing rate. The results of this study can help inform future efforts to increase the fidelity with which MBC is delivered and to make consultation as efficient and effective as possible.

Organization-level variation in therapists' attitudes toward and use of measurement-based care

PURPOSE

Despite significant interest in improving behavioral health therapists' implementation of measurement-based care (MBC)-and widespread acknowledgment of the potential importance of organization-level determinants-little is known about the extent to which therapists' use of, and attitudes toward, MBC vary across and within provider organizations or the multilevel factors that predict this variation.

METHODS

Data were collected from 177 therapists delivering psychotherapy to youth in 21 specialty outpatient clinics in the USA. Primary outcomes were use of MBC for progress monitoring and treatment modification, measured by the nationally-normed Current Assessment of Practice Evaluation-Revised. Secondary outcomes were therapist attitudes towards MBC. Linear multilevel regression models tested the association of theory-informed clinic and therapist characteristics with these outcomes.

RESULTS

Use of MBC varied significantly across clinics, with means on progress monitoring ranging from values at the 25th to 93rd percentiles and means on treatment modification ranging from the 18th to 71st percentiles. At the clinic level, the most robust predictor of both outcomes was clinic climate for evidence-based practice implementation; at the therapist level, the most robust predictors were: attitudes regarding practicality, exposure to MBC in graduate training, and prior experience with MBC. Attitudes were most consistently related to clinic climate for evidence-based practice implementation, exposure to MBC in graduate training, and prior experience with MBC.

CONCLUSIONS

There is important variation in therapists' attitudes toward and use of MBC across clinics. Implementation strategies that target clinic climate for evidence-based practice implementation, graduate training, and practicality may enhance MBC implementation in behavioral health.

Characterizing measurement-based care implementation using therapist report

The effectiveness of measurement-based care (MBC), an evidence-based practice that uses regularly collected assessment data to guide clinical decision-making, is impacted by whether and how therapists use information from MBC tools in treatment. Improved characterization of how therapists use MBC in treatment sessions with youth is needed to guide implementation and understand variability in MBC effectiveness. To meet this need, this study examined therapists' sharing and discussion of MBC in treatment sessions. Thirty therapists were randomly assigned to the MBC condition as part of a comparative effectiveness trail of treatments for adolescent anxiety and depression. A qualitative content analysis was conducted on therapists' written explanations of changes made to the session based on the MBC data. Therapists reported sharing data with youth and caregivers in an average of 34.6% and 27.4% of sessions, respectively. Therapists reported incorporating MBC data in an average of 21.1% of sessions. When data were used, therapists predominately focused changes on short-term (e.g., current symptoms, treatment skill) rather than long-term (e.g., symptom progress, treatment goals) decision-making. Therapists inconsistently used MBC data, highlighting the need for improved training in and monitoring of how therapists use MBC in session to guide collaborative treatment decision-making with youth and caregivers.

How low can you go? Examining the effects of brief online training and post-training consultation dose on implementation mechanisms and outcomes for measurement-based care

BACKGROUND

Initial training and ongoing post-training consultation (i.e., ongoing support following training, provided by an expert) are among the most common implementation strategies used to change clinician practice. However, extant research has not experimentally investigated the optimal dosages of consultation necessary to produce desired outcomes. Moreover, the degree to which training and consultation engage theoretical implementation mechanisms-such as provider knowledge, skills, and attitudes-is not well understood. This study examined the effects of a brief online training and varying dosages of post-training consultation (BOLT+PTC) on implementation mechanisms and outcomes for measurement-based care (MBC) practices delivered in the context of education sector mental health services.

METHODS

A national sample of 75 clinicians who provide mental health interventions to children and adolescents in schools were randomly assigned to BOLT+PTC or control (services as usual). Those in BOLT+PTC were further randomized to 2-, 4-, or 8-week consultation conditions. Self-reported MBC knowledge, skills, attitudes, and use (including standardized assessment, individualized assessment, and assessment-informed treatment modification) were collected for 32 weeks. Multilevel models were used to examine main effects of BOLT+PTC versus control on MBC use at the end of consultation and over time, as well as comparisons among PTC dosage conditions and theorized mechanisms (skills, attitudes, knowledge).

RESULTS

There was a significant linear effect of BOLT+PTC over time on standardized assessment use (b = .02, p < .01), and a significant quadratic effect of BOLT+PTC over time on individualized assessment use (b = .04, p < .001), but no significant effect on treatment modification. BOLT + any level of PTC resulted in higher MBC knowledge and larger growth in MBC skill over the intervention period as compared to control. PTC dosage levels were inconsistently predictive of outcomes, providing no clear evidence for added benefit of higher PTC dosage.

CONCLUSIONS

Online training and consultation in MBC had effects on standardized and individualized assessment use among clinicians as compared to services as usual with no consistent benefit detected for increased consultation dosage. Continued research investigating optimal dosages and mechanisms of these established implementation strategies is needed to ensure training and consultation resources are deployed efficiently to impact clinician practices.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05041517 . Retrospectively registered on 10 September 2021.

Implementation strategies to promote measurement-based care in schools: evidence from mental health experts across the USA

BACKGROUND

Despite an established taxonomy of implementation strategies, minimal guidance exists for how to select and tailor strategies to specific practices and contexts. We employed a replicable method to obtain stakeholder perceptions of the most feasible and important implementation strategies to increase mental health providers' use of measurement-based care (MBC) in schools. MBC is the routine use of patient-reported progress measures throughout treatment to inform patient-centered, data-driven treatment adjustments.

METHODS

A national sample of 52 school mental health providers and researchers completed two rounds of modified Delphi surveys to rate the relevance, importance, and feasibility of 33 implementation strategies identified for school settings. Strategies were reduced and definitions refined using a multimethod approach. Final importance and feasibility ratings were plotted on "go-zone" graphs and compared across providers and researchers to identify top-rated strategies.

RESULTS

The initial 33 strategies were rated as "relevant" or "relevant with changes" to MBC in schools. Importance and feasibility ratings were high overall for both survey rounds; on a scale of 1 to 5, importance ratings (3.61-4.48) were higher than feasibility ratings (2.55-4.06) on average. Survey 1 responses resulted in a reduced, refined set of 21 strategies, and six were rated most important and feasible on Survey 2: (1) assess for readiness and identify barriers and facilitators; (2) identify and prepare champions; (3) develop a usable implementation plan; (4) offer a provider-informed menu of free, brief measures; (5) develop and provide access to training materials; and (6) make implementation easier by removing burdensome documentation tasks. Provider and researcher ratings were not significantly different, with a few exceptions: providers reported higher feasibility and importance of removing burdensome paperwork than researchers, providers reported higher feasibility of train-the trainer approaches than researchers, and researchers reported higher importance of monitoring fidelity than providers.

CONCLUSIONS

The education sector is the most common setting for child and adolescent mental health service delivery in the USA. Effective MBC implementation in schools has the potential to elevate the quality of care received by many children, adolescents, and their families. This empirically derived, targeted list of six implementation strategies offers potential efficiencies for future testing of MBC implementation in schools.

Drug and alcohol practitioners' attitudes toward the use of standardized assessment

Despite their importance to evidence-based assessment, standardized assessments remain underutilized by mental health practitioners in practice. The underutilization has been attributed to a lack of appreciation of the importance of standardized assessments, lack of knowledge of standardized assessments, and practical barriers to implementation. This study sought to gather the first descriptive data on alcohol and other drug (AOD) practitioners' attitudes toward, and knowledge and self-reported use of, standardized assessments. Practical barriers to implementation in initial assessment and progress monitoring were also assessed. Ninety-nine Australian AOD practitioners recruited via newsletters of national representative bodies and practitioner networks completed an online survey. While practitioners' attitudes towards using standardized assessments for initial assessment and progress monitoring were generally positive and consistent with other populations of health practitioners, assessments remained underutilized in practice. Most AOD practitioners did not consider standardized assessments to be feasible to implement. The current findings highlight the importance of practical barriers, particularly organization-level barriers, in the underutilization of standardized assessments in AOD practice. Findings support an extension of dialogue surrounding evidence-based practice beyond treatment selection to include assessment practices at a more general level. The present study offers a starting point from which efforts to improve practitioner compliance with evidence-based best practices can be conceived, designed, and implemented.

A Roadmap for Measurement-based Care Implementation in Intensive Outpatient Treatment Settings for Children and Adolescents

Measurement-based care (MBC), the routine collection and use of patient-reported data to monitor progress and tailor treatment, has been predominantly studied in adult treatment settings. Although growing evidence supports MBC effectiveness with youth in outpatient settings and university training clinics, there is a substantial dearth of findings about successful implementation of MBC in "real world" youth treatment settings, particularly intensive settings offering group-based treatment. The current manuscript provides a foundational model of MBC implementation for "real world" intensive outpatient programs (IOP) for youth using the organizational framework of the Consolidated Framework for Implementation Research (CFIR). We also illustrate MBC implementation within a hospital-based adolescent psychiatric IOP, including enhancements to the foundational model and timely discussion of adjustments necessitated by the COVID-19 pandemic and transition to telehealth. Given the promising transdiagnostic and transtheoretical applicability of MBC, coupled with the MBC mandate for Joint Commission accredited health-care systems, IOP programs are well-positioned to adopt, implement and sustain MBC with careful attention to a phased, multilevel implementation approach.

Investigating the compatibility, interchangeability, and clinical usability of BOT2-BF and BOT2-SF in primary school-aged children with and without developmental dyslexia

The aim of our study was to determine the sensitivity and specificity levels of BOT2-BF and BOT2-SF tests applied to children with developmental dyslexia and to examine their clinical usability. This study was designed as a case-controlled study. 82 (study group 41, control group 41) 7-12 year children were included. The study occurred three stages. In the first place, the similarities and differences of BOT2-BF and BOT2-SF tests observed in clinical practices were examined by taking the feedback of experts and test correlations. In the second step, BOT2-BF and BOT2-SF tests were examined with reference to the Core Outcome Set (COS) evaluation criteria. In the last step, in order to examine the sensitivity and specificity levels of the tests, assessments were performed in two groups with 1-week intervals. According to the COS results, the BOT2-BF test was examined better than the BOT2-SF test under cross-cultural validity title. The sensitivity and specificity of BOT2-SF were 0.92 and 0.51, and for BOT2-BF were 0.92 and 0.41, respectively. This study will help clinicians working on children with developmental dyslexia to decide the type of test they will use to evaluate and easy access to materials motor proficiency.

Factors affecting the implementation of an outcome measurement feedback system in youth mental health settings

Objective: Measurement feedback systems provide clinicians with regular snapshots of a client's mental health status, which can be used in treatment planning and client feedback. There are numerous barriers to clinicians using outcome measures routinely. This study aimed to investigate factors affecting the use of a measurement feedback system across youth mental health settings. Methods: The participants were 210 clinicians from headspace youth mental health services across Australia. They were surveyed on predictors and use of MyLifeTracker, a routine outcome measure. This was explored through three processes: looking at MyLifeTracker before session, using MyLifeTracker in treatment planning, and providing feedback of MyLifeTracker scores to clients. Results: Clinicians were more likely to look at MyLifeTracker before session, less likely to use it in treatment planning, and least likely to provide MyLifeTracker scores to clients. Each measurement feedback system process had a distinct group of predictors. Perceptions of MyLifeTracker's practicality was the only significant predictor of all three processes. Conclusion: Practically, organisations and supervisors can increase the use of measurement feedback systems through targeted supports.

Improving measurement feedback systems for measurement-based care

Abstract Objectives: Measurement feedback systems (MFS) are a class of health information technologies developed to facilitate measurement-based care. The individual clinical decision support features within MFS are diverse and their influence on clinicians is largely unknown. This study tested the impact of MFS features on clinicians' progress assessments and treatment decisions in different scenarios. Method: Clinicians (n = 299) were randomly assigned to view one of six combinations of the following MFS features: graph, expected change trajectory line, alert, and treatment suggestions. The assigned feature combination was paired with three vignettes and clinical data representing three clinical scenarios: patient deterioration, no progress, and approaching remission. Clinicians answered questions after each vignette, and at the conclusion. Results: MFS features differentially impacted clinicians' progress assessment accuracy, their likelihood of making a treatment change, and their treatment choices. Which feature was most impactful varied depending on the clinical scenario. Clinicians reported graphs influenced their assessments and choices significantly more than the other features, and the majority stated they would prefer to use all of the features. Conclusions: Specific MFS features impact clinicians' assessments and choices to greater degrees, and the impact of those features can be influenced by the clinical state of the patient.

A Method for Tracking Implementation Strategies: An Exemplar Implementing Measurement-Based Care in Community Behavioral Health Clinics

Implementation experts suggest tailoring strategies to the intended context may enhance outcomes. However, it remains unclear which strategies are best suited to address specific barriers to implementation, in part because few measurement methods exist that adhere to recommendations for reporting. In the context of a dynamic cluster randomized trial comparing a standardized to tailored approach to implementing measurement-based care (MBC), this study aimed to (a) describe a method for tracking implementation strategies, (b) demonstrate the method by tracking strategies generated by teams tasked with implementing MBC at their clinics in the tailored condition, and (c) conduct preliminary examinations of the relation between strategy use and implementation outcomes (i.e., self-reported fidelity to MBC). The method consisted of a coding form based on Proctor, Powell, and McMillen (2013) implementation strategy reporting guidelines and Powell et al.'s (2012) taxonomy to facilitate specification of the strategies. A trained research specialist coded digitally recorded implementation team meetings. The method allowed for the following characterization of strategy use. Each site generated 39 unique strategies across an average of six meetings in five months. There was little variability in the use of types of implementation strategies across sites with the following order of prevalence: quality management (50.00%), restructuring (16.53%), communication (15.68%), education (8.90%), planning (7.20%), and financing (1.69%). We identified a new category of strategies not captured by the existing taxonomy, labeled "communication." There was no evidence that number of implementation strategies enacted was statistically significantly associated with changes in self-reported fidelity to MBC-however, financing strategies were associated with increased fidelity. This method has the capacity to yield rich data that will inform investigations into tailored implementation approaches.