Assessing stability between treatment planning documents in a system of care

Getting More Out of Clinical Documentation: Can Clinical Dashboards Yield Clinically Useful Information?

This study investigated coded data retrieved from clinical dashboards, which are decision-support tools that include a graphical display of clinical progress and clinical activities. Data were extracted from clinical dashboards representing 256 youth (M age = 11.9) from 128 practitioners who were trained in the Managing and Adapting Practice (MAP) system (Chorpita & Daleiden in BF Chorpita EL Daleiden 2014 Structuring the collaboration of science and service in pursuit of a shared vision. 43(2):323 338. 2014, Chorpita & Daleiden in BF Chorpita EL Daleiden 2018 Coordinated strategic action: Aspiring to wisdom in mental health service systems. 25(4):e12264. 2018) in 55 agencies across 5 regional mental health systems. Practitioners labeled up to 35 fields (i.e., descriptions of clinical activities), with the options of drawing from a controlled vocabulary or writing in a client-specific activity. Practitioners then noted when certain activities occurred during the episode of care. Fields from the extracted data were coded and reliability was assessed for Field Type, Practice Element Type, Target Area, and Audience (e.g., Caregiver Psychoeducation: Anxiety would be coded as Field Type = Practice Element; Practice Element Type = Psychoeducation; Target Area = Anxiety; Audience = Caregiver). Coders demonstrated moderate to almost perfect interrater reliability. On average, practitioners recorded two activities per session, and clients had 10 unique activities across all their sessions. Results from multilevel models showed that clinical activity characteristics and sessions accounted for the most variance in the occurrence, recurrence, and co-occurrence of clinical activities, with relatively less variance accounted for by practitioners, clients, and regional systems. Findings are consistent with patterns of practice reported in other studies and suggest that clinical dashboards may be a useful source of clinical information. More generally, the use of a controlled vocabulary for clinical activities appears to increase the retrievability and actionability of healthcare information and thus sets the stage for advancing the utility of clinical documentation.

Do Treatment Plans Matter? Moving From Recommendations to Action

We investigated whether a service-planning document outlining recommendations for what providers should address in treatment (i.e., targets) and the associated clinical techniques they should employ (i.e., practices) influenced the targets and practices that providers reported actually implementing during the subsequent treatment episode. Participants included 94 youths ages 4 to 17 (M = 13.57, SD = 3.59) who received community-based mental health services from the Hawai'i Child and Adolescent Mental Health Division. Data on targets and practices were compared across initial Mental Health Treatment Plans and Monthly Treatment and Progress Summaries. Data were analyzed using two-level, generalized mixed effects models with two-way cross-classification or linear mixed effects models. Providers were more likely to report the use of targets and practices in treatment if they were included within the treatment plan. In addition, the more closely targets addressed during treatment followed the recommended targets from the treatment plan, the more closely implemented practices followed the recommended practices listed in the treatment plan. Furthermore, as providers shifted their focus to different targets, a shift in their use of practices was also evident over time. Last, practices for which there is demonstrated efficacy for particular targets were more likely to be used. Service planning documents appear to help organize care; however, results also suggest possible limitations to the current system. These findings highlight potential areas for improvement in planning and care delivery.

Predictors and moderators of agreement between clinical and research diagnoses for children and adolescents

OBJECTIVE

Diagnoses play an important role in treatment planning and monitoring, but extensive research has shown low agreement between clinician-generated diagnoses and those from structured diagnostic interviews. However, most prior studies of agreement have not used research diagnoses based on gold standard methods, and researchers need to identify characteristics of diagnostically challenging clients. This study examined agreement between youth diagnoses generated through the research-based LEAD (Longitudinal, Expert, and All Data) standard to clinician diagnoses.

METHOD

Participants were 391 families seeking outpatient community mental health services for youths ages 6-18 (39.1% female, 88.2% African American). Youths and parents completed research interviews and clinic diagnoses were extracted from clinic records. LEAD diagnoses synthesized results of the Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime (KSADS-PL) and the youth's developmental, family, and psychiatric history.

RESULTS

Agreement between the LEAD and chart diagnoses was low, not exceeding "poor" agreement for most diagnostic categories (κs = .10-.46, median = .37). Disagreement was largely driven by missed diagnoses, although clinicians also did assign extra diagnoses for some clients. Fewer diagnostic errors occurred when the youth's clinical picture was more clear (e.g., high or low symptom severity, lower comorbidity), when the youth was older, when the family was higher functioning, and when the parent had more depression. However, youth and family characteristics explained very little of the variability in diagnostic errors.

CONCLUSIONS

RESULTS support the need to investigate strategies to improve clinician diagnostic accuracy.

From distal to proximal: Routine educational data monitoring in school-based mental health

Research and practice in school-based mental health (SBMH) typically includes educational variables only as distal outcomes, resulting from improvements in mental health symptoms rather than directly from mental health intervention. Although sometimes appropriate, this approach also has the potential to inhibit the integration of mental health and schools. The current paper applies an existing model of data-driven decision making (Daleiden & Chorpita, 2005) to detail how SBMH can better integrate routine monitoring of school and academic outcomes into four evidence bases: general services research evidence, case histories, local aggregate, and causal mechanisms. The importance of developing new consultation protocols specific to data-driven decision making in SBMH as well as supportive infrastructure (e.g., measurement feedback systems) to support the collection and use of educational data is also described.

Toward the effective and efficient measurement of implementation fidelity

Implementation science in mental health is informed by other academic disciplines and industries. Conceptual and methodological territory charted in psychotherapy research is pertinent to two elements of the conceptual model of implementation posited by Aarons and colleagues (2010)--implementation fidelity and innovation feedback systems. Key characteristics of scientifically validated fidelity instruments, and of the feasibility of their use in routine care, are presented. The challenges of ensuring fidelity measurement methods are both effective (scientifically validated) and efficient (feasible and useful in routine care) are identified as are examples of implementation research attempting to balance these attributes of fidelity measurement.