Complexity and potentials of clinical feedback in mental health: an in-depth study of patient processes

Patients' experiences with the routine use of a clinical feedback system prior to consultations in ostomy care: a qualitative study

PURPOSE

Ostomy surgery involves significant bodily changes, and the adjustment process encompasses a broad spectrum of physical and psychosocial challenges. A clinical feedback system (CFS) has been developed to collect patient-reported outcomes as part of routine outpatient follow-up, reviewed by stoma care nurses, to better address patients' needs during their adjustment process. The intervention appears promising; however, empirical evidence supporting its benefits remains limited. Thus, we explored patients' experiences with the routine use of the CFS prior to consultations in ostomy care.

METHODS

A qualitative design involved 27 semi-structured individual interviews with patients using CFS as part of routine care. The data were analysed using Braun and Clarke's reflexive approach to thematic analysis.

RESULTS

The overarching theme CFS-a tool with potential and multiple mechanisms of action was developed with three themes: (1) Grasping the purpose can be challenging, (2) Preparatory learning-triggering reflection and self-awareness, and (3) Means of communication and potential for being understood. Engaging with the CFS had personal utility value with many dimensions, varying in strength and significance for each patient. Even though not everyone grasped the purpose of its use, it was part of a preparatory learning process for consultations and the adjustment process itself. It triggered reflection and self-awareness and served as a means of communication with potential for follow-up.

CONCLUSION

Although the CFS appears promising, future research should focus on identifying the most effective methods for educating patients on its use.

A Digital Approach for Addressing Suicidal Ideation and Behaviors in Youth Mental Health Services: Observational Study

BACKGROUND

Long wait times for mental health treatments may cause delays in early detection and management of suicidal ideation and behaviors, which are crucial for effective mental health care and suicide prevention. The use of digital technology is a potential solution for prompt identification of youth with high suicidality.

OBJECTIVE

The primary aim of this study was to evaluate the use of a digital suicidality notification system designed to detect and respond to suicidal needs in youth mental health services. Second, the study aimed to characterize young people at different levels of suicidal ideation and behaviors.

METHODS

Young people aged between 16 and 25 years completed multidimensional assessments using a digital platform, collecting demographic, clinical, social, functional, and suicidality information. When the suicidality score exceeded a predetermined threshold, established based on clinical expertise and service policies, a rule-based algorithm configured within the platform immediately generated an alert for treating clinicians. Subsequent clinical actions and response times were analyzed.

RESULTS

A total of 2021 individuals participated, of whom 266 (11%) triggered one or more high suicidal ideation and behaviors notification. Of the 292 notifications generated, 76% (222/292) were resolved, with a median response time of 1.9 (range 0-50.8) days. Clinical actions initiated to address suicidality included creating safety plans (60%, 134/222), conducting safety checks (18%, 39/222), psychological therapy (8%, 17/222), transfer to another service (3%, 8/222), and scheduling of new appointments (2%, 4/222). Young people with high levels of suicidality were more likely to present with more severe and comorbid symptoms, including low engagement in work or education, heterogenous psychopathology, substance misuse, and recurrent illness.

CONCLUSIONS

The digital suicidality notification system facilitated prompt clinical actions by alerting clinicians to high levels of suicidal ideation and behaviors detected among youth. Further, the multidimensional assessment revealed complex and comorbid symptoms exhibited in youth with high suicidality. By expediting and personalizing care for those displaying elevated suicidality, the digital notification system can play a pivotal role in preventing rapid symptom progression and its detrimental impacts on young people's mental health.

Response processes for patients providing quantitative self-report data: a qualitative study

OBJECTIVE

To identify factors that influence response processes for patients providing quantitative self-report data. Secondly, due to the lack of integrative and explanatory models in this area, to develop a model of patients' response processes that can guide what to look for when considering validity evidence and interpreting scores on individual items.

METHODS

Participants (n = 13) were recruited from a specialized substance use disorder treatment clinic and interviewed while responding to items from a clinical feedback system implemented for routine outcome monitoring in that setting. The interview approach was based on cognitive interviewing. Data collection and analysis were inspired by a grounded theory approach.

RESULTS

We identified several variables that influenced the participants' response processes. The variables were organized into five categories: context-related variables; item-related variables; response base variables; reasoning strategies; and response selection strategies. We also found that the participants' responses for many items were affected by different aspects of the response process in ways that are relevant to interpretation but not necessarily discernible from the numerical scores alone, and we developed response categories to capture this.

CONCLUSION

The findings suggest that patients providing quantitative self-report data encounter conditions in the response process that challenge and influence their ability to convey meaning and accuracy. This results in responses that for many of the items reflect messages important for interpretation and follow-up, even if it does not appear from the numerical scores alone. The proposed model may be a useful tool when developing items, assessing validity, and interpreting responses.

Routine Outcome Monitoring and Clinical Feedback in Psychotherapy: Recent Advances and Future Directions

In the past decade, there has been an increase in research related to the routine collection and active use of standardized patient data in psychotherapy. Research has increasingly focused on personalization of care to patients, clinical skills and interventions that modulate treatment outcomes, and implementation strategies, all of which appear to enhance the beneficial effects of ROM and feedback. In this article, we summarize trends and recent advances in the research on this topic and identify several essential directions for the field in the short to medium term. We anticipate a broadening of research from the focus on average effects to greater specificity around what kinds of feedback, provided at what time, to which individuals, in what settings, are most beneficial. We also propose that the field needs to focus on issues of health equity, ensuring that ROM can be a vehicle for increased wellbeing for those who need it most. The complexity of mental healthcare systems means that there may be multiple viable measurement solutions with varying costs and benefits to diverse stakeholders in different treatment contexts, and research is needed to identify the most influential components in each of these contexts.

The Connections Matter: Bi-Directional Learning in Program Evaluation and Practice-Oriented Research

Intended for researchers and clinical leaders, this article suggests that embedded program evaluation is a good fit with the desired features of practice-oriented research. The systematic nature of evaluation that is built into the operational workflow of a practice setting may increase the diversity of methods available to explore processes and outcomes of interest. We propose a novel conceptual framework that uses a human-centered systems lens to foster such embedded evaluation in clinical routine. This approach emphasizes the evaluator-practitioner partnership to build confidence in the bi-directional learning of practice-based evidence with evidence-based practice. The iterative cycles inherent to design thinking are aimed at developing better evaluation questions. The attention to structure and context inherent to systems thinking is intended to support meaningful perspectives in the naturally complex world of health care. Importantly, the combined human-centered systems lens can create greater awareness of the influence of individual and systemic biases that exist in any endeavor or institution that involves people. Recommended tools and strategies include systems mapping, program theory development, and visual facilitation using a logic model to represent the complexity of mental health treatment for communication, shared understanding, and connection to the broader evidence base. To illustrate elements of the proposed conceptual framework, two case examples are drawn from routine outcome monitoring (ROM) and progress feedback. We conclude with questions for future collaboration and research that may strengthen the partnership of evaluators and practitioners as a community of learners in service of local and system-level improvement.

Smartphone-based serious games for mental health: a scoping review

The use of smartphone-based Serious Games in mental health care is an emerging and promising research field. Combining the intrinsic characteristics of games (e.g., interactiveness, immersiveness, playfulness, user-tailoring and engaging nature) with the capabilities of smartphones (e.g., versatility, ubiquitous connectivity, built-in sensors and anywhere-anytime nature) yields great potential to deliver innovative psychological treatments, which are engaging, effective, fun and always available. This article presents a scoping review, based on the PRISMA (scoping review extension) guidelines, of the field of smartphone-based serious games for mental health care. The review combines an analysis of the technical characteristics, including game design, smartphone and game-specific features, with psychological dimensions, including type and purpose of use, underlying psychological frameworks and strategies. It also explores the integration of psychological features into Serious Games and summarizes the findings of evaluations performed. A systematic search identified 40 smartphone-based Serious Games for mental health care. The majority consist of standalone and self-administrable interventions, applying a myriad of psychological strategies to address a wide range of psychological symptoms and disorders. The findings explore the potential of Serious Games as treatments and for enhancing patient engagement; we conclude by proposing several avenues for future research in order to identify best practices and success factors.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11042-024-18971-w.

Therapists' and patients' experiences of using patients' self-reported data in ongoing psychotherapy processes-A systematic review and meta-analysis of qualitative studies

OBJECTIVE

Using patient-generated quantitative data in psychotherapy (feedback) appears to enhance treatment outcome, but there is variability in its effect. Different ways and reasons to implement routine outcome measurement might explain such variability. The goal of this review is to address the insufficient knowledge on how these data are used by therapists and patients.

METHODS

The present study is a systematic review and meta-analysis of qualitative reports of therapists' and patients' experiences using patient-generated quantitative data during ongoing psychotherapy.

RESULTS

Four main categories of use were identified: (1) uses of patients' self-reported data as nomothetic/objective markers for assessment, process monitoring, and treatment planning; (2) intrapersonal uses that enhance self-awareness, initiate reflection, and influence patients' mood or responses; (3) uses that prompt interactional processes by facilitating communication, supporting exploration, creating ownership in patients, changing treatment focus, enhancing therapeutic alliance, or disturbing the psychotherapy process; and (4) patients responding for specific purposes due to uncertainty and interpersonal motives, or strategic responding to achieve a desired result.

CONCLUSION

These results demonstrate that patient-reported data, when used in active psychotherapy, is very clearly not just an objective measurement of client functioning: the inclusion of patient-data has the potential to influence psychotherapy in numerous ways.

Case report: A dual case study of how clinical feedback can be a communication aide and influence therapeutic work

Background

While routine outcome monitoring and clinical feedback may improve outcomes after psychotherapy, results from efficiency studies have been mixed. Moreover, how clinical feedback is implemented influences how it works for patients and clinicians, and working mechanisms are hitherto not thoroughly explored. Researchers have argued that inviting and using feedback from patients is best conceived of as a clinical skill. In this paper, we use case study methodology to explore and describe feedback's functions within three clinical skill themes: actualizing alliance work, concretizing change and stagnation and verbalizing the non-verbal.

Case presentation

Sonja is a young adult patient with a trauma background. She has a history of serious suicide attempts and distrust in relationships. She attended psychotherapy for eight months. Harald is a middle-aged man with a stable family. Traumatic events in his past has made him conceal own needs and developing depression. He attended psychotherapy for 19 months. Case material include the patient's clinical feedback over a range of life areas, medical health notes and the therapist's process notes.

Conclusion

Clinical feedback can be a positive supplement to the therapeutic work and process. The importance of making this as a joint tool between the client and the therapist is significant.

Routine Outcome Monitoring (ROM) and Feedback: Research Review and Recommendations

OBJECTIVE

To provide a research review of the components and outcomes of routine outcome monitoring (ROM) and recommendations for research and therapeutic practice.

METHOD

A narrative review of the three phases of ROM - data collection, feeding back data, and adapting therapy - and an overview of patient outcomes from 11 meta-analytic studies.

RESULTS

Patients support ROM when its purpose is clear and integrated within therapy. Greater frequency of data collection is more important for shorter-term therapies, and use of graphs, greater specificity of feedback, and alerts are helpful. Overall effects on patient outcomes are statistically significant (g ≈ 0.15) and increase when clinical support tools (CSTs) are used for not-on-track cases (g ≈ 0.36-0.53). Effects are additive to standard effects of psychological therapies. Organizational, personnel, and resource issues remain the greatest obstacles to the successful adoption of ROM.

CONCLUSION

ROM offers a low-cost method for enhancing patient outcomes, on average resulting in an ≈ 8% advantage (success rate difference; SRD) over standard care. CSTs are particularly effective for not-on-track patients (SRD between ≈ 20% and 29%), but ROM does not work for all patients and successful implementation is a major challenge, along with securing appropriate cultural adaptations.

Relationship and Alliance Formation Processes in Psychotherapy: A Dual-Perspective Qualitative Study

Objective

To explore how therapists and clients act dyadically to establish a therapeutic relationship during the first five sessions of psychotherapy. The study aimed to identify both relational facilitative and hindering processes occurring in routine care.

Methods

Using the method ‘interpersonal process recall’ (IPR), we videotaped the third and fifth session of 12 psychotherapy dyads, and conducted video-assisted interviews with each therapist and client separately. In total, the data material consist of 47 IPR interviews. Data were analyzed using a thematic approach.

Results

The analysis process revealed two main groups. The first group consisted of dyads with a positive relational outcome, and the second group consisted of dyads with a troubled or frail relational outcome. During the initial phase of therapy, clients described feeling overwhelmed by fear and shame. Positive relational development occurred when these emotions were successfully accommodated and replaced with a growing sense of safety with the therapist. However, the relationship became troubled when the client experienced an increase in shame and/or fear during the first sessions. When forming a therapeutic relationship, it is vital that the client experience the therapist as genuine and skilled, and that the therapist is able to engage and connect deeply with the client on a person-to-person level. The article further provides a discussion on how these dyadic experiences align with the working alliance and real relationship, and how the two consolidate during the first sessions of psychotherapy.

Conclusion

The current study explored the complex relational processes underlying the formation of the therapeutic relationship. Core aspects of the real relationship are prerequisites to forming a collaborative working alliance in which both therapist and client are actively engaged. Facilitating a positive relationship is crucial in the early phase of psychotherapy, and therapists can actively identify and repair ruptures at this time.

How therapists and patients need to develop a clinical feedback system after 18 months of use in a practice-research network: a qualitative study

BACKGROUND

A personalized computer-adaptive system for clinical feedback and routine outcome monitoring in mental health, Norse Feedback aims to bridge the needs for standardized and idiographic measures in ordinary practice.

METHODS

Item response theory analyses of completed treatment processes (n = 800) informed a qualitative study comprising individual in-depth interviews and focus groups with patients (n = 9) and clinicians (n = 10). The research question was: How do clinicians and patients contribute to developing a clinical feedback system in a continuous process aimed at refining its clinical usefulness?

RESULTS

We conducted thematic analyses and found five themes: 1. Added clinical needs, 2. Needs for re-organizing the clinician report, 3.Needs for differentiation of clinical content, 4. User-interface needs, and 5. Item level suggestions.

CONCLUSION

In this article, we detail resulting needs for continuous adaptation to practice, and discuss implications of the concrete experiences with the Norse action research program for the larger field of ROM/CFS implementation.

Initial quantitative development of the Norse Feedback system: a novel clinical feedback system for routine mental healthcare

Purpose

As routine outcome monitoring has become prevalent in psychological practice, there is need for measurement tools covering diverse symptoms, treatment processes, patient strengths, and risks. Here we describe the development and initial tests of the psychometric properties of a multi-scale system for use in mental healthcare, Norse Feedback.

Methods

In Study 1, we present the item-generation process and structure of the Norse Feedback, a 17-scale digital-first measurement tool for psychopathology and treatment-relevant variables. In Study 2, we present analyses of this initial measure in a nonclinical sample of 794 healthy controls and a sample of 222 mental health patients. In Study 3, we present the analysis of a revised 20-scale system in two separate samples of patients. In each analysis, we investigate item and test information in particular, including analysis of differential item functioning on gender, age, site, and sample differences where applicable.

Results

Scales performed variably. Changes to items and scales are described. Several scales appeared to reliably discriminate individuals entering mental health treatment on severity, and others are less reliable. Marked improvements in scale internal consistency and measurement precision were observed between the first and second implemented versions.

Conclusion

This system includes some scales with reasonable structural validity, though several areas for future development are identified. The system was developed to be iteratively re-evaluated, to strengthen the validity of its scales over time. There are currently a number of limitations on inferences from these scores, which future developments should address.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11136-021-02825-1.

Clinical dilemmas of routine outcome monitoring and clinical feedback: A qualitative study of patient experiences

ABSTRACT Purpose: Routine outcome monitoring (ROM) and clinical feedback systems (CFS) are becoming prevalent in mental health services, but there are several challenges to successful implementation. ROM/CFS seem to be helpful for some patients, but not for others. To investigate this, we explored patients' experiences with ROM/CFS as an interpersonal and psychotherapeutic process, in naturalistic settings. Method: We used video-assisted interpersonal process recall interviews to investigate the experiences of 12 patients using ROM/CFS in a Norwegian mental health outpatient clinic. Data were analyzed through systematic text condensation. Results: Our analysis resulted in three pairs of experiences with ROM/CFS: (1) Explicit vs. implicit use of information, (2) Directing focus towards or away from therapeutic topics, and (3) Giving and receiving feedback. These experiences could be helpful or hindering, depending on participants' needs and preferences. All participants needed to know that the CFS was used in a meaningful way. If not, it could be detrimental to the therapeutic process. Conclusion: In order to be helpful for patients, ROM/CFS should be used in a way that is flexible, meaningful to patients, and sensitive to individual needs and preferences. Future research should further explore this how-to aspect of ROM/CFS with different CFS and populations.