A pragmatic randomized control trial and realist evaluation on the implementation and effectiveness of an internet application to support self-management among individuals seeking specialized mental health care: a study protocol

The Role of Moderators in Facilitating and Encouraging Peer-to-Peer Support in an Online Mental Health Community: A Qualitative Exploratory Study

Online peer support platforms have gained popularity as a potential way for people struggling with mental health problems to share information and provide support to each other. While these platforms can offer an open space to discuss emotionally difficult issues, unsafe or unmoderated communities can allow potential harm to users by spreading triggering content, misinformation or hostile interactions. The purpose of this study was to explore the role of moderators in these online communities, and how moderators can facilitate peer-to-peer support, while minimizing harms to users and amplifying potential benefits. Moderators of the Togetherall peer support platform were recruited to participate in qualitative interviews. The moderators, referred to as 'Wall Guides', were asked about their day-to-day responsibilities, positive and negative experiences they have witnessed on the platform and the strategies they employ when encountering problems such as lack of engagement or posting of inappropriate content. The data were then analyzed qualitatively using thematic content analysis and consensus codes were deduced and reviewed to reach final results and representative themes. In total, 20 moderators participated in this study, and described their experiences and efforts to follow a consistent and shared protocol for responding to common scenarios in the online community. Many reported the deep connections formed by the online community, the helpful and thoughtful responses that members give each other and the satisfaction of seeing progress in members' recovery. They also reported occasional aggressive, sensitive or inconsiderate comments and posts on the platform. They respond by removing or revising the hurtful post or reaching out to the affected member to maintain the 'house rules'. Lastly, many discussed strategies they elicit to promote engagement from members within the community and ensure each member is supported through their use of the platform. This study sheds light on the critical role of moderators of online peer support communities, and their ability to contribute to the potential benefits of digital peer support while minimizing risks to users. The findings reported here accentuate the importance of having well-trained moderators on online peer support platforms and can guide future efforts to effectively train and supervise prospective peer support moderators. Moderators can become an active 'shaping force' and bring a cohesive culture of expressed empathy, sensitivity and care. The delivery of a healthy and safe community contrasts starkly with non-moderated online forums, which can become unhealthy and unsafe as a result.

Approaches to enabling rapid evaluation of innovations in health and social care: a scoping review of evidence from high-income countries

OBJECTIVE

The COVID-19 pandemic increased the demand for rapid evaluation of innovation in health and social care. Assessment of rapid methodologies is lacking although challenges in ensuring rigour and effective use of resources are known. We mapped reports of rapid evaluations of health and social care innovations, categorised different approaches to rapid evaluation, explored comparative benefits of rapid evaluation, and identified knowledge gaps.

DESIGN

Scoping review.

DATA SOURCES

MEDLINE, EMBASE and Health Management Information Consortium (HMIC) databases were searched through 13 September 2022.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

We included publications reporting primary research or methods for rapid evaluation of interventions or services in health and social care in high-income countries.

DATA EXTRACTION AND SYNTHESIS

Two reviewers developed and piloted a data extraction form. One reviewer extracted data, a second reviewer checked 10% of the studies; disagreements and uncertainty were resolved through consensus. We used narrative synthesis to map different approaches to conducting rapid evaluation.

RESULTS

We identified 16 759 records and included 162 which met inclusion criteria.We identified four main approaches for rapid evaluation: (1) Using methodology designed specifically for rapid evaluation; (2) Increasing rapidity by doing less or using less time-intensive methodology; (3) Using alternative technologies and/or data to increase speed of existing evaluation method; (4) Adapting part of non-rapid evaluation.The COVID-19 pandemic resulted in an increase in publications and some limited changes in identified methods. We found little research comparing rapid and non-rapid evaluation.

CONCLUSIONS

We found a lack of clarity about what 'rapid evaluation' means but identified some useful preliminary categories. There is a need for clarity and consistency about what constitutes rapid evaluation; consistent terminology in reporting evaluations as rapid; development of specific methodologies for making evaluation more rapid; and assessment of advantages and disadvantages of rapid methodology in terms of rigour, cost and impact.

Hybrid Concept Analysis of Self-Management Support: School Nurses Supporting Students with Psychogenic Nonepileptic Seizures

Self-management support has been identified as an effective nursing intervention for improving outcomes for people with chronic conditions, yet this concept lacks a clear definition. Furthermore, the concept has not been used in school nursing literature despite the clear connection between school nursing practice and tenets of self-management support. Additionally, the concept has not been explored in the context of difficult-to-manage mental health concerns, such as psychogenic nonepileptic seizures. A conversion disorder in which seizure events in the absence of abnormal brainwave activity result from stress, psychogenic nonepileptic seizures affect the quality of life and school experience for students experiencing them and could be addressed through self-management support. This hybrid concept analysis included a review of extant literature and semi-structured interviews with school nurses to ascertain a definition of self-management support in the context of school nursing using care of students with psychogenic nonepileptic seizures as an exemplar.

Closing the Virtual Gap in Health Care: A Series of Case Studies Illustrating the Impact of Embedding Evaluation Alongside System Initiatives

Early decisions relating to the implementation of virtual care relied on necessity and clinical judgement, but there is a growing need for the generation of evidence to inform policy and practice designs. The need for stronger partnerships between researchers and decision-makers is well recognized, but how these partnerships can be structured and how research can be embedded alongside existing virtual care initiatives remain unclear. We present a series of case studies that illustrate how embedded research can inform policy decisions related to the implementation of virtual care, where decisions are either to (1) discontinue (red light), (2) redesign (yellow light), or (3) scale up existing initiatives (green light). Data were collected through document review and informal interviews with key study personnel. Case 1 involved an evaluation of a mobile diabetes platform that demonstrated a mismatch between the setting and the technology (decision outcome: discontinue). Case 2 involved an evaluation of a mental health support platform that suggested evidence-based modifications to the delivery model (decision outcome: redesign). Case 3 involved an evaluation of video visits that generated evidence to inform the ideal model of implementation at scale (decision outcome: scale up). In this paper, we highlight the characteristics of the partnership and the process that enabled success and use the cases to illustrate how these characteristics were operationalized. Structured communication included monthly check-ins and iterative report development. We also outline key characteristics of the partnership (ie, trust and shared purpose) and the process (ie, timeliness, tailored reporting, and adaptability) that drove the uptake of evidence in decision-making. Across each case, the evaluation was designed to address policy questions articulated by our partners. Furthermore, structured communication provided opportunities for knowledge mobilization. Structured communication was operationalized through monthly meetings as well as the delivery of interim and final reports. These case studies demonstrate the importance of partnering with health system decision-makers to generate and mobilize scientific evidence. Embedded research partnerships founded on a shared purpose of system service provided an effective strategy to bridge the oft-cited gap between science and policy. Structured communication provided a mechanism for collaborative problem-solving and real-time feedback, and it helped contextualize emerging insights.

A Web-Based Mental Health Platform for Individuals Seeking Specialized Mental Health Care Services: Multicenter Pragmatic Randomized Controlled Trial

BACKGROUND

Web-based self-directed mental health applications are rapidly emerging to address health service gaps and unmet needs for information and support.

OBJECTIVE

The aim of this study was to determine if a multicomponent, moderated Web-based mental health application could benefit individuals with mental health symptoms severe enough to warrant specialized mental health care.

METHODS

A multicenter, pragmatic randomized controlled trial was conducted across several outpatient mental health programs affiliated with 3 hospital programs in Ontario, Canada. Individuals referred to or receiving treatment, aged 16 years or older, with access to the internet and an email address, and having the ability to navigate a Web-based mental health application were eligible. A total of 812 participants were randomized 2:1 to receive immediate (immediate treatment group, ITG) or delayed (delayed treatment group, DTG) access for 3 months to the Big White Wall (BWW), a multicomponent Web-based mental health intervention based in the United Kingdom and New Zealand. The primary outcome was the total score on the Recovery Assessment Scale, revised (RAS-r) which measures mental health recovery. Secondary outcomes were total scores on the Patient Health Questionnaire-9 item (PHQ-9), the Generalized Anxiety Disorder Questionnaire-7 item (GAD-7), the EuroQOL 5-dimension quality of life questionnaire (EQ-5D-5L), and the Community Integration Questionnaire. An exploratory analysis examined the association between actual BWW use (categorized into quartiles) and outcomes among study completers.

RESULTS

Intervention participants achieved small, statistically significant increases in adjusted RAS-r score (4.97 points, 95% CI 2.90 to 7.05), and decreases in PHQ-9 score (-1.83 points, 95% CI -2.85 to -0.82) and GAD-7 score (-1.55 points, 95% CI -2.42 to -0.70). Follow-up was achieved for 55% (446/812) at 3 months, 48% (260/542) of ITG participants and 69% (186/270) of DTG participants. Only 58% (312/542) of ITG participants logged on more than once. Some higher BWW user groups had significantly greater improvements in PHQ-9 and GAD-7 relative to the lowest use group.

CONCLUSIONS

The Web-based application may be beneficial; however, many participants did not engage in an ongoing way. This has implications for patient selection and engagement as well as delivery and funding structures for similar Web-based interventions.

TRIAL REGISTRATION

ClinicalTrials.gov NCT02896894; https://clinicaltrials.gov/ct2/show/NCT02896894 (Archived by WebCite at http://www.webcitation.org/78LIpnuRO).

Extending access to a web-based mental health intervention: who wants more, what happens to use over time, and is it helpful? Results of a concealed, randomized controlled extension study

Background

Web-based mental health applications may be beneficial, but adoption is often low leaving optimal implementation and payment models unclear. This study examined which users were interested in extended access to a web-based application beyond an initial 3-month trial period and evaluated if an additional 3 months of access was beneficial.

Methods

This study was a concealed extension of a multi-center, pragmatic randomized controlled trial that assessed the benefit of 3 months of access to the Big White Wall (BWW), an anonymous web-based moderated, multi-component mental health application offering self-directed activities and peer support. Trial participants were 16 years of age or older, recruited from hospital-affiliated mental health programs. Participants who received access to the intervention in the main trial and completed 3-month outcome assessments were offered participation. We compared those who were and were not interested in an extension of the intervention, and re-randomized consenting participants 1:1 to receive extended access or not over the subsequent 3 months. Use of the intervention was monitored in the extension group and outcomes were measured at 3 months after re-randomization in both groups. The primary outcome was mental health recovery as assessed by total score on the Recovery Assessment Scale (RAS-r), as in the main trial. Linear mixed models were used to examine the time by group interaction to assess for differences in responses over the 3-month extension study.

Results

Of 233 main trial participants who responded, 119 (51.1%) indicated an interest in receiving extended BWW access. Those who were interested had significantly higher baseline anxiety symptoms compared to those who were not interested. Of the 119, 112 were re-randomized (55 to extended access, 57 to discontinuation). Only 21 of the 55 extended access participants (38.2%) used the intervention during the extension period. Change in RAS-r scores over time was not significantly different between groups (time by group, F(1,77) = 1.02; P = .31).

Conclusions

Only half of eligible participants were interested in extended access to the intervention with decreasing use over time, and no evidence of added benefit. These findings have implications for implementation and payment models for this type of web-based mental health intervention.

Trial registration

Clinicaltrials.govNCT02896894. Registered retrospectively on September 12, 2016.

Examining Tensions That Affect the Evaluation of Technology in Health Care: Considerations for System Decision Makers From the Perspective of Industry and Evaluators

Virtual technologies have the potential to mitigate a range of challenges for health care systems. Despite the widespread use of mobile devices in everyday life, they currently have a limited role in health service delivery and clinical care. Efforts to integrate the fast-paced consumer technology market with health care delivery exposes tensions among patients, providers, vendors, evaluators, and system decision makers. This paper explores the key tensions between the high bar for evidence prior to market approval that guides health care regulatory decisions and the “fail fast” reality of the technology industry. We examine three core tensions: balancing user needs versus system needs, rigor versus responsiveness, and the role of pre- versus postmarket evidence generation. We use these to elaborate on the structure and appropriateness of evaluation mechanisms for virtual care solutions. Virtual technologies provide a foundation for personalized, patient-centered medicine on the user side, coupled with a broader understanding of impact on the system side. However, mechanisms for stakeholder discussion are needed to clarify the nature of the health technology marketplace and the drivers of evaluation priorities.

Adoption, Acceptability, and Effectiveness of a Mobile Health App for Personalized Prostate Cancer Survivorship Care: Protocol for a Realist Case Study of the Ned App

Background

By 2030, prostate cancer will be the most commonly diagnosed cancer in North America. To mitigate this impending challenge, comprehensive support mechanisms for disease- and treatment-specific changes in health and well-being must be proactively designed and thoughtfully implemented for streamlined survivorship care. mHealth apps have been lauded as a promising complement to current outpatient treatment and monitoring strategies, but have not yet been widely used to support prostate cancer survivorship needs. A realist evaluation is needed to examine the impact of such apps on the prostate cancer survivorship experience.

Objective

We seek to gain an understanding of how an mHealth app for prostate cancer survivorship care called Ned (No Evident Disease) is adopted and accepted by patients, caregivers, and clinicians. We also aim to determine the effect of Ned on health-related quality of life, satisfaction with cancer care, unmet needs, self-efficacy, and prostate cancer-related levels of anxiety.

Methods

The Ned case study is a 12-month mixed-methods embedded single-case study with a nested within-group pre-post comparison of health outcomes. We will give 400 patients, 200 caregivers, and 10 clinicians access to Ned. Participants will be asked to complete study assessments at baseline, 2 months, 6 months, and 12 months. We will conduct 30 semistructured qualitative interviews with patients (n=20) and their caregivers (n=10) poststudy to gain insight into their experience with the app.

Results

We recruited our first survivor in October 2017 and anticipate completing this study by May 2019.

Conclusions

This will, to our knowledge, be the first realist case study to evaluate an app for prostate cancer survivorship care. Prostate cancer survivors are set to increase in number and longevity, heightening the need for integrated survivorship solutions to provide them with optimal and durable outcomes. The knowledge gained from this study will comprehensively inform how and why Ned works, for whom, and in what circumstances. Understanding the impact of digital health interventions such as Ned on how survivors care for themselves is critical to realizing patient-centered care.