Trials of Intervention Principles: Evaluation Methods for Evolving Behavioral Intervention Technologies

Development and pilot-testing of an app to complement trauma-focused cognitive behavioral therapy for adolescents

Background: Apps have the potential to support psychological treatments by providing psychoeducation, increasing homework compliance, and generalizing therapeutic skills outside of sessions. However, there are few apps developed to support specific evidence-based treatment for PTSD for adolescents.Objective: This paper shares experiences developing an app to complement Trauma-Focused Cognitive Behavioral Therapy (TF-CBT). It also describes adolescents' and therapists' perceived usefulness and the potential effect of the app 'My everyday life' on posttraumatic stress symptoms (PTSS).Methods: We did a needs assessment with adolescents, therapists, and TF-CBT trainers, followed by programming and beta-testing. We carried out a mixed methods study of 59 adolescent patients (aged 13-18 years) and their therapists. Using a triangulation design, we combined quantitative data on app use and changes in PTSS with qualitative data from interviews with 10 adolescents and 10 therapists who described their experiences with the app.Results: The app includes psychoeducation, mental health self-monitoring, coping tools, and goal-setting functionality. The most used features were psychoeducation and breathing exercises. Few therapists actively used the monitoring progress feature. Both adolescents and therapists found the app beneficial for everyday use, though some therapists were unsure about how to integrate it with TF-CBT. Adolescents reported significant PTSS improvement during TF-CBT with the app as a complement. Adolescents with post-treatment data (n = 13) did not report significantly different PTSD improvement compared to adolescents without access to app, but the study lacked power to draw strong conclusions.Conclusions: The app 'My everyday life' can be useful for adolescents receiving TF-CBT. The therapists may benefit from clearer guidelines on how to integrate apps with evidence-based treatments. Digital tools are a promising avenue for supporting the implementation of evidence-based treatment.

Methodological approaches and author-reported limitations in evaluation studies of digital health technologies (DHT): A scoping review of DHT interventions for cancer, diabetes mellitus, and cardiovascular diseases

Digital health technology (DHT) holds the potential to improve health services, and its adoption has proliferated in recent decades owing to technological advancement. Optimal evaluation methodologies appropriate for generating quality evidence on DHT have yet to be established; traditional comparative designs present several limitations. This study aimed to scope the literature to highlight common methodological approaches used and their limitations to inform considerations for designing robust DHT evaluation studies. A scoping review was conducted following the Joanna Briggs Institute (JBI) scoping review guidelines. A systematic search was conducted using the CINAHL (EBSCO), MEDLINE (EBSCOhost), PsycINFO (EBSCO), EMBASE (Elsevier) and Web of Science (Clarivate Analytics) databases using iteratively developed search terms. We selected studies published in English between January 2016 and March 2022 and focussed on primary research evaluating the effectiveness of DHT with technology-user interactive or asynchronous features for adults (≥18 years) with cancer, diabetes or cardiovascular conditions. The final number of articles, after the screening and selection process, comprised 140 records. Data were analysed descriptively (frequency and percentages) and summarised thematically. Results showed most studies (n = 104, 74.3%) employed the standard two-arm parallel RCT design, with usual/standard care as the preferred comparator in nearly half (n = 65, 47.1%) of all included studies. Of the 104 comparative studies reviewed, limitations in recruitment were most frequently reported (n = 70, 37%), followed by limitations in evaluation/measurement techniques (n = 57, 27%), presence of confounding factors (n = 50, 24%) and short duration of studies (n = 24, 11%). The review highlights the need to consider inclusive approaches to recruitment and adoption of the emerging methodological approaches that account for the fast-paced, multi-component and group contamination problem resulting from the unconcealable nature of DHT interventions.

A Narrative Review to Identify Promising Approaches for Digital Health Interventions to Support Emotion Regulation for Adolescents With Attention-Deficit/Hyperactivity Disorder

Emotion regulation difficulties affect many adolescents with attention-deficit/hyperactivity disorder (ADHD), and previous research has highlighted a need for accessible interventions to support them in this domain, especially in real-life contexts. Digital health interventions (DHIs) can be embedded in adolescents' daily lives and thus offer considerable promise for meeting this need. However, there is a lack of information to guide the development of suitable emotion regulation DHIs for this population. The goal of this study is, therefore, to identify recommendations to guide the development of emotion regulation DHIs for adolescents with ADHD. This narrative review synthesizes diverse relevant evidence to inform their development, including promising therapeutic approaches and components and relevant design and development considerations. We find that there is very little direct evidence of "what works" for emotion regulation DHIs and emotion regulation interventions more generally for adolescents with ADHD; however, we identify promising therapeutic approaches for new DHIs. We also recommend following a co-design or coproduction approach with adolescents with ADHD, including exploring elements designed to motivate and engage young people to support sustained adherence. We conclude that DHIs are a promising approach for emotion regulation interventions for adolescents with ADHD, could draw on a range of existing therapeutic approaches, and should be co-designed with users themselves.

Supporting the Individualized Use of Digital Tools in Community Mental Health: The Technology Specialist Pilot Study

This pilot study evaluated the feasibility of the technology specialist intervention, which assists clients in achieving mental health recovery and well-being goals via existing digital tools in a real-world community mental health setting. Thirteen adult clients with serious mental illness and their providers completed baseline, 3-, and 6-month assessments, including goal setting, self-efficacy, activation, and acceptability measures, along with weekly ecological momentary assessments. Clients selected goals and corresponding tools, used the tools steadily, and showed improvement in activation and self-efficacy. Most participating clients (82%, n = 9) and providers (80%, n = 8) found the intervention acceptable. These preliminary findings show that the technology specialist intervention is promising and warrants further testing.

Systematic optimization and evaluation of a Dutch sexual health intervention: Role model stories for chlamydia prevention, testing, and treatment

Background

The rapidly evolving nature of eHealth necessitates regular optimization and subsequent evaluation. Within the Dutch sexual health intervention Sense.info, we utilized a mixed-methods cyclic evaluation process to assess and optimize the potential impact of the chlamydia page. This paper reports on the page's optimization through the development of role model stories for chlamydia prevention and the subsequent evaluation of these stories.

Method

The experiences of 10 young individuals served as the basis of role model stories using the behavior change principle modeling based on social cognitive theory. These stories aimed to motivate young individuals to undergo sexually transmitted infection testing, use condoms, and notify sexual partners. Once the stories were posted online, we tracked use data between July and September 2022 and investigated end-user perspectives through a think-aloud study combined with semistructured interviews (N = 20, M age = 19.7, SDage = 2.65). Template analyses were used for the analysis of the think-aloud study.

Results

Use data revealed that all stories were accessed by website visitors, yet other page elements on the chlamydia page interacted with more. The exploration of end-user perspectives indicated a positive impact of the personal stories on normalization, self-efficacy, and skills related to chlamydia preventive behaviors. Mixed results were found regarding some conditions for the effectiveness of the behavior change principle modeling.

Discussion and conclusion

This study provided valuable insights into the cyclic evaluation process for evaluating and optimizing web-based public health interventions, as well as the potential impact of role model stories on sexual health prevention. Also, aspects of the stories that could be optimized in future optimization rounds were identified. Overall, this research contributes to enhancing the impact of eHealth interventions through iterative evaluation and optimization processes.

Practitioner Review: Treatments for young people who self-harm - challenges and recommendations for research and clinical practice

BACKGROUND

Self-harm is very common in young people and is associated with suicide. Rates of both self-harm and suicide have increased in young people, particularly in females. There is a clear need to identify new approaches to prevent repeat self-harm.

METHOD

We significantly update and build on previous reviews with the aim of identifying issues in research relevant to clinical practice. We identify challenges in developing, implementing and evaluating treatments for self-harm in children and adolescents, suggest a way forward for research, and provide clear and practical guidance for clinicians on how to apply current research evidence in the real world.

RESULTS

Currently, there is limited evidence for effective interventions, other than some support for dialectical behaviour therapy for adolescents (DBT-A). To improve research and, by extension, clinical practice, future studies need to address psychosocial factors associated with youth self-harm and suicide, investigate the critical mechanism(s) of action, ensure trials are sufficiently powered and representative, and involve young people more actively in the design, implementation and evaluation of these approaches. Consideration should also be given to alternative research designs, such as pragmatic or adaptive clinical trials, as well as registry-based randomised controlled trials which leverage administrative data collected in routine clinical practice, to help meet these goals.

CONCLUSIONS

Recommendations for practice include undertaking comprehensive assessment and formulation, and offering DBT-A where indicated. There should be further development and evaluation (with input from young people) of Cognitive Behavioural-based Therapy adapted for young people. Greater attention to the role of the therapeutic relationship and family involvement (where possible) is also an important considerations, irrespective of the specific therapeutic modality. Finally, more consideration should be given to improving staff training to ensure all clinical staff feel equipped to treat young people who self-harm in a person-centred and compassionate manner.

A personal sensing technology enabled service versus a digital psychoeducation control for primary care patients with depression and anxiety: a pilot randomized controlled trial

BACKGROUND

Technology-enabled services (TES; clinical services that include both technology-driven [e.g., personal sensing technologies] and person-powered support elements) may address gaps in depression and anxiety treatments in healthcare settings. The current study: (1) developed a TES tailored for Primary Care patients with depression and/or anxiety, and (2) conducted a pilot randomized controlled trial to assess the efficacy of the TES compared to a digital psychoeducation control app.

METHODS

Participants were randomized to either: (1) TES: the "Vira" smartphone app (Ksana Health Inc.), informed by behavioral activation and using passive sensing technology to provide behavioral "insights" and target behaviors associated with mental health symptoms, alongside lay-provider coaching, or (2) Control: the Mood Education mobile app (ME), containing static psychoeducational resources designed to target mental health symptoms. Both apps collected usage data. Participants completed assessments on depression (PHQ-9), anxiety (GAD-7), health-related quality of life (PedsQL), and engagement (TWEETs) at baseline, mid-treatment (week 4), end-of-treatment (week 8), and post-treatment (week 12).

RESULTS

Participants (N = 130) were randomized to receive either the TES (Vira; Mage= 30) or ME (Mage= 33). Linear mixed-effects models determined significant improvements in PHQ-9 and GAD-7 scores for participants across both conditions (ps < 0.001). There was no interactive effect of intervention and time for the PHQ-9 (p = .90) nor the GAD-7 (p = .49). Adjusting for baseline differences and randomization strata, TES participants reported a greater change in a quality of life rating of Physical Functioning (PedsQL) across time (p = .018). TES participants also reported higher levels of engagement and demonstrated higher app usage. However, adjusting for baseline symptom severity, neither app usage nor coach interaction frequency moderated outcomes (ps ≥ 0.2).

CONCLUSIONS

While the TES demonstrated superior engagement, improvements in depressive and anxious symptoms for both conditions speak to the potential benefit of both TES and low-intensity psychoeducation treatments in care settings. Future research is needed to better understand which patients might differentially benefit from TESs and broader personal sensing technologies over low-intensity treatments.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05406791.

CoDesign of a digital intervention for parents with bipolar disorder informed by integrated knowledge translation principles

OBJECTIVES

To provide detailed information on the codesign of a digital intervention to support parents with bipolar disorder (BD) who have young children. Each step of this process is reported, as well as a detailed description of the final version of the intervention in line with the TIDieR framework.

METHODS

Clinical experience and lived experience experts participated in online workshops, meetings, and remote feedback requests, informed by Integrated Knowledge Translation (IKT) principles. The IKT research group responded to each phase of recommendations from the knowledge users.

RESULTS

Five clinical experience experts and six lived experience experts engaged with the codesign process. Their recommendations for principles, content, look, and feel, and functionality of the digital intervention were structured over five iterative phases. This led to a final implemented design that was identified by the clinical and lived experience experts (referred to together as the knowledge users group) as genuinely reflecting their input.

CONCLUSIONS

The IKT principles offer an accessible structure for engaging with clinical and lived experience experts throughout a codesign process, in this case for a digital intervention for parents with BD. The resulting intervention is described in detail for transparency to aid further evaluation and development and to help other teams planning codesign approaches to intervention development.

Immersive Virtual Reality Cognitive Training for Improving Cognition and Depressive Symptoms Among Older Adults. Current Evidence and Future Recommendations. A Systematic Review

Older adults, particularly impacted by the pandemic, are anticipated to face considerable challenges to their cognitive and psychological health. Various cognitive training methods have been proposed to improve their cognitive decline through regular practice. Immersive virtual reality (IVR) offers a promising avenue to bolster cognitive and mental health care accessibility for older adults and has shown potential in augmenting traditional cognitive training methods. This systematic review aims to assess the efficacy of IVR-based cognitive training on improving cognitive impairments and depressive symptoms among older adults, irrespective of their level of cognitive decline. Systematic searches were conducted across MEDLINE-PubMed, EMBASE, and Web of Science databases from inception to December 2022. Only peer-reviewed articles reporting results from randomized controlled trials and pilot studies, focusing on the effects of VR-based cognitive training on adults aged 55 or older, were deemed eligible. Using the Cochrane Handbook for Systematic Reviews of Interventions, the risk of bias was appraised for each selected study. Out of 3,835 studies identified, 9 met the inclusion criteria. The qualitative analyses concluded that IVR cognitive training interventions enhanced certain cognitive functions and well-being among older adults, notably those with mild cognitive impairments. These interventions were particularly effective in boosting memory, visuospatial abilities, and measures related to depressive symptoms and overall well-being. However, enhancements were less pronounced for executive functions, attention, and global cognition. Based on these findings, we propose a set of tailored, inclusive, and evidence-based recommendations to adapt emerging technologies to better accommodate the needs of older adults.

Challenges and Alternatives to Evaluation Methods and Regulation Approaches for Medical Apps as Mobile Medical Devices: International and Multidisciplinary Focus Group Discussion

BACKGROUND

The rapid proliferation of medical apps has transformed the health care landscape by giving patients and health care providers unprecedented access to personalized health information and services. However, concerns regarding the effectiveness and safety of medical apps have raised questions regarding the efficacy of randomized controlled trials (RCTs) in the evaluation of such apps and as a requirement for their regulation as mobile medical devices.

OBJECTIVE

This study aims to address this issue by investigating alternative methods, apart from RCTs, for evaluating and regulating medical apps.

METHODS

Using a qualitative approach, a focus group study with 46 international and multidisciplinary public health experts was conducted at the 17th World Congress on Public Health in May 2023 in Rome, Italy. The group was split into 3 subgroups to gather in-depth insights into alternative approaches for evaluating and regulating medical apps. We conducted a policy analysis on the current regulation of medical apps as mobile medical devices for the 4 most represented countries in the workshop: Italy, Germany, Canada, and Australia. We developed a logic model that combines the evaluation and regulation domains on the basis of these findings.

RESULTS

The focus group discussions explored the strengths and limitations of the current evaluation and regulation methods and identified potential alternatives that could enhance the quality and safety of medical apps. Although RCTs were only explicitly mentioned in the German regulatory system as one of many options, an analysis of chosen evaluation methods for German apps on prescription pointed toward a "scientific reflex" where RCTs are always the chosen evaluation method. However, this method has substantial limitations when used to evaluate digital interventions such as medical apps. Comparable results were observed during the focus group discussions, where participants expressed similar experiences with their own evaluation approaches. In addition, the participants highlighted numerous alternatives to RCTs. These alternatives can be used at different points during the life cycle of a digital intervention to assess its efficacy and potential harm to users.

CONCLUSIONS

It is crucial to recognize that unlike analog tools, digital interventions constantly evolve, posing challenges to inflexible evaluation methods such as RCTs. Potential risks include high dropout rates, decreased adherence, and nonsignificant results. However, existing regulations do not explicitly advocate for other evaluation methodologies. Our research highlighted the necessity of overcoming the gap between regulatory demands to demonstrate safety and efficacy of medical apps and evolving scientific practices, ensuring that digital health innovation is evaluated and regulated in a way that considers the unique characteristics of mobile medical devices.

Evolution of Research Reporting Standards: Adapting to the Influence of Artificial Intelligence, Statistics Software, and Writing Tools

Reporting standards are essential to health research as they improve accuracy and transparency. Over time, significant changes have occurred to the requirements for reporting research to ensure comprehensive and transparent reporting across a range of study domains and foster methodological rigor. The establishment of the Declaration of Helsinki, Consolidated Standards of Reporting Trials (CONSORT), Strengthening the Reporting of Observational Studies in Epidemiology (STROBE), and Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) are just a few of the historic initiatives that have increased research transparency. Through enhanced discoverability, statistical analysis facilitation, article quality enhancement, and language barrier reduction, artificial intelligence (AI)-in particular, large language models like ChatGPT-has transformed academic writing. However, problems with errors that could occur and the need for transparency while utilizing AI tools still exist. Modifying reporting rules to include AI-driven writing tools such as ChatGPT is ethically and practically challenging. In academic writing, precautions for truth, privacy, and responsibility are necessary due to concerns about biases, openness, data limits, and potential legal ramifications. The CONSORT-AI and Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT)-AI Steering Group expands the CONSORT guidelines for AI clinical trials-new checklists like METRICS and CLEAR help to promote transparency in AI studies. Responsible usage of technology in research and writing software adoption requires interdisciplinary collaboration and ethical assessment. This study explores the impact of AI technologies, specifically ChatGPT, on past reporting standards and the need for revised guidelines for open, reproducible, and robust scientific publications.

Key Considerations for Designing Clinical Studies to Evaluate Digital Health Solutions

Evidence of clinical impact is critical to unlock the potential of digital health solutions (DHSs), yet many solutions are failing to deliver positive clinical results. We argue in this viewpoint that this failure is linked to current approaches to DHS evaluation design, which neglect numerous key characteristics (KCs) requiring specific scientific and design considerations. We first delineate the KCs of DHSs: (1) they are implemented at health care system and patient levels; (2) they are "complex" interventions; (3) they can drive multiple clinical outcomes indirectly through a multitude of smaller clinical benefits; (4) their mechanism of action can vary between individuals and change over time based on patient needs; and (5) they develop through short, iterative cycles-optimally within a real-world use context. Following our objective to drive better alignment between clinical evaluation design and the unique traits of DHSs, we then provide methodological suggestions that better address these KCs, including tips on mechanism-of-action mapping, alternative randomization methods, control-arm adaptations, and novel end-point selection, as well as innovative methods utilizing real-world data and platform research.

HCI Contributions in Mental Health: A Modular Framework to Guide Psychosocial Intervention Design

Many people prefer psychosocial interventions for mental health care or other concerns, but these interventions are often complex and unavailable in settings where people seek care. Intervention designers use technology to improve user experience or reach of interventions, and HCI researchers have made many contributions toward this goal. Both HCI and mental health researchers must navigate tensions between innovating on and adhering to the theories of change that guide intervention design. In this paper, we propose a framework that describes design briefs and evaluation approaches for HCI contributions at the scopes of capabilities, components, intervention systems, and intervention implementations. We show how theories of change (from mental health) can be translated into design briefs (in HCI), and that these translations can bridge and coordinate efforts across fields. It is our hope that this framework can support researchers in motivating, planning, conducting, and communicating work that advances psychosocial intervention design.

Methodological Frameworks and Dimensions to Be Considered in Digital Health Technology Assessment: Scoping Review and Thematic Analysis

BACKGROUND

Digital health technologies (dHTs) offer a unique opportunity to address some of the major challenges facing health care systems worldwide. However, the implementation of dHTs raises some concerns, such as the limited understanding of their real impact on health systems and people's well-being or the potential risks derived from their use. In this context, health technology assessment (HTA) is 1 of the main tools that health systems can use to appraise evidence and determine the value of a given dHT. Nevertheless, due to the nature of dHTs, experts highlight the need to reconsider the frameworks used in traditional HTA.

OBJECTIVE

This scoping review (ScR) aimed to identify the methodological frameworks used worldwide for digital health technology assessment (dHTA); determine what domains are being considered; and generate, through a thematic analysis, a proposal for a methodological framework based on the most frequently described domains in the literature.

METHODS

The ScR was performed in accordance with the guidelines established in the PRISMA-ScR guidelines. We searched 7 databases for peer reviews and gray literature published between January 2011 and December 2021. The retrieved studies were screened using Rayyan in a single-blind manner by 2 independent authors, and data were extracted using ATLAS.ti software. The same software was used for thematic analysis.

RESULTS

The systematic search retrieved 3061 studies (n=2238, 73.1%, unique), of which 26 (0.8%) studies were included. From these, we identified 102 methodological frameworks designed for dHTA. These frameworks revealed great heterogeneity between them due to their different structures, approaches, and items to be considered in dHTA. In addition, we identified different wording used to refer to similar concepts. Through thematic analysis, we reduced this heterogeneity. In the first phase of the analysis, 176 provisional codes related to different assessment items emerged. In the second phase, these codes were clustered into 86 descriptive themes, which, in turn, were grouped in the third phase into 61 analytical themes and organized through a vertical hierarchy of 3 levels: level 1 formed by 13 domains, level 2 formed by 38 dimensions, and level 3 formed by 11 subdimensions. From these 61 analytical themes, we developed a proposal for a methodological framework for dHTA.

CONCLUSIONS

There is a need to adapt the existing frameworks used for dHTA or create new ones to more comprehensively assess different kinds of dHTs. Through this ScR, we identified 26 studies including 102 methodological frameworks and tools for dHTA. The thematic analysis of those 26 studies led to the definition of 12 domains, 38 dimensions, and 11 subdimensions that should be considered in dHTA.

Adapting an Effective Health-Promoting Intervention-Choose to Move-for Chinese Older Adults in Canada

Evidence is sparse on how community-based health-promoting programs can be culturally adapted for racially minoritized, immigrant older adult populations. Choose to Move (CTM) is an evidence-based health-promoting program that enhances physical activity and mobility and diminished social isolation and loneliness in older adults in British Columbia, Canada. However, racially minoritized older adults were not reached in initial offerings. We purposively sampled CTM delivery staff (n = 8) from three not-for-profit organizations, in Metro Vancouver, British Columbia, that serve Chinese older adults. We used semistructured interviews, ethnographic observations, and meeting minutes to understand delivery staff's perspectives on factors that influence CTM adaptations for Chinese older adults. Deductive framework analysis guided by an adaptation framework, Framework for Reporting Adaptations and Modifications-Enhanced, found three dominant cultural- and immigration-related factors influenced CTM adaptations for Chinese older adults: (a) prioritizations, (b) familiarity, and (c) literacy. Findings may influence future program development and delivery to meet the needs of racially minoritized older adult populations.

iTHRIVE 365: A Community-Led, Multicomponent Health Promotion Intervention for Black Same Gender Loving Men

Although health inequities among Black same gender loving men (SGLM) are well documented (e.g., chronic psychological disorders, HIV, suicide), there are few accessible, culturally affirming, and community-led interventions designed to reduce these inequities. The present manuscript describes the process through which we developed iTHRIVE 365, a multicomponent health-promotion intervention designed by Black SGLM for Black SGLM. We utilized a community-based participatory research approach (CBPR) that included collaboration between THRIVE SS, a Black SGLM-run community-based organization, and a multisectoral team of public health, research, and digital design professionals to develop the intervention. A five-phase development process included four phases of focus groups and a technical pilot to assess community priorities and incorporate input on each feature of the intervention. Directed content analysis indicated that participants wanted a multicomponent and technology-mediated intervention that promotes health knowledge and motivation, Black SGLM social support, access to affirming healthcare, and housing and economic resources. iTHRIVE 365 combines multilevel and culturally affirming intervention features to combat the effects of oppression and ultimately promote Black SGLM's biopsychosocial health.

From formative design to service-ready therapeutic: A pragmatic approach to designing digital mental health interventions across domains

As digital mental health interventions (DMHIs) proliferate, there is a growing need to understand the complexities of moving these tools from concept and design to service-ready products. We highlight five case studies from a center that specializes in the design and evaluation of digital mental health interventions to illustrate pragmatic approaches to the development of digital mental health interventions, and to make transparent some of the key decision points researchers encounter along the design-to-product pipeline. Case studies cover different key points in the design process and focus on partnership building, understanding the problem or opportunity, prototyping the product or service, and testing the product or service. We illustrate lessons learned and offer a series of questions researchers can use to navigate key decision points in the digital mental health intervention (DMHI) development process.

A randomised controlled trial of clinician supported vs self-help delivery of online cognitive behaviour therapy for Bulimia Nervosa

High dropout rates and poor adherence associated with digital interventions have prompted research into modifications of these treatments to improve engagement and completion rates. This trial aimed to investigate the added benefit of clinician support when paired alongside a ten-session, online cognitive behaviour therapy (CBT) self-help intervention for bulimia nervosa (BN). As part of a three-arm, phase II randomised controlled trial, 114 participants (16 years or over) with full or subthreshold BN were randomly assigned to complete the intervention in a self-help mode (with administrative researcher contact; n = 38), with adjunct clinician support (weekly 30-minute videoconferencing sessions; n = 37), or a no-treatment waitlist control (WLC; n = 39). Baseline to post-treatment (12-weeks) decreases in objective binge episode frequency were significantly greater for clinician-supported participants as compared to WLC, but not for self-help when compared to WLC. However, due to continued improvements for self-help across follow-up (24-weeks), both arms outperformed WLC when analysed as an overall rate of change across three timepoints. Clinician-supported participants outperformed self-help in regards to laxative use and dietary restraint. Our results demonstrate that good clinical outcomes can be achieved with a relatively brief online CBT-based program even in the absence of structured clinical support, indicating a possible overreliance upon clinician support as a primary adherence-facilitating mechanism.

Feasibility, Acceptability, and Preliminary HIV Care and Psychological Health Effects of iTHRIVE 365 for Black Same Gender Loving Men

OBJECTIVES

This uncontrolled pilot study examined the feasibility, acceptability, and preliminary HIV and psychological health effects of iTHRIVE 365, a multicomponent intervention designed by and for Black same gender loving men (SGLM) to promote: health knowledge and motivation, Black SGLM social support, affirming health care, and housing and other economic resources.

DESIGN METHODS

We conducted a 14-day daily diary study with 32 Black SGLM living with HIV connected to THRIVE SS in Atlanta, GA. Daily surveys assessed intervention engagement, antiretroviral medication (ART) use, depressive symptoms, anxiety symptoms, and emotion regulation difficulties. App paradata (ie, process data detailing app usage) assessed amount of intervention engagement via page access. Participants began receiving access to the intervention on day 7. After the 14-day daily diary period, participants responded to follow-up items on the user-friendliness, usefulness, helpfulness, and whether they would recommend iTHRIVE 365 to others. Chi-square analyses examined associations between intervention engagement and ART use, and dynamic structural equation modelling assessed longitudinal associations from intervention engagement to next-day psychological health. This intervention trial is registered on ClinicalTrials.gov (NCT05376397).

RESULTS

On average, participants engaged with iTHRIVE 365 over once every other day and accessed intervention pages 4.65 times per day. Among participants who engaged with the intervention, 78% reported it was helpful to extremely helpful, 83% reported it was moderately to extremely useful, and 88% reported it was user-friendly and they would recommend it to others. On intervention engagement days, participants had higher odds of ART use, χ 2 (1) = 4.09, P = 0.04, than intervention nonengagement days. On days after intervention engagement, participants showed non-null decreases in depressive symptoms (τ = -0.14; 95% CI : = [-0.23, -0.05]) and emotion regulation difficulties (τ = -0.16; 95% CI : = [-0.24, -0.02]).

CONCLUSIONS

Findings suggest iTHRIVE 365 is feasible, acceptable, and positively affects daily ART use, depressive symptoms, and emotion regulation difficulties.

Digital health innovation to prevent relapse and support recovery in young people with first-episode psychosis: A pilot study of Horyzons-Canada

Digital health innovations may help to improve access to psychosocial therapy and peer support; however, the existence of evidence-based digital health interventions for individuals recovering from a first-episode psychosis (FEP) remains limited. This study aims to investigate the feasibility, acceptability, safety, and pre-post outcomes of Horyzons-Canada (HoryzonsCa), a Canadian adaptation of a digital mental health intervention consisting of psychosocial interventions, online social networking, and clinical and peer support moderation. Using a convergent mixed-methods research design, we recruited participants from a specialized early intervention clinic for FEP in Montreal, Canada. Twenty-three participants (mean age = 26.8) completed baseline assessments, and 20 completed follow-up assessments after 8 weeks of intervention access. Most participants provided positive feedback on general experience (85%, 17/20) and the utility of Horyzons for identifying their strengths (70%, 14/20). Almost all perceived the platform as easy to use (95%, 19/20) and felt safe using it (90%, 18/20). There were no adverse events related to the intervention. Participants used HoryzonsCa to learn about their illness and how to get better (65%, 13/20), receive support (60%, 12/20), and access social networking (35%, 7/20) and peer support (30%, 6/20). Regarding adoption, 65% (13/20) logged in at least 4 times over 8 weeks. There was a nonsignificant increase in social functioning and no deterioration on the Clinical Global Impression Scale. Overall, HoryzonsCa was feasible to implement and perceived as safe and acceptable. More research is needed with larger sample sizes and using in-depth qualitative methods to better understand the implementation and impact of HoryzonsCa.

A Methodological Approach for Documenting Multi-Component Interventions Targeting Family Caregivers

Managing the complex care needs of seriously ill patients transitioning from hospital to home can have detrimental effects on family caregivers (FCG). Multi-component interventions tailored to FCG needs are most effective at reducing caregiver burden, distress, and depression. However, gaps exist in determining best methods to assess, document, and analyze intervention components for FCGs. Common methods used to capture patient data during transitions in care may not be appropriate or allowed for FCG needs. As such, we present a methodological approach for electronically capturing, reporting, and analyzing multiple intervention components. This approach uses a standardized terminology and pathway for tailoring intervention components in real time while evaluating intervention effects across time. We use examples from a randomized controlled trial to illustrate the benefits of the current approach for analyzing the effectiveness of multi-component interventions in the context of caregiving research.

The role of accelerator programmes in supporting the adoption of digital health technologies: A qualitative study of the perspectives of small- and medium-sized enterprises

Objective

Evidence-based digital health technologies are increasingly important in delivering care to an ageing population with constrained resources. In the United Kingdom, accelerator programmes (APs) have been developed to support the adoption of digital health technologies within the National Health Service. This study aims to explore the perspectives of stakeholders using APs.

Methods

Stakeholders representing nine small -and medium-sized enterprises (SMEs) that were engaged with three different APs (n = 9). Semi-structured interviews were conducted with key informants between April and September 2018. Framework analysis of the data was performed to explore their perspectives on APs.

Results

Four key themes were generated. Informants reported the need to generate evidence before and during the programme, appreciating different types of evidence and their importance. Informants identified several key factors that were a catalyst for success, including involvement in the programme and access to individuals and organisations that were crucial for support. However, several barriers were identified at the programme and system levels. Finally, informants identified key supporting processes that enhanced the adoption of their innovations.

Conclusion

SMEs that develop digital health technologies report that, while APs are useful in supporting the adoption of these technologies, some issues remain. These relate to the emphasis on traditional research evidence that remains a challenge for SMEs to generate. Also, several system-level barriers to innovation in healthcare persist. As APs and SMEs continue to create an entrepreneurial ecosystem, there is increased potential for the development of supporting processes and infrastructure to accelerate the efficient and timely adoption of new digital health technologies.

Digital interventions for hypertension and asthma to support patient self-management in primary care: the DIPSS research programme including two RCTs

Background

Digital interventions offer a potentially cost-effective means to support patient self-management in primary care, but evidence for the feasibility, acceptability and cost-effectiveness of digital interventions remains mixed. This programme focused on the potential for self-management digital interventions to improve outcomes in two common, contrasting conditions (i.e. hypertension and asthma) for which care is currently suboptimal, leading to excess deaths, illness, disability and costs for the NHS.

Objectives

The overall purpose was to address the question of how digital interventions can best provide cost-effective support for patient self-management in primary care. Our aims were to develop and trial digital interventions to support patient self-management of hypertension and asthma. Through the process of planning, developing and evaluating these interventions, we also aimed to generate a better understanding of what features and methods for implementing digital interventions could make digital interventions acceptable, feasible, effective and cost-effective to integrate into primary care.

Design

For the hypertension strand, we carried out systematic reviews of quantitative and qualitative evidence, intervention planning, development and optimisation, and an unmasked randomised controlled trial comparing digital intervention with usual care, with a health economic analysis and nested process evaluation. For the asthma strand, we carried out a systematic review of quantitative evidence, intervention planning, development and optimisation, and a feasibility randomised controlled trial comparing digital intervention with usual care, with nested process evaluation.

Setting

General practices (hypertension, n = 76; asthma, n = 7) across Wessex and Thames Valley regions in Southern England.

Participants

For the hypertension strand, people with uncontrolled hypertension taking one, two or three antihypertensive medications. For the asthma strand, adults with asthma and impaired asthma-related quality of life.

Interventions

Our hypertension intervention (i.e. HOME BP) was a digital intervention that included motivational training for patients to self-monitor blood pressure, as well as health-care professionals to support self-management; a digital interface to send monthly readings to the health-care professional and to prompt planned medication changes when patients’ readings exceeded recommended targets for 2 consecutive months; and support for optional patient healthy behaviour change (e.g. healthy diet/weight loss, increased physical activity and reduced alcohol and salt consumption). The control group were provided with a Blood Pressure UK (London, UK) leaflet for hypertension and received routine hypertension care. Our asthma intervention (i.e. My Breathing Matters) was a digital intervention to improve the functional quality of life of primary care patients with asthma by supporting illness self-management. Motivational content intended to facilitate use of pharmacological self-management strategies (e.g. medication adherence and appropriate health-care service use) and non-pharmacological self-management strategies (e.g. breathing retraining, stress reduction and healthy behaviour change). The control group were given an Asthma UK (London, UK) information booklet on asthma self-management and received routine asthma care.

Main outcome measures

The primary outcome for the hypertension randomised controlled trial was difference between intervention and usual-care groups in mean systolic blood pressure (mmHg) at 12 months, adjusted for baseline blood pressure, blood pressure target (i.e. standard, diabetic or aged > 80 years), age and general practice. The primary outcome for the asthma feasibility study was the feasibility of the trial design, including recruitment, adherence, intervention engagement and retention at follow-up. Health-care utilisation data were collected via notes review.

Review methods

The quantitative reviews included a meta-analysis. The qualitative review comprised a meta-ethnography.

Results

A total of 622 hypertensive patients were recruited to the randomised controlled trial, and 552 (89%) were followed up at 12 months. Systolic blood pressure was significantly lower in the intervention group at 12 months, with a difference of –3.4 mmHg (95% confidence interval –6.1 to –0.8 mmHg), and this gave an incremental cost per unit of systolic blood pressure reduction of £11 (95% confidence interval £5 to £29). Owing to a cost difference of £402 and a quality-adjusted life-year (QALY) difference of 0.044, long-term modelling puts the incremental cost per QALY at just over £9000. The probability of being cost-effective was 66% at willingness to pay £20,000 per quality-adjusted life-year, and this was higher at higher thresholds. A total of 88 patients were recruited to the asthma feasibility trial (target n = 80; n = 44 in each arm). At 3-month follow-up, two patients withdrew and six patients did not complete outcome measures. At 12 months, two patients withdrew and four patients did not complete outcome measures. A total of 36 out of 44 patients in the intervention group engaged with My Breathing Matters [with a median of four (range 0–25) logins].

Limitations

Although the interventions were designed to be as accessible as was feasible, most trial participants were white and participants of lower socioeconomic status were less likely to take part and complete follow-up measures. Challenges remain in terms of integrating digital interventions with clinical records.

Conclusions

A digital intervention using self-monitored blood pressure to inform medication titration led to significantly lower blood pressure in participants than usual care. The observed reduction in blood pressure would be expected to lead to a reduction of 10–15% in patients suffering a stroke. The feasibility trial of My Breathing Matters suggests that a fully powered randomised controlled trial of the intervention is warranted. The theory-, evidence- and person-based approaches to intervention development refined through this programme enabled us to identify and address important contextual barriers to and facilitators of engagement with the interventions.

Future work

This research justifies consideration of further implementation of the hypertension intervention, a fully powered randomised controlled trial of the asthma intervention and wide dissemination of our methods for intervention development. Our interventions can also be adapted for a range of other health conditions.

Trial and study registration

The trials are registered as ISRCTN13790648 (hypertension) and ISRCTN15698435 (asthma). The studies are registered as PROSPERO CRD42013004773 (hypertension review) and PROSPERO CRD42014013455 (asthma review).

Funding

This project was funded by the National Institute for Health and Care Research (NIHR) Programme Grants for Applied Research programme and will be published in full in Programme Grants for Applied Research; Vol. 10, No. 11. See the NIHR Journals Library website for further information.

Advancing digital health Equity: Directions for behavioral and social science research

The field of digital health is evolving rapidly and encompasses a wide range of complex and changing technologies used to support individual and population health. The COVID-19 pandemic has augmented digital health expansion and significantly changed how digital health technologies are used. To ensure that these technologies do not create or exacerbate existing health disparities, a multi-pronged and comprehensive research approach is needed. In this commentary, we outline five recommendations for behavioral and social science researchers that are critical to promoting digital health equity. These recommendations include: (i) centering equity in research teams and theoretical approaches, (ii) focusing on issues of digital health literacy and engagement, (iii) using methods that elevate perspectives and needs of underserved populations, (iv) ensuring ethical approaches for collecting and using digital health data, and (v) developing strategies for integrating digital health tools within and across systems and settings. Taken together, these recommendations can help advance the science of digital health equity and justice.

How Digital Therapeutics Are Urging the Need for a Paradigm Shift: From Evidence-Based Health Care to Evidence-Based Well-being

A scientific paradigm consists of a set of shared rules, beliefs, values, methods, and instruments for addressing scientific problems. Currently, health care embraces the paradigm of evidence-based health care (EBH). This paradigm prompts health care institutions to base decisions on the best available evidence, which is commonly generated in large-scale randomized controlled trials. We illustrate the application of EBH via the evaluation of drugs. We show how EBH is challenged when it is applied to the evaluation of digital therapeutics, which refers to technology and data to prevent, manage, or treat a medical disorder or disease. We conclude that amid the growing application of digital therapeutics, the paradigm of EBH is challenged in four domains: population, intervention, comparison, outcome. In the second part of this viewpoint, we argue for a paradigm shift in health care so we can optimally evaluate and implement digital therapeutics, and we sketch out the contours of this novel paradigm. We address the need for considering design in health care and evaluation processes, studying user values so that health care can move from a focus on health to well-being, focusing on individual experiences rather than the average, addressing the need for evaluation in authentic use contexts, and stressing the need for continuous evaluation of the dynamic relations between users, context, and digital therapeutics. We conclude that the transition from EBH toward evidence-based well-being would improve the successful implementation of digital technologies in health care.

Personalized Data Science and Personalized (N-of-1) Trials: Promising Paradigms for Individualized Health Care

The term 'data science' usually refers to the process of extracting value from big data obtained from a large group of individuals. An alternative rendition, which we call personalized data science (Per-DS), aims to collect, analyze, and interpret personal data to inform personal decisions. This article describes the main features of Per-DS, and reviews its current state and future outlook. A Per-DS investigation is of, by, and for an individual, the Per-DS investigator, acting simultaneously as her own investigator, study participant, and beneficiary, and making personalized decisions for study design and implementation. The scope of Per-DS studies may include systematic monitoring of physiological or behavioral patterns, case-crossover studies for symptom triggers, pre-post trials for exposure-outcome relationships, and personalized (N-of-1) trials for effectiveness. Per-DS studies produce personal knowledge generalizable to the individual's future self (thus benefiting herself) rather than knowledge generalizable to an external population (thus benefiting others). This endeavor requires a pivot from data mining or extraction to data gardening, analogous to home gardeners producing food for home consumption-the Per-DS investigator needs to 'cultivate the field' by setting goals, specifying study design, identifying necessary data elements, and assembling instruments and tools for data collection. Then, she can implement the study protocol, harvest her personal data, and mine the data to extract personal knowledge. To facilitate Per-DS studies, Per-DS investigators need support from community-based, scientific, philanthropic, business, and government entities, to develop and deploy resources such as peer forums, mobile apps, 'virtual field guides,' and scientific and regulatory guidance.

A review of smartphone applications designed to improve occupational health, safety, and well-being at workplaces

Background

As smartphones become more widespread, software applications for occupational health, safety and well-being (OHS&W) at work are increasing. There is sparse knowledge about the available apps and the research evidence of their effects. This study aims to identify available smartphone applications designed to improve OHS&W at workplaces, and examine to what extent the apps are scientifically validated.

Methods

We searched the Danish App Store and Google Play for free OHS&W apps. Apps were included if they targeted OHS&W and were designed for workplace use. After categorizing the apps, we searched bibliographic databases to identify scientific studies on the ‘intervention apps’.

Results

Altogether, 57 apps were included in the study; 19 apps were categorized as digital sources of information, 37 apps contained an intervention designed for workplace changes, and one app had too sparse information to be classified. Based on the publicly available information about the 37 intervention apps, only 13 had references to research. The bibliographic database search returned 531 publications, resulting in four relevant studies referring to four apps aimed at ergonomic measures, noise exposure, and well-being, which showed either limited effect or methodological limitations.

Conclusion

There is no conceptual clarity about what can be categorized as an OHS&W app. Although some of the apps were developed based on scientific research, there is a need to evaluate the apps’ effects in promoting OHS&W. The sparse documentation of evidence should be kept in mind when applying apps to improve OHS&W.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-022-13821-6.

A Gamified Personalized Normative Feedback App to Reduce Drinking Among Sexual Minority Women: Randomized Controlled Trial and Feasibility Study

BACKGROUND

Sexual minority women disproportionately engage in heavy drinking and shoulder the burden of alcohol dependence. Although several intensive interventions are being developed to meet the needs of treatment-seeking sexual minority women, there remains a lack of preventive interventions to reduce drinking and its consequences among women not yet motivated to reduce their alcohol consumption.

OBJECTIVE

We aimed to examine the feasibility and efficacy of reducing alcohol-related risks via personalized normative feedback (PNF) on alcohol use and coping delivered within LezParlay, a social media-inspired digital competition designed to challenge negative stereotypes about lesbian, bisexual, and queer (LBQ)-identified sexual minority women.

METHODS

Feasibility was assessed by examining engagement with LezParlay outside the context of an incentivized research study, assessing the characteristics of the LBQ women taking part, and examining the competition's ability to derive risk-reducing actual norms as well as levels of acceptability and perceived benefits reported by participants. Intervention efficacy was examined by randomizing a subsample of 499 LBQ alcohol consumers (ie, drinkers) already taking part in the competition to receive sexual identity-specific PNF on alcohol use and coping, alcohol use only, or control topics over only 2 rounds of play. Changes in alcohol use and negative consequences were examined 2 and 4 months after the delivery of treatment PNF.

RESULTS

A total of 2667 diverse LBQ women played ≥1 round of LezParlay. The competition attracted large numbers of moderate and heavy drinkers; however, risk-reducing actual norms could still be derived from competition rounds and featured in PNF. Efficacy results revealed that drinkers who received PNF on alcohol use and both alcohol use and coping had similar reductions in their weekly drinks (P=.003; P<.001), peak drinks (P<.001; P<.001), and negative consequences (P<.001; P<.001) relative to those who received PNF on control topics at the 2-month follow-up. However, at the 4-month follow-up, reductions in alcohol consumption outcomes faded among those who received alcohol PNF only (weekly: P=.06; peak: P=.11), whereas they remained relatively robust among those who received PNF on both alcohol use and coping (weekly: P=.02; peak: P=.03). Finally, participants found the competition highly acceptable and psychologically beneficial as a whole.

CONCLUSIONS

The LezParlay competition was found to be a feasible and efficacious means of reducing alcohol-related risks in this population. Our findings demonstrate the utility of correcting sexual identity-specific drinking and coping norms to reduce alcohol-related risks among LBQ women and suggest that this approach may also prove fruitful in other stigmatized health disparity populations. To engage these populations in the real world and expand the psychological benefits associated with PNF, our findings also point to packaging PNF within a broader, culturally tailored competition designed to challenge negative group stereotypes.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03884478; https://clinicaltrials.gov/ct2/show/NCT03884478.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/24647.

Evaluation Methods Applied to Digital Health Interventions: What Is Being Used beyond Randomised Controlled Trials?-A Scoping Review

Despite the potential of digital health interventions (DHIs), evaluations of their effectiveness face challenges. DHIs are complex interventions and currently established evaluation methods, e.g., the randomised controlled trial (RCT), are limited in their application. This study aimed at identifying alternatives to RCTs as potentially more appropriate evaluation approaches. A scoping review was conducted to provide an overview of existing evaluation methods of DHIs beyond the RCT. Cochrane Central Register of Controlled Trials, MEDLINE, Web of Science, and EMBASE were screened in May 2021 to identify relevant publications, while using defined inclusion and exclusion criteria. Eight studies were extracted for a synthesis comprising four alternative evaluation designs. Factorial designs were mostly used to evaluate DHIs followed by stepped-wedge designs, sequential multiple assignment randomised trials (SMARTs), and micro randomised trials (MRTs). Some of these methods allow for the adaptation of interventions (e.g., SMART or MRT) and the evaluation of specific components of interventions (e.g., factorial designs). Thus, they are appropriate for addressing some specific needs in the evaluation of DHIs. However, it remains unsolved how to establish these alternative evaluation designs in research practice and how to deal with the limitations of the designs.

Understanding and Addressing Occupational Stressors in Internet-Delivered Therapy for Public Safety Personnel: A Qualitative Analysis

Internet-delivered cognitive behavioral therapy (ICBT) is effective when tailored to meet the needs of public safety personnel (PSP). Nevertheless, there is limited research on the nature of the occupational stressors faced by PSP who seek ICBT and how PSP use ICBT to address occupational stressors. We provided tailored ICBT to PSP (N = 126; 54% women) and conducted a qualitative content analysis on clinicians’ eligibility screening notes, clients’ emails, and clients’ survey responses to understand the occupational stressors faced by PSP and their use of ICBT to address such stressors. Clients described several occupational stressors, including operational stressors (e.g., potentially psychologically traumatic events and sleep/shiftwork issues) and organizational stressors (e.g., issues with leadership, resources, and workload). More clients shared occupational concerns during the screening process (97%) than during treatment (58%). The most frequently cited occupational stressor was exposure to potentially psychologically traumatic events. Clients reported using course skills (e.g., controlled breathing and graduated exposure) to manage occupational stressors (e.g., responding to calls, workplace conflict, and work–family conflict). Thought challenging was the most frequently reported strategy used to manage occupational stressors. The current results provide insights into the occupational stressors PSP experience and endeavor to manage using ICBT, which can inform further efforts to tailor ICBT for PSP (e.g., adapting course materials and examples to take into account these operational and occupational stressors).

From Digital Health to Digital Well-being: Systematic Scoping Review

BACKGROUND

Digital health refers to the proper use of technology for improving the health and well-being of people and enhancing the care of patients through the intelligent processing of clinical and genetic data. Despite increasing interest in well-being in both health care and technology, there is no clear understanding of what constitutes well-being, which leads to uncertainty in how to create well-being through digital health. In an effort to clarify this uncertainty, Brey developed a framework to define problems in technology for well-being using the following four categories: epistemological problem, scope problem, specification problem, and aggregation problem.

OBJECTIVE

This systematic scoping review aims to gain insights into how to define and address well-being in digital health.

METHODS

We followed the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist. Papers were identified from 6 databases and included if they addressed the design or evaluation of digital health and reported the enhancement of patient well-being as their purpose. These papers were divided into design and evaluation papers. We studied how the 4 problems in technology for well-being are considered per paper.

RESULTS

A total of 117 studies were eligible for analysis (n=46, 39.3% design papers and n=71, 60.7% evaluation papers). For the epistemological problem, the thematic analysis resulted in various definitions of well-being, which were grouped into the following seven values: healthy body, functional me, healthy mind, happy me, social me, self-managing me, and external conditions. Design papers mostly considered well-being as healthy body and self-managing me, whereas evaluation papers considered the values of healthy mind and happy me. Users were rarely involved in defining well-being. For the scope problem, patients with chronic care needs were commonly considered as the main users. Design papers also regularly involved other users, such as caregivers and relatives. These users were often not involved in evaluation papers. For the specification problem, most design and evaluation papers focused on the provision of care support through a digital platform. Design papers used numerous design methods, whereas evaluation papers mostly considered pre-post measurements and randomized controlled trials. For the aggregation problem, value conflicts were rarely described.

CONCLUSIONS

Current practice has found pragmatic ways of circumventing or dealing with the problems of digital health for well-being. Major differences exist between the design and evaluation of digital health, particularly regarding their conceptualization of well-being and the types of users studied. In addition, we found that current methodologies for designing and evaluating digital health can be improved. For optimal digital health for well-being, multidisciplinary collaborations that move beyond the common dichotomy of design and evaluation are needed.

Informing the Future of Integrated Digital and Clinical Mental Health Care: Synthesis of the Outcomes From Project Synergy

BACKGROUND

Globally, there are fundamental shortcomings in mental health care systems, including restricted access, siloed services, interventions that are poorly matched to service users' needs, underuse of personal outcome monitoring to track progress, exclusion of family and carers, and suboptimal experiences of care. Health information technologies (HITs) hold great potential to improve these aspects that underpin the enhanced quality of mental health care.

OBJECTIVE

Project Synergy aimed to co-design, implement, and evaluate novel HITs, as exemplified by the InnoWell Platform, to work with standard health care organizations. The goals were to deliver improved outcomes for specific populations under focus and support organizations to enact significant system-level reforms.

METHODS

Participating health care organizations included the following: Open Arms-Veterans & Families Counselling (in Sydney and Lismore, New South Wales [NSW]); NSW North Coast headspace centers for youth (Port Macquarie, Coffs Harbour, Grafton, Lismore, and Tweed Heads); the Butterfly Foundation's National Helpline for eating disorders; Kildare Road Medical Centre for enhanced primary care; and Connect to Wellbeing North Coast NSW (administered by Neami National), for population-based intake and assessment. Service users, families and carers, health professionals, and administrators of services across Australia were actively engaged in the configuration of the InnoWell Platform to meet service needs, identify barriers to and facilitators of quality mental health care, and highlight potentially the best points in the service pathway to integrate the InnoWell Platform. The locally configured InnoWell Platform was then implemented within the respective services. A mixed methods approach, including surveys, semistructured interviews, and workshops, was used to evaluate the impact of the InnoWell Platform. A participatory systems modeling approach involving co-design with local stakeholders was also undertaken to simulate the likely impact of the platform in combination with other services being considered for implementation within the North Coast Primary Health Network to explore resulting impacts on mental health outcomes, including suicide prevention.

RESULTS

Despite overwhelming support for integrating digital health solutions into mental health service settings and promising impacts of the platform simulated under idealized implementation conditions, our results emphasized that successful implementation is dependent on health professional and service readiness for change, leadership at the local service level, the appropriateness and responsiveness of the technology for the target end users, and, critically, funding models being available to support implementation. The key places of interoperability of digital solutions and a willingness to use technology to coordinate health care system use were also highlighted.

CONCLUSIONS

Although the COVID-19 pandemic has resulted in the widespread acceptance of very basic digital health solutions, Project Synergy highlights the critical need to support equity of access to HITs, provide funding for digital infrastructure and digital mental health care, and actively promote the use of technology-enabled, coordinated systems of care.

Effectiveness of a chatbot for eating disorders prevention: A randomized clinical trial

OBJECTIVE

Prevention of eating disorders (EDs) is of high importance. However, digital programs with human moderation are unlikely to be disseminated widely. The aim of this study was to test whether a chatbot (i.e., computer program simulating human conversation) would significantly reduce ED risk factors (i.e., weight/shape concerns, thin-ideal internalization) in women at high risk for an ED, compared to waitlist control, as well as whether it would significantly reduce overall ED psychopathology, depression, and anxiety and prevent ED onset.

METHOD

Women who screened as high risk for an ED were randomized (N = 700) to (1) chatbot based on the StudentBodies© program; or (2) waitlist control. Participants were followed for 6 months.

RESULTS

For weight/shape concerns, there was a significantly greater reduction in intervention versus control at 3- (d = -0.20; p = .03) and 6-m-follow-up (d = -0.19; p = .04). There were no differences in change in thin-ideal internalization. The intervention was associated with significantly greater reductions than control in overall ED psychopathology at 3- (d = -0.29; p = .003) but not 6-month follow-up. There were no differences in change in depression or anxiety. The odds of remaining nonclinical for EDs were significantly higher in intervention versus control at both 3- (OR = 2.37, 95% CI [1.37, 4.11]) and 6-month follow-ups (OR = 2.13, 95% CI [1.26, 3.59]).

DISCUSSION

Findings provide support for the use of a chatbot-based EDs prevention program in reducing weight/shape concerns through 6-month follow-up, as well as in reducing overall ED psychopathology, at least in the shorter-term. Results also suggest the intervention may reduce ED onset.

PUBLIC SIGNIFICANCE

We found that a chatbot, or a computer program simulating human conversation, based on an established, cognitive-behavioral therapy-based eating disorders prevention program, was successful in reducing women's concerns about weight and shape through 6-month follow-up and that it may actually reduce eating disorder onset. These findings are important because this intervention, which uses a rather simple text-based approach, can easily be disseminated in order to prevent these deadly illnesses.

TRIAL REGISTRATION

OSF Registries; https://osf.io/7zmbv.

An overview of and recommendations for more accessible digital mental health services

Mental health concerns are common, and various evidence-based interventions for mental health conditions have been developed. However, many people have difficulty accessing appropriate mental health care and this has been exacerbated by the COVID-19 pandemic. Digital mental health services, such as those delivered by mobile phone or web-based platforms, offer the possibility of expanding the reach and accessibility of mental health care. To achieve this goal, digital mental health interventions and plans for their implementation must be designed with the end users in mind. In this Review, we describe the evidence base for digital mental health interventions across various diagnoses and treatment targets. Then, we explain the different formats for digital mental health intervention delivery, and offer considerations for their use across key age groups. We discuss the role that the COVID-19 pandemic has played in emphasizing the value of these interventions, and offer considerations for ensuring equity in access to digital mental health interventions among diverse populations. As healthcare providers continue to embrace the role that technology can play in broadening access to care, the design and implementation of digital mental healthcare solutions must be carefully considered to maximize their effectiveness and accessibility.

User Experience, Engagement, and Popularity in Mental Health Apps: Secondary Analysis of App Analytics and Expert App Reviews

Background

User experience and engagement are critical elements of mental health apps’ abilities to support users. However, work examining the relationships among user experience, engagement, and popularity has been limited. Understanding how user experience relates to engagement with and popularity of mental health apps can demonstrate the relationship between subjective and objective measures of app use. In turn, this may inform efforts to develop more effective and appealing mental health apps and ensure that they reach wide audiences.

Objective

We aimed to examine the relationship among subjective measures of user experience, objective measures of popularity, and engagement in mental health apps.

Methods

We conducted a preregistered secondary data analysis in a sample of 56 mental health apps. To measure user experience, we used expert ratings on the Mobile App Rating Scale (MARS) and consumer ratings from the Apple App Store and Google Play. To measure engagement, we acquired estimates of monthly active users (MAU) and user retention. To measure app popularity, we used download count, total app revenue, and MAU again.

Results

MARS total score was moderately positively correlated with app-level revenue (Kendall rank [T]=0.30, P=.002), MAU (T=0.39, P<.001), and downloads (T=0.41, P<.001). However, the MARS total score and each of its subscales (Engagement, Functionality, Aesthetics, and Information) showed extremely small correlations with user retention 1, 7, and 30 days after downloading. Furthermore, the total MARS score only correlated with app store rating at T=0.12, which, at P=.20, did not meet our threshold for significance.

Conclusions

More popular mental health apps receive better ratings of user experience than less popular ones. However, user experience does not predict sustained engagement with mental health apps. Thus, mental health app developers and evaluators need to better understand user experience and engagement, as well as to define sustained engagement, what leads to it, and how to create products that achieve it. This understanding might be supported by better collaboration between industry and academic teams to advance a science of engagement.

The Challenges in Designing a Prevention Chatbot for Eating Disorders: Observational Study

BACKGROUND

Chatbots have the potential to provide cost-effective mental health prevention programs at scale and increase interactivity, ease of use, and accessibility of intervention programs.

OBJECTIVE

The development of chatbot prevention for eating disorders (EDs) is still in its infancy. Our aim is to present examples of and solutions to challenges in designing and refining a rule-based prevention chatbot program for EDs, targeted at adult women at risk for developing an ED.

METHODS

Participants were 2409 individuals who at least began to use an EDs prevention chatbot in response to social media advertising. Over 6 months, the research team reviewed up to 52,129 comments from these users to identify inappropriate responses that negatively impacted users' experience and technical glitches. Problems identified by reviewers were then presented to the entire research team, who then generated possible solutions and implemented new responses.

RESULTS

The most common problem with the chatbot was a general limitation in understanding and responding appropriately to unanticipated user responses. We developed several workarounds to limit these problems while retaining some interactivity.

CONCLUSIONS

Rule-based chatbots have the potential to reach large populations at low cost but are limited in understanding and responding appropriately to unanticipated user responses. They can be most effective in providing information and simple conversations. Workarounds can reduce conversation errors.

Efficacy, Effectiveness, and Quality of Resilience-Building Mobile Health Apps for Military, Veteran, and Public Safety Personnel Populations: Scoping Literature Review and App Evaluation

BACKGROUND

Military members (MMs) and public safety personnel (PSP) are vulnerable to occupational stress injuries because of their job demands. When MMs and PSP transition out of these professions, they may continue to experience mental health challenges. The development and implementation of resilience-building mobile health (mHealth) apps as an emergent mental health intervention platform has allowed for targeted, cost-effective, and easily accessible treatment when in-person therapy may be limited or unavailable. However, current mHealth app development is not regulated, and often lacks both clear evidence-based research and the input of health care professionals.

OBJECTIVE

This study aims to evaluate the evidence-based quality, efficacy, and effectiveness of resilience-building mobile apps targeted toward the MMs, PSP, and veteran populations via a scoping literature review of the current evidence base regarding resilience apps for these populations and an evaluation of free resilience apps designed for use among these populations.

METHODS

The studies were selected using a comprehensive search of MEDLINE, CINAHL Plus, PsycINFO, SocINDEX, Academic Search Complete, Embase, and Google and were guided by PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews). A narrative synthesis of the resulting papers was performed. The Alberta Rating Index for Apps was used to conduct a review of each of the identified apps. The inclusion criteria consisted of apps that were free to download in either the Google Play Store or the Apple App Store; updated within the last 3 years; available in English and in Canada; and intended for use by MMs, veterans, and PSP.

RESULTS

In total, 22 apps met the inclusion criteria for evaluation. The resilience strategies offered by most apps included psychoeducation, mindfulness, cognitive behavioral therapy, and acceptance and commitment therapy. Overall, 50% (11/22) of apps had been tested in randomized controlled trials, 7 (32%) apps had been evaluated using other research methods, and 5 (23%) apps had not been studied. Using the Alberta Rating Index for Apps, the app scores ranged from 37 to 56 out of 72, with higher rated apps demonstrating increased usability and security features.

CONCLUSIONS

The mHealth apps reviewed are well-suited to providing resilience strategies for MMs, PSP, and veterans. They offer easy accessibility to evidence-based tools while working to encourage the use of emotional and professional support with safety in mind. Although not intended to function as a substitute for professional services, research has demonstrated that mHealth apps have the potential to foster a significant reduction in symptom severity for posttraumatic stress disorder, depression, anxiety, and other mental health conditions. In clinical practice, apps can be used to supplement treatment and provide clients with population-specific confidential tools to increase engagement in the treatment process.

Decision-making given surrogate outcomes

BACKGROUND

Opinions differ about the extent to which intervention research should and can directly assess the main patient-important health outcomes, what role surrogate endpoints can play, and which requirements should then apply to the scientific underpinning of clinical and policy decisions.

METHOD

In a commentary we elaborate on this and provide guidance for dealing with related dilemmas.

CONCLUSIONS

Ethical, methodological and practical reasons for decision making based on surrogate endpoints can be that (1) reaching the intended patient-important health outcome would take too long to await direct RCT-based evidence, (2) experimental conditions have limited sustainability over time; and (3) the plausibility of an intervention's clinical efficacy, given the already available evidence regarding surrogate endpoints, goes beyond equipoise. Given an expected increase of interventions with a long term patient-important health outcome perspective, dealing with surrogate endpoints will remain an important challenge. Appropriately dealing with a surrogate endpoint includes (1) the assessment of its predictive value for the intended patient-important outcome, where GRADE guidelines for assessing 'indirectness' and 'causal chain analysis' can be helpful; (2) transparency of (absence of) evidence; (3) adequately updating the 'knowledge mosaic'; (4) weighing different perspectives and values, and (5) monitoring whether adjustments need to be made. The remaining level of uncertainty must be balanced against the urgency of clinical or societal decision making and the disadvantages of postponing this. Criteria for using surrogate endpoints are suggested. Patients, citizens and policy makers can be involved in agreeing upon these criteria.

Web-Based Structured Education for Type 2 Diabetes: Interdisciplinary User-Centered Design Approach

BACKGROUND

Digital health research encompasses methods from human-computer interaction and health research.

OBJECTIVE

This paper aims to describe how these methods were combined to develop HeLP-Diabetes: Starting Out, a web-based structured education program for people newly diagnosed with type 2 diabetes.

METHODS

The development process consisted of three phases: initial design for effectiveness, optimization for usability, and in the wild testing in the National Health Service with people newly diagnosed with type 2 diabetes, and further revisions. We adopted an iterative user-centered approach and followed steps from the human-computer interaction design life cycle and the Medical Research Council guidelines on developing and evaluating complex interventions.

RESULTS

The initial design process resulted in an 8-session program containing information and behavior change techniques targeting weight loss, being more active, and taking medication. The usability testing was highlighted at an early stage, where changes needed to be made to the language and layout of the program. The in the wild testing provided data on uptake of and barriers to use. The study suggested low uptake and completion of the program, but those who used it seemed to benefit from it. The qualitative findings suggested that barriers to use included an expectation that the program would take too long. This informed refinements to the program.

CONCLUSIONS

The use of interdisciplinary methods resulted in an iterative development process and refinements to the program that were based on user needs and data on uptake. The final intervention was more suitable for a definitive evaluation than the initial version. The description of our approach informs other digital health researchers on how to make interventions more sensitive to user needs.

A national effectiveness trial of an eHealth program to prevent alcohol and cannabis misuse: responding to the replication crisis

BACKGROUND

The burden of disease attributable to alcohol and other drug (AOD) use in young people is considerable. Prevention can be effective, yet few programs have demonstrated replicable effects. This study aimed to replicate research behind Climate Schools: Alcohol and Cannabis course among a large cohort of adolescents.

METHODS

Seventy-one secondary schools across three States participated in a cluster-randomised controlled trial. Year 8 students received either the web-based Climate Schools: Alcohol and Cannabis course (Climate, n = 3236), or health education as usual (Control, n = 3150). Outcomes were measured via self-report and reported here for baseline, 6- and 12-months for alcohol and cannabis knowledge, alcohol, cannabis use and alcohol-related harms.

RESULTS

Compared to Controls, students in the Climate group showed greater increases in alcohol- [standardised mean difference (SMD) 0.51, p < 0.001] and cannabis-related knowledge (SMD 0.49, p < 0.001), less increases in the odds of drinking a full standard drink[(odds ratio (OR) 0.62, p = 0.014], and heavy episodic drinking (OR 0.49, p = 0.022). There was no evidence for differences in change over time in the odds of cannabis use (OR 0.57, p = 0.22) or alcohol harms (OR 0.73, p = 0.17).

CONCLUSIONS

The current study provides support for the effectiveness of the web-based Climate Schools: Alcohol and Cannabis course in increasing knowledge and reducing the uptake of alcohol. It represents one of the first trials of a web-based AOD prevention program to replicate alcohol effects in a large and diverse sample of students. Future research and/or adaptation of the program may be warranted with respect to prevention of cannabis use and alcohol harms.

Centering Lived Experience in Developing Digital Interventions for Suicide and Self-injurious Behaviors: User-Centered Design Approach

BACKGROUND

The prevalence of self-injurious thoughts and behaviors (SITB) signals a growing public health crisis. Despite a recognized need for improved and scalable interventions, the field of SITB intervention faces several challenges: existing interventions are often time and resource intensive, most individuals with SITB do not seek formal mental health care, and efficacious treatments are characterized by small effects. Combined, these challenges indicate a need for improved SITB interventions for individuals in formal treatment and those who are not treatment engaged but are at high risk of worsening mental health and future suicide attempts.

OBJECTIVE

We present a methodological approach and set of techniques that may address these challenges by centering the lived experience of individuals with SITB in the process of developing needed services: user-centered design (UCD).

METHODS

We highlight the value of UCD in the context of digital interventions for SITB by describing the UCD approach and explicating how it can be leveraged to include lived experience throughout the development and evaluation process. We provide a detailed case example highlighting 3 phases of the early development process that can be used to design an intervention that is engaging and meets end-user needs. In addition, we point to novel applications of UCD to complement new directions in SITB research.

RESULTS

In this paper, we offer a 2-pronged approach to meet these challenges. First, in terms of addressing access to effective interventions, digital interventions hold promise to extend the reach of evidence-based treatments outside of brick-and-mortar health care settings. Second, to address challenges related to treatment targets and engagement, we propose involving individuals with lived experience in the design and research process.

CONCLUSIONS

UCD offers a well-developed and systematic process to center the unique needs, preferences, and perceived barriers of individuals with lived SITB experience in the development and evaluation of digital interventions.

Insights into internet-delivered cognitive behavioural therapy for public safety personnel: Exploration of client experiences during and after treatment

Canadian public safety personnel (PSP) experience high rates of mental health problems and barriers to receiving care. Internet-delivered cognitive behavioural therapy (ICBT) may help reduce barriers to care; however, there is no literature involving qualitative analyses of client feedback to describe PSP experiences with ICBT. Identifying these experiences is important because it can inform future use of ICBT with this group that has unique needs. The current study was designed to explore how clients (N = 82) experienced ICBT that had been tailored to meet their needs; specifically, the study assessed their perceptions of program impacts, what clients found helpful, and client suggestions for improvements. The ICBT course included five core lessons, client stories, and nine initial additional resources, as well as flexible frequencies (optional, once weekly, or twice weekly) and durations (8 to 16 weeks) of therapist support. A qualitative reliability thematic analysis was used to analyze client communications and feedback. Responses to a Treatment Satisfaction Questionnaire administered at eight weeks post-enrollment were available for 57 clients. Client emails with therapists were also examined among all clients, including an additional 25 clients who did not complete the Treatment Satisfaction Questionnaire. Themes identified in the qualitative analyses were related to: reported impacts and hindering events, helpful and challenging course skills and content, helpful aspects of the course, and areas for improvement. Clients who completed the Treatment Satisfaction Questionnaire and those who did not reported beneficial impacts from the program, with the most commonly endorsed themes being skill development and normalizing mental health issues. Hindering events were experienced by both groups and included timeline challenges, technology challenges, and negative effects. Comments from both groups suggested that clients had more success than challenges when practicing the skills. Thought challenging was the skill most frequently identified as helpful. Clients described many aspects of the program as helpful with the most frequently endorsed themes being the course format and content, the flexible nature of the course, access to additional materials and case stories, and therapist assistance. Clients also provided suggestions for improving the course (e.g., case stories, additional resources, timelines audio and videos). Overall, client communications suggest that ICBT is accepted and perceived as beneficial among PSP. These results informed rapid improvements to the ICBT program tailored for PSP and may inform others seeking to provide digital mental health services to PSP.

A web-based, peer-supported self-management intervention to reduce distress in relatives of people with psychosis or bipolar disorder: the REACT RCT

BACKGROUND

Relatives caring for people with severe mental health problems find information and emotional support hard to access. Online support for self-management offers a potential solution.

OBJECTIVE

The objective was to determine the clinical effectiveness and cost-effectiveness of an online supported self-management tool for relatives: the Relatives' Education And Coping Toolkit (REACT).

DESIGN AND SETTING

This was a primarily online (UK), single-blind, randomised controlled trial, comparing REACT plus a resource directory and treatment as usual with the resource directory and treatment as usual only, by measuring user distress and other well-being measures at baseline and at 12 and 24 weeks.

PARTICIPANTS

A total of 800 relatives of people with severe mental health problems across the UK took part; relatives who were aged ≥ 16 years, were experiencing high levels of distress, had access to the internet and were actively seeking help were recruited.

INTERVENTION

REACT comprised 12 psychoeducation modules, peer support through a group forum, confidential messaging and a comprehensive resource directory of national support. Trained relatives moderated the forum and responded to messages.

MAIN OUTCOME MEASURE

The main outcome was the level of participants' distress, as measured by the General Health Questionnaire-28 items.

RESULTS

Various online and offline strategies, including social media, directed potential participants to the website. Participants were randomised to one of two arms: REACT plus the resource directory (n = 399) or the resource directory only (n = 401). Retention at 24 weeks was 75% (REACT arm, n = 292; resource directory-only arm, n = 307). The mean scores for the General Health Questionnaire-28 items reduced substantially across both arms over 24 weeks, from 40.2 (standard deviation 14.3) to 30.5 (standard deviation 15.6), with no significant difference between arms (mean difference -1.39, 95% confidence interval -3.60 to 0.83; p = 0.22). At 12 weeks, the General Health Questionnaire-28 items scores were lower in the REACT arm than in the resource directory-only arm (-2.08, 95% confidence interval -4.14 to -0.03; p = 0.027), but this finding is likely to be of limited clinical significance. Accounting for missing data, which were associated with higher distress in the REACT arm (0.33, 95% confidence interval -0.27 to 0.93; p = 0.279), in a longitudinal model, there was no significant difference between arms over 24 weeks (-0.56, 95% confidence interval -2.34 to 1.22; p = 0.51). REACT cost £142.95 per participant to design and deliver (£62.27 for delivery only), compared with £0.84 for the resource directory only. A health economic analysis of NHS, health and Personal Social Services outcomes found that REACT has higher costs (£286.77), slightly better General Health Questionnaire-28 items scores (incremental General Health Questionnaire-28 items score adjusted for baseline, age and gender: -1.152, 95% confidence interval -3.370 to 1.065) and slightly lower quality-adjusted life-year gains than the resource directory only; none of these differences was statistically significant. The median time spent online was 50.8 minutes (interquartile range 12.4-172.1 minutes) for REACT, with no significant association with outcome. Participants reported finding REACT a safe, confidential environment (96%) and reported feeling supported by the forum (89%) and the REACT supporters (86%). No serious adverse events were reported.

LIMITATIONS

The sample comprised predominantly white British females, 25% of participants were lost to follow-up and dropout in the REACT arm was not random.

CONCLUSIONS

An online self-management support toolkit with a moderated group forum is acceptable to relatives and, compared with face-to-face programmes, offers inexpensive, safe delivery of National Institute for Health and Care Excellence-recommended support to engage relatives as peers in care delivery. However, currently, REACT plus the resource directory is no more effective at reducing relatives' distress than the resource directory only.

FUTURE WORK

Further research in improving the effectiveness of online carer support interventions is required.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN72019945.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 24, No. 32. See the NIHR Journals Library website for further project information.

Technology and psychotherapeutic interventions: Bibliometric analysis of the past four decades

BACKGROUND

Rapid growth of the integration of technology and psychotherapeutic interventions has been noted, but no clear quantification of this growth has been done.

AIMS

This bibliometric analysis seeks to quantify the growth, trends, and applications of technology in psychotherapeutic interventions over the last 40 years.

METHODS

Searches were conducted in the Web of Science (WOS) database for all existing technology-psychotherapy-related publications from 1981 to October 2020. Search terms were refined using a systematic screening strategy, based upon Cochrane protocol, generating 52 technology terms. Analyses across 40 years and by decade from 1981 to 2020 were conducted.

RESULTS

A total of 13,934 peer-reviewed articles were identified. Yearly publication rate has increased from one in 1981 to 1902 by October 2020. The growth rate of publications across decades consistently tripled in size (762.50% from the 1980s to 1990s, 539.71% from the 1990s to 2000s, and 337.24% from the 2000s to 2010s). The author, country, journal, and institution with the most publications were Andersson, G., USA, Journal of Medical Internet Research, and Karolinska Institute, respectively. The most frequent technology search term across all four decades was "internet*." The trends in percentages of peer-reviewed publications within each decade showed: 1) a declining trend for the term "computer," 2) an upward trend for the combined terms, "internet," "online," and "web," 3) and a steady but smaller proportion of publications for other terms ("cell phone," "phone/telephone," "technology," "video," "virtual reality or VR," "apps," "digital," "machine learning," "electronic," "robo," and "telehealth").

DISCUSSION

The rapid growth and trends identified in technology and psychotherapy publications can inform related policies addressing the role of technology in mental health. Moreover, pattern analyses may provide direction for a standard nomenclature to address terminology usage inconsistencies across the field.

An iterative, interdisciplinary, collaborative framework for developing and evaluating digital behavior change interventions

The rapid expansion of technology promises to transform the behavior science field by revolutionizing the ways in which individuals can monitor and improve their health behaviors. To fully live into this promise, the behavior science field must address distinct challenges, including: building interventions that are not only scientifically sound but also engaging; using evaluation methods to precisely assess intervention components for intervention optimization; and building personalized interventions that acknowledge and adapt to the dynamic ecosystem of individual and contextual variables that impact behavior change. The purpose of this paper is to provide a framework to address these challenges by leveraging behavior science, human-centered design, and data science expertise throughout the cycle of developing and evaluating digital behavior change interventions (DBCIs). To define this framework, we reviewed current models and practices for intervention development and evaluation, as well as technology industry models for product development. The framework promotes an iterative process, aiming to maximize outcomes by incorporating faster and more frequent testing cycles into the lifecycle of a DBCI. Within the framework provided, we describe each phase, from development to evaluation, to discuss the optimal practices, necessary stakeholders, and proposed evaluation methods. The proposed framework may inform practices in both academia and industry, as well as highlight the need to offer collaborative platforms to ensure successful partnerships that can lead to more effective DBCIs that reach broad and diverse populations.

Collaborative care for depression management in primary care: A randomized roll-out trial using a type 2 hybrid effectiveness-implementation design

Background

The Collaborative Care Model (CoCM) is a well-established treatment for depression in primary care settings. The critical drivers and specific strategies for improving implementation and sustainment are largely unknown. Rigorous pragmatic research is needed to understand CoCM implementation processes and outcomes.

Methods

This study is a hybrid Type 2 randomized roll-out effectiveness-implementation trial of CoCM in 11 primary care practices affiliated with an academic medical center. The Collaborative Behavioral Health Program (CBHP) was developed as a means of improving access to effective mental health services for depression. Implementation strategies are provided to all practices. Using a sequential mixed methods approach, we will assess key stakeholders’ perspectives on barriers and facilitators of implementation and sustainability of CBHP. The speed and quantity of implementation activities completed over a 30-month period for each practice will be assessed. Economic analyses will be conducted to determine the budget impact and cost offset of CBHP in the healthcare system. We hypothesize that CBHP will be effective in reducing depressive symptoms and spillover effects on chronic health conditions. We will also examine differential outcomes among racial/ethnic minority patients.

Discussion

This study will elucidate critical drivers of successful CoCM implementation. It will be among the first to conduct economic analyses on a fee-for-service model utilizing billing codes for CoCM. Data may inform ways to improve implementation efficiency with an optimization approach to successive practices due to the roll-out design. Changes to the protocol and current status of the study are discussed.

Evaluating mobile health technologies: does the traditional randomized controlled trial serve our needs?

Mobile health technologies are increasingly used by cardiovascular nurses and allied health clinicians to support behaviour change and self-care in patients. Randomized controlled trials provide rigorous methodology for evaluation but progress slowly, so the technology is often outdated by completion. A helpful modification to the trial design includes identification of the behaviour intervention principles involved and concurrently track in-app usage data, allowing updates to be made. These data collected alongside user experiences enables analysis of effects on outcomes and determination of the relative influence of component parts. Cardiovascular and mental health mHealth strategies are used to exemplify these recommendations.

Health technology assessment for digital technologies that manage chronic disease: a systematic review

OBJECTIVE

A growing number of evaluation frameworks have emerged over recent years addressing the unique benefits and risk profiles of new classes of digital health technologies (DHTs). This systematic review aims to identify relevant frameworks and synthesize their recommendations into DHT-specific content to be considered when performing Health Technology Assessments (HTAs) for DHTs that manage chronic noncommunicable disease at home.

METHODS

Searches were undertaken of Medline, Embase, Econlit, CINAHL, and The Cochrane Library (January 2015 to March 2020), and relevant gray literature (January 2015 to August 2020) using keywords related to HTA, evaluation frameworks, and DHTs. Included framework reference lists were searched from 2010 until 2015. The EUNetHTA HTA Core Model version 3.0 was selected as a scaffold for content evaluation.

RESULTS

Forty-four frameworks were identified, mainly covering clinical effectiveness (n = 30) and safety (n = 23) issues. DHT-specific content recommended by framework authors fell within 28 of the 145 HTA Core Model issues. A further twenty-two DHT-specific issues not currently in the HTA Core Model were recommended.

CONCLUSIONS

Current HTA frameworks are unlikely to be sufficient for assessing DHTs. The development of DHT-specific content for HTA frameworks is hampered by DHTs having varied benefit and risk profiles. By focusing on DHTs that actively monitor/treat chronic noncommunicable diseases at home, we have extended DHT-specific content to all nine HTA Core Model domains. We plan to develop a supplementary evaluation framework for designing research studies, undertaking HTAs, and appraising the completeness of HTAs for DHTs.