1
|
Raje A, Rozatkar AR, Mehta UM, Shrivastava R, Bondre A, Ahmad MA, Malviya A, Sen Y, Tugnawat D, Bhan A, Modak T, Das N, Nagendra S, Lane E, Castillo J, Naslund JA, Torous J, Choudhary S. Designing smartphone-based cognitive assessments for schizophrenia: Perspectives from a multisite study. Schizophr Res Cogn 2025; 40:100347. [PMID: 39995813 PMCID: PMC11848491 DOI: 10.1016/j.scog.2025.100347] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2024] [Revised: 02/03/2025] [Accepted: 02/03/2025] [Indexed: 02/26/2025]
Abstract
Introduction Cognitive deficits represent a core symptom of schizophrenia and a principal contributor to illness disability, yet evaluating cognition in routine clinical settings is often not feasible as cognitive assessments take longer than a standard doctor's visit. Using smartphones to assess cognition in schizophrenia offers the advantages of convenience in that patients can complete assessments outside of the clinic, temporality in that longitudinal trends can be identified, and contextuality in that cognitive scores can be interpreted with other measures captured by the phone (e.g. sleep). The current study aims to design a battery of cognitive assessments corresponding to the MATRICs Consensus Battery of Cognition (MCCB), in partnership with people living with schizophrenia. Methodology Focus group discussions (FGDs) and interviews were conducted with people diagnosed with schizophrenia across three sites (Boston, Bhopal, and Bangalore) to help design, refine, and assess the proposed smartphone battery of cognitive tests on the mindLAMP app. Interviews were conducted between December 2023 and March 2024. Inductive thematic analysis was used to analyze data. Results Participants found the app and its proposed cognitive assessments to be acceptable, helpful, and easy to use. They particularly found the gamified nature of the cognitive tests to be appealing and engaging. However, they also proposed ways to further increase engagement by including more information about each cognitive test, more visual instructions, and more information about scoring. Across all sites, there were many similarities in themes. Discussion & conclusion People living with schizophrenia, from different sites in the US and India, appear interested in using smartphone apps to track their cognition. Thematic analysis reinforces the importance of feedback and data sharing, although this presents a challenge, given the novel nature of smartphone-based cognitive measures that have not yet been standardized or validated.
Collapse
Affiliation(s)
| | | | | | | | | | | | | | | | | | | | - Tamonud Modak
- All India Institute of Medical Sciences, Bhopal, India
| | - Nabagata Das
- National Institute of Mental Health and Neurosciences, Bengaluru, India
| | | | - Erlend Lane
- Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, USA
| | - Juan Castillo
- Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, USA
| | - John A. Naslund
- Department of Global Health and Social Medicine, Harvard Medical School, Boston, USA
| | - John Torous
- Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, USA
| | - Soumya Choudhary
- National Institute of Mental Health and Neurosciences, Bengaluru, India
| |
Collapse
|
2
|
Torous J, Linardon J, Goldberg SB, Sun S, Bell I, Nicholas J, Hassan L, Hua Y, Milton A, Firth J. The evolving field of digital mental health: current evidence and implementation issues for smartphone apps, generative artificial intelligence, and virtual reality. World Psychiatry 2025; 24:156-174. [PMID: 40371757 PMCID: PMC12079407 DOI: 10.1002/wps.21299] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 05/16/2025] Open
Abstract
The expanding domain of digital mental health is transitioning beyond traditional telehealth to incorporate smartphone apps, virtual reality, and generative artificial intelligence, including large language models. While industry setbacks and methodological critiques have highlighted gaps in evidence and challenges in scaling these technologies, emerging solutions rooted in co-design, rigorous evaluation, and implementation science offer promising pathways forward. This paper underscores the dual necessity of advancing the scientific foundations of digital mental health and increasing its real-world applicability through five themes. First, we discuss recent technological advances in digital phenotyping, virtual reality, and generative artificial intelligence. Progress in this latter area, specifically designed to create new outputs such as conversations and images, holds unique potential for the mental health field. Given the spread of smartphone apps, we then evaluate the evidence supporting their utility across various mental health contexts, including well-being, depression, anxiety, schizophrenia, eating disorders, and substance use disorders. This broad view of the field highlights the need for a new generation of more rigorous, placebo-controlled, and real-world studies. We subsequently explore engagement challenges that hamper all digital mental health tools, and propose solutions, including human support, digital navigators, just-in-time adaptive interventions, and personalized approaches. We then analyze implementation issues, emphasizing clinician engagement, service integration, and scalable delivery models. We finally consider the need to ensure that innovations work for all people and thus can bridge digital health disparities, reviewing the evidence on tailoring digital tools for historically marginalized populations and low- and middle-income countries. Regarding digital mental health innovations as tools to augment and extend care, we conclude that smartphone apps, virtual reality, and large language models can positively impact mental health care if deployed correctly.
Collapse
Affiliation(s)
- John Torous
- Division of Digital Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| | - Jake Linardon
- SEED Lifespan Strategic Research Centre, School of Psychology, Faculty of Health, Deakin University, Geelong, VIC, Australia
| | - Simon B Goldberg
- Department of Counseling Psychology and Center for Healthy Minds, University of Wisconsin, Madison, WI, USA
| | - Shufang Sun
- Department of Behavioral and Social Sciences, Brown University School of Public Health, Providence, RI, USA
- Mindfulness Center, Brown University, Providence, RI, USA
- Center for Global Public Health, Brown University, Providence, RI, USA
| | - Imogen Bell
- Orygen, Parkville, VIC, Australia
- Centre for Youth Mental Health, University of Melbourne, Melbourne, VIC, Australia
| | - Jennifer Nicholas
- Mindfulness Center, Brown University, Providence, RI, USA
- Centre for Youth Mental Health, University of Melbourne, Melbourne, VIC, Australia
| | - Lamiece Hassan
- School for Health Sciences, University of Manchester, Manchester, UK
| | - Yining Hua
- Division of Digital Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
- Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| | - Alyssa Milton
- Central Clinical School, Faculty of Medicine and Health, University of Sydney, Sydney, NSW, Australia
- Australian Research Council (ARC) Centre of Excellence for Children and Families Over the Life, Sydney, NSW, Australia
| | - Joseph Firth
- Division of Psychology and Mental Health, University of Manchester, and Greater Manchester Mental Health NHS Foundation Trust, Manchester Academic Health Science Centre, Manchester, UK
| |
Collapse
|
3
|
Calvert E, Cipriani M, Chen K, Dhima A, Burns J, Torous J. Evaluating clinical outcomes for anxiety and depression: A real-world comparison of the digital clinic and primary care. J Affect Disord 2025; 377:275-283. [PMID: 39988138 DOI: 10.1016/j.jad.2025.02.051] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/09/2024] [Revised: 02/06/2025] [Accepted: 02/17/2025] [Indexed: 02/25/2025]
Abstract
BACKGROUND Hybrid care models that blend synchronous and asynchronous tools have shown promise in increasing access to care. However, few studies have compared their effectiveness to an appropriate control group, and no RCT has been conducted to tease apart the role of the clinician, app and Digital Navigator components. OBJECTIVE We aim to evaluate the effectiveness of a blended hybrid care model, the Digital Clinic, in reducing anxiety and depression, compared to a large primary care control group. METHODS Effectiveness was assessed by comparing GAD-7 and PHQ-9 scores from intake to end of follow-up. Independent t-tests were used to evaluate mean percentage score changes for each subgroup, with Cohen's d calculated to estimate effect sizes. We constructed univariate logistic regression models to identify predictors of improvement for depression and anxiety. RESULTS Patients in the hybrid model (n = 208) experienced greater reductions in PHQ-9 and GAD-7 scores compared to patients in primary care with 0-3 months follow-up (n = 1077), yielding effect sizes of 0.50 and 0.37, respectively. Improvements in self-efficacy predicted better outcomes for both depression and anxiety, while increases in emotional self-awareness were predictive of anxiety improvement. Total app time and therapist alliance were significant predictors of depression improvement. CONCLUSION Eight-week blended hybrid care models can offer effective treatment outcomes for depression and anxiety, even when compared to the real-world setting of primary care. These findings can inform future research aimed at elucidating the mechanisms that drive individual clinical effects.
Collapse
Affiliation(s)
- Elombe Calvert
- Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| | - Maddalena Cipriani
- Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| | - Kelly Chen
- Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| | - Alex Dhima
- Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| | - James Burns
- Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| | - John Torous
- Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA.
| |
Collapse
|
4
|
Linardon J, Fuller-Tyszkiewicz M, Firth J, Goldberg SB, Anderson C, McClure Z, Torous J. Systematic review and meta-analysis of adverse events in clinical trials of mental health apps. NPJ Digit Med 2024; 7:363. [PMID: 39695173 DOI: 10.1038/s41746-024-01388-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2024] [Accepted: 12/10/2024] [Indexed: 12/20/2024] Open
Abstract
Mental health apps are efficacious, yet they may pose risks in some. This review (CRD42024506486) examined adverse events (AEs) from mental health apps. We searched (May 2024) the Medline, PsycINFO, Web of Science, and ProQuest databases to identify clinical trials of mental health apps. The risk of bias was assessed using the Cochrane Risk of Bias tool. Only 55 of 171 identified clinical trials reported AEs. AEs were more likely to be reported in trials sampling schizophrenia and delivering apps with symptom monitoring technology. The meta-analytic deterioration rate from 13 app conditions was 6.7% (95% CI = 4.3, 10.1, I2 = 75%). Deterioration rates did not differ between app and control groups (OR = 0.79, 95% CI = 0.62-1.01, I2 = 0%). Reporting of AEs was heterogeneous, in terms of assessments used, events recorded, and detail provided. Overall, few clinical trials of mental health apps report AEs. Those that do often provide insufficient information to properly judge risks related to app use.
Collapse
Affiliation(s)
- Jake Linardon
- School of Psychology, Deakin University, Geelong, VIC, Australia.
- Center for Social and Early Emotional Development, Deakin University, Burwood, VIC, Australia.
| | - Matthew Fuller-Tyszkiewicz
- School of Psychology, Deakin University, Geelong, VIC, Australia
- Center for Social and Early Emotional Development, Deakin University, Burwood, VIC, Australia
| | - Joseph Firth
- Division of Psychology and Mental Health, University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
- Greater Manchester Mental Health NHS Foundation Trust, Manchester Academic Health Science Centre, Manchester, UK
| | - Simon B Goldberg
- Department of Counselling Psychology, University of Wisconsin - Madison, Madison, WI, USA
- Centre for Healthy Minds, University of Wisconsin - Madison, Madison, WI, USA
| | - Cleo Anderson
- School of Psychology, Deakin University, Geelong, VIC, Australia
| | - Zoe McClure
- School of Psychology, Deakin University, Geelong, VIC, Australia
| | - John Torous
- Division of Digital Psychiatry, Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
5
|
Dwyer B, Flathers M, Burns J, Mikkelson J, Perlmutter E, Chen K, Ram N, Torous J. Assessing Digital Phenotyping for App Recommendations and Sustained Engagement: Cohort Study. JMIR Form Res 2024; 8:e62725. [PMID: 39560976 PMCID: PMC11615540 DOI: 10.2196/62725] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2024] [Revised: 08/22/2024] [Accepted: 09/20/2024] [Indexed: 11/20/2024] Open
Abstract
BACKGROUND Low engagement with mental health apps continues to limit their impact. New approaches to help match patients to the right app may increase engagement by ensuring the app they are using is best suited to their mental health needs. OBJECTIVE This study aims to pilot how digital phenotyping, using data from smartphone sensors to infer symptom, behavioral, and functional outcomes, could be used to match people to mental health apps and potentially increase engagement. METHODS After 1 week of collecting digital phenotyping data with the mindLAMP app (Beth Israel Deaconess Medical Center), participants were randomly assigned to the digital phenotyping arm, receiving feedback and recommendations based on those data to select 1 of 4 predetermined mental health apps (related to mood, anxiety, sleep, and fitness), or the control arm, selecting the same apps but without any feedback or recommendations. All participants used their selected app for 4 weeks with numerous metrics of engagement recorded, including objective screentime measures, self-reported engagement measures, and Digital Working Alliance Inventory scores. RESULTS A total of 82 participants enrolled in the study; 17 (21%) dropped out of the digital phenotyping arm and 18 (22%) dropped out from the control arm. Across both groups, few participants chose or were recommended the insomnia or fitness app. The majority (39/47, 83%) used a depression or anxiety app. Engagement as measured by objective screen time and Digital Working Alliance Inventory scores were higher in the digital phenotyping arm. There was no correlation between self-reported and objective metrics of app use. Qualitative results highlighted the importance of habit formation in sustained app use. CONCLUSIONS The results suggest that digital phenotyping app recommendation is feasible and may increase engagement. This approach is generalizable to other apps beyond the 4 apps selected for use in this pilot, and practical for real-world use given that the study was conducted without any compensation or external incentives that may have biased results. Advances in digital phenotyping will likely make this method of app recommendation more personalized and thus of even greater interest.
Collapse
Affiliation(s)
- Bridget Dwyer
- Division of Digital Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, United States
| | - Matthew Flathers
- Division of Digital Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, United States
| | - James Burns
- Division of Digital Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, United States
| | - Jane Mikkelson
- Division of Digital Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, United States
| | - Elana Perlmutter
- Division of Digital Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, United States
| | - Kelly Chen
- Division of Digital Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, United States
| | - Nanik Ram
- Division of Digital Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, United States
| | - John Torous
- Division of Digital Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, United States
| |
Collapse
|
6
|
Torous J, Smith KA, Hardy A, Vinnikova A, Blease C, Milligan L, Hidalgo-Mazzei D, Lambe S, Marzano L, Uhlhaas PJ, Ostinelli EG, Anmella G, Zangani C, Aronica R, Dwyer B, Cipriani A. Digital health interventions for schizophrenia: Setting standards for mental health. Schizophr Res 2024; 267:392-395. [PMID: 38640849 DOI: 10.1016/j.schres.2024.04.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/12/2024] [Revised: 04/02/2024] [Accepted: 04/08/2024] [Indexed: 04/21/2024]
Affiliation(s)
- John Torous
- Division of Digital Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA 02446, USA.
| | - Katharine A Smith
- Oxford Precision Psychiatry Lab, NIHR Oxford Health Biomedical Research Centre, Oxford, UK; Department of Psychiatry, University of Oxford, Oxford, UK; Oxford Health NHS Foundation Trust, Warneford Hospital, Oxford, UK
| | - Amy Hardy
- Department of Psychology, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK; South London & Maudsley NHS Foundation Trust, London, UK
| | | | - Charlotte Blease
- Participatory eHealth and Health Data Research Group, Department of Women's and Children's Health, Uppsala University, Uppsala, Sweden; Digital Psychiatry, Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, USA
| | | | - Diego Hidalgo-Mazzei
- Department of Psychiatry and Psychology, Institute of Neuroscience, Hospital Clínic de Barcelona, Barcelona, Catalonia, Spain; Bipolar and Depressive Disorders Unit, Digital Innovation Group, Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Catalonia, Spain; Biomedical Research Networking Centre Consortium on Mental Health (CIBERSAM), Instituto de Salud Carlos III, Madrid, Spain; Department of Experimental Psychology, University of Oxford, UK
| | - Sinead Lambe
- Oxford Health NHS Foundation Trust, Warneford Hospital, Oxford, UK; Department of Experimental Psychology, University of Oxford, UK
| | - Lisa Marzano
- School of Science and Technology, Middlesex University, UK
| | - Peter J Uhlhaas
- Institute of Neuroscience and Psychology, University of Glasgow, Glasgow, UK; Charité - Universitätsmedizin Berlin, Freie Universität Berlin and Humboldt-Universität zu Berlin, Department of Child and Adolescent Psychiatry, Augustenburger Platz 1, 13353 Berlin, Germany
| | - Edoardo G Ostinelli
- Oxford Precision Psychiatry Lab, NIHR Oxford Health Biomedical Research Centre, Oxford, UK; Department of Psychiatry, University of Oxford, Oxford, UK; Oxford Health NHS Foundation Trust, Warneford Hospital, Oxford, UK
| | - Gerard Anmella
- Department of Psychiatry and Psychology, Institute of Neuroscience, Hospital Clínic de Barcelona, Barcelona, Catalonia, Spain; Bipolar and Depressive Disorders Unit, Digital Innovation Group, Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Catalonia, Spain; Biomedical Research Networking Centre Consortium on Mental Health (CIBERSAM), Instituto de Salud Carlos III, Madrid, Spain; Department of Medicine, School of Medicine and Health Sciences, Institute of Neurosciences (UBNeuro), University of Barcelona (UB), Barcelona, Catalonia, Spain
| | - Caroline Zangani
- Oxford Precision Psychiatry Lab, NIHR Oxford Health Biomedical Research Centre, Oxford, UK; Department of Psychiatry, University of Oxford, Oxford, UK; Oxford Health NHS Foundation Trust, Warneford Hospital, Oxford, UK
| | - Rosario Aronica
- Fondazione IRCCS Ca' Granda- Ospedale Maggiore Policlinico, Department of Neurosciences and Mental Health, Milan, Italy; Department of Pathophysiology and Transplantation, University of Milan, Milan, Italy
| | - Bridget Dwyer
- Division of Digital Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA 02446, USA
| | - Andrea Cipriani
- Oxford Precision Psychiatry Lab, NIHR Oxford Health Biomedical Research Centre, Oxford, UK; Department of Psychiatry, University of Oxford, Oxford, UK; Oxford Health NHS Foundation Trust, Warneford Hospital, Oxford, UK
| |
Collapse
|