Clinical state tracking in serious mental illness through computational analysis of speech

Using speech analysis in virtual agent conversations to differentiate PTSD patients from control participants

Diagnosing posttraumatic stress disorder (PTSD) remains challenging due to clinical overlap with conditions like depression and patients' reluctance to disclose symptoms. This study analyzed pre-existing data to investigate whether speech features extracted from semi-structured interviews with a virtual avatar could enhance traditional diagnostic approaches by utilizing automated voice analysis, which can detect nuances beyond human hearing, potentially improving sensitivity and specificity in diagnosing PTSD. Leveraging the DAIC-WoZ dataset (Gratch et al., 2014), which includes 142 dialogues (average length: 954 s) between participants and a virtual avatar, we extracted content and acoustic features from transcripts and audio recordings. Group comparisons aimed to identify differences between PTSD and non-PTSD participants, as well as distinctions between PTSD participants with and without comorbid depression. Key findings revealed that PTSD participants expressed more negative sentiment in their speech (η²=0.82, p < 0.001) and used less frequent words (η²=0.65, p < 0.001) compared to controls. To demonstrate the utility of speech features, classification models were developed to differentiate PTSD and control participants. A linear model with twenty-one speech features (balanced accuracy = 0.70) outperformed a model based solely on demographic data (balanced accuracy = 0.53). Subgroup analyses found no significant differences in speech features between PTSD participants with and without comorbid depression. These results suggest that automated speech analysis, facilitated by virtual avatars, can identify linguistic markers of PTSD, potentially aiding diagnosis and reducing barriers to disclosure. However, further research is needed to generalize these findings and explore their utility in clinical practice.

Speech and Voice Quality as Digital Biomarkers in Depression: A Systematic Review

OBJECTIVE

To review the current evidence on the use of artificial intelligence-driven speech and voice analysis as a biomarker for depression.

METHODS

PubMed, Scopus, and Cochrane databases were reviewed by two independent investigators for studies investigating the use of artificial intelligence-driven speech and voice quality outcomes as biomarkers for depression according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statements. The methodological quality and risk of bias of each included study were assessed using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool.

RESULTS

Of the 108 identified records, 12 studies met the inclusion criteria. The studies examined 16 872 participants, including patients with major depressive disorder (n = 1535), bipolar disorder (n = 111), schizophrenia spectrum disorders (n = 35), and anxiety disorders (n = 224). Control groups included a total of 1204 healthy individuals. Speech and voice quality outcomes consistently distinguished depression from controls (AUC = 0.71-0.93), with prosodic, spectral, and perturbation measures showing significant correlations with standardized depression scales. Classification accuracies ranged from 78% to 96.5%. Six studies demonstrated high risk of methodological bias, primarily in patient selection and validation techniques. Voice recording contexts varied between clinical settings and mobile technologies.

CONCLUSION

The findings of this review highlight the potential of voice biomarkers as a novel tool for depression detection and monitoring. While current evidence demonstrates promising classification accuracy, methodological heterogeneity and generalizability concerns must be addressed before widespread clinical adoption.

Language and neurological assessment in stroke: Rethinking the role of linguistic proficiency in clinical outcomes

This correspondence underscores the potential impact of language proficiency on clinical outcomes in patients with acute ischemic stroke (AIS), especially those experiencing large-vessel occlusion (LVO). Citing recent findings that non-English-speaking patients often present with more severe symptoms and poorer discharge outcomes despite receiving comparable treatment, this commentary examines how linguistic barriers may confound clinical assessments such as the NIHSS. From a psycholinguistic perspective, language functions not merely as a medium of communication but as a cognitive instrument that actively shapes the expression of symptoms. We advocate for the incorporation of linguistic competence into stroke assessment protocols to promote equity and diagnostic precision in neurology.

Associations of Vocal Features, Psychiatric Symptoms, and Cognitive Functions in Schizophrenia

Purpose

This study explored the use of advanced computational techniques in vocal analysis to improve the assessment of psychiatric symptoms and cognitive functions in schizophrenia. We hypothesized that digital signal processing techniques, such as mel spectrogram and mel-frequency cepstral coefficients (MFCC), could be used for objective evaluation of psychiatric symptoms and cognitive functions based on the analysis of alterations in the vocal characteristics.

Patients and Methods

Voice samples from 14 participants diagnosed with schizophrenia (92.9% female) were collected using a microphone array, and vocal features were extracted from the samples using mel spectrogram and MFCC techniques. Psychiatric symptoms and cognitive functions were assessed using the Positive and Negative Syndrome Scale (PANSS) and the computer-based tool Cognitrax.

Results

We found significant negative correlations between specific vocal features (mel spectrogram and MFCC) and cognitive functions, particularly working memory (β = -0.645, p = 0.023) and sustained attention (β = -0.626, p = 0.029). No direct correlations were found between vocal features and psychiatric symptoms, as measured by PANSS scores. However, the correlations between cognitive functions and PANSS total scores were significant (β = -0.604, p = 0.037), suggesting that cognitive functions may mediate the relationship between psychiatric symptoms and vocal characteristics.

Conclusion

This study underscores the potential of vocal analysis as a non-invasive tool for assessing cognitive impairment in schizophrenia. Future research should focus on expanding the sample size and including diverse populations to enhance the generalizability of these findings.

Acoustic and Natural Language Markers for Bipolar Disorder: A Pilot, mHealth Cross-Sectional Study

Background

Monitoring symptoms of bipolar disorder (BD) is a challenge faced by mental health services. Speech patterns are crucial in assessing the current experiences, emotions, and thought patterns of people with BD. Natural language processing (NLP) and acoustic signal processing may support ongoing BD assessment within a mobile health (mHealth) framework.

Objective

Using both acoustic and NLP-based features from the speech of people with BD, we built an app-based tool and tested its feasibility and performance to remotely assess the individual clinical status.

Methods

We carried out a pilot, observational study, sampling adults diagnosed with BD from the caseload of the Nord Milano Mental Health Trust (Italy) to explore the relationship between selected speech features and symptom severity and to test their potential to remotely assess mental health status. Symptom severity assessment was based on clinician ratings, using the Young Mania Rating Scale (YMRS) and Montgomery-Åsberg Depression Rating Scale (MADRS) for manic and depressive symptoms, respectively. Leveraging a digital health tool embedded in a mobile app, which records and processes speech, participants self-administered verbal performance tasks. Both NLP-based and acoustic features were extracted, testing associations with mood states and exploiting machine learning approaches based on random forest models.

Results

We included 32 subjects (mean [SD] age 49.6 [14.3] years; 50% [16/32] females) with a MADRS median (IQR) score of 13 (21) and a YMRS median (IQR) score of 5 (16). Participants freely managed the digital environment of the app, without perceiving it as intrusive and reporting an acceptable system usability level (average score 73.5, SD 19.7). Small-to-moderate correlations between speech features and symptom severity were uncovered, with sex-based differences in predictive capability. Higher latency time (ρ=0.152), increased silences (ρ=0.416), and vocal perturbations correlated with depressive symptomatology. Pressure of speech based on the mean intraword time (ρ=-0.343) and lower voice instability based on jitter-related parameters (ρ ranging from -0.19 to -0.27) were detected for manic symptoms. However, a higher contribution of NLP-based and conversational features, rather than acoustic features, was uncovered, especially for predictive models for depressive symptom severity (NLP-based: R2=0.25, mean squared error [MSE]=110.07, mean absolute error [MAE]=8.17; acoustics: R2=0.11, MSE=133.75, MAE=8.86; combined: R2=0.16; MSE=118.53, MAE=8.68).

Conclusions

Remotely collected speech patterns, including both linguistic and acoustic features, are associated with symptom severity levels and may help differentiate clinical conditions in individuals with BD during their mood state assessments. In the future, multimodal, smartphone-integrated digital ecological momentary assessments could serve as a powerful tool for clinical purposes, remotely complementing standard, in-person mental health evaluations.

Applying AI in the Context of the Association Between Device-Based Assessment of Physical Activity and Mental Health: Systematic Review

BACKGROUND

Wearable technology is used by consumers worldwide for continuous activity monitoring in daily life but more recently also for classifying or predicting mental health parameters like stress or depression levels. Previous studies identified, based on traditional approaches, that physical activity is a relevant factor in the prevention or management of mental health. However, upcoming artificial intelligence methods have not yet been fully established in the research field of physical activity and mental health.

OBJECTIVE

This systematic review aims to provide a comprehensive overview of studies that integrated passive monitoring of physical activity data measured via wearable technology in machine learning algorithms for the detection, prediction, or classification of mental health states and traits.

METHODS

We conducted a review of studies processing wearable data to gain insights into mental health parameters. Eligibility criteria were (1) the study uses wearables or smartphones to acquire physical behavior and optionally other sensor measurement data, (2) the study must use machine learning to process the acquired data, and (3) the study had to be published in a peer-reviewed English language journal. Studies were identified via a systematic search in 5 electronic databases.

RESULTS

Of 11,057 unique search results, 49 published papers between 2016 and 2023 were included. Most studies examined the connection between wearable sensor data and stress (n=15, 31%) or depression (n=14, 29%). In total, 71% (n=35) of the studies had less than 100 participants, and 47% (n=23) had less than 14 days of data recording. More than half of the studies (n=27, 55%) used step count as movement measurement, and 44% (n=21) used raw accelerometer values. The quality of the studies was assessed, scoring between 0 and 18 points in 9 categories (maximum 2 points per category). On average, studies were rated 6.47 (SD 3.1) points.

CONCLUSIONS

The use of wearable technology for the detection, prediction, or classification of mental health states and traits is promising and offers a variety of applications across different settings and target groups. However, based on the current state of literature, the application of artificial intelligence cannot realize its full potential mostly due to a lack of methodological shortcomings and data availability. Future research endeavors may focus on the following suggestions to improve the quality of new applications in this context: first, by using raw data instead of already preprocessed data. Second, by using only relevant data based on empirical evidence. In particular, crafting optimal feature sets rather than using many individual detached features and consultation with in-field professionals. Third, by validating and replicating the existing approaches (ie, applying the model to unseen data). Fourth, depending on the research aim (ie, generalization vs personalization) maximizing the sample size or the duration over which data are collected.

Linguistic style as a digital marker for depression severity: An ambulatory assessment pilot study in patients with depressive disorder undergoing sleep deprivation therapy

BACKGROUND

Digital phenotyping and monitoring tools are the most promising approaches to automatically detect upcoming depressive episodes. Especially, linguistic style has been seen as a potential behavioral marker of depression, as cross-sectional studies showed, for example, less frequent use of positive emotion words, intensified use of negative emotion words, and more self-references in patients with depression compared to healthy controls. However, longitudinal studies are sparse and therefore it remains unclear whether within-person fluctuations in depression severity are associated with individuals' linguistic style.

METHODS

To capture affective states and concomitant speech samples longitudinally, we used an ambulatory assessment approach sampling multiple times a day via smartphones in patients diagnosed with depressive disorder undergoing sleep deprivation therapy. This intervention promises a rapid change of affective symptoms within a short period of time, assuring sufficient variability in depressive symptoms. We extracted word categories from the transcribed speech samples using the Linguistic Inquiry and Word Count.

RESULTS

Our analyses revealed that more pleasant affective momentary states (lower reported depression severity, lower negative affective state, higher positive affective state, (positive) valence, energetic arousal and calmness) are mirrored in the use of less negative emotion words and more positive emotion words.

CONCLUSION

We conclude that a patient's linguistic style, especially the use of positive and negative emotion words, is associated with self-reported affective states and thus is a promising feature for speech-based automated monitoring and prediction of upcoming episodes, ultimately leading to better patient care.

Family Conflict, Perceived Criticism, and Aggression in Symptomatic Offspring of Parents With Mood Disorders: Results From a Clinical Trial of Family-Focused Therapy

Objective

In offspring of parents with bipolar or major depressive disorder, we examined the longitudinal association between parents' and youths' ratings of family conflict and criticism and youths' levels of impulsive aggression during a 6-month randomized trial of family intervention.

Method

Following a diagnostic evaluation, we offered adolescents (aged 13-19 years) and parents with mood disorders a 12-session, 18-week protocol of family-focused therapy, with random assignment to mobile applications that enabled mood tracking or encouraged practice of mood management and family communication skills, also with mood tracking. At pretreatment, 9 weeks, 18 weeks (posttreatment), and 27 weeks, parents completed measures of adolescent aggression and irritability, and parents/adolescents completed measures of dyadic conflict and perceived criticism.

Results

Parent- and youth-rated dyadic conflict scores and perceived criticism ratings were concurrently associated with youths' composite aggression scores across the 4 timepoints. In lagged association analyses, parent-rated dyadic conflict scores in 1 9-week study interval predicted youths' aggression scores in subsequent 9-week intervals (F 1,109 = 7.36, p = .008). In contrast, youths' aggression scores in 1 interval predicted youths' ratings of dyadic conflict in subsequent intervals (F 1,107 = 8.16, p = .005). Levels of family conflict, perceived criticism, and youth aggression decreased over 6 months in both mobile app conditions.

Conclusion

In offspring of parents with mood disorders, levels of family conflict, criticism, and adolescent aggression are intercorrelated over time and suggest bidirectional, mutually influential processes within families. Aggression and its precipitants within family interactions should be a focus of interventions for youths with or at risk for mood disorders.

Clinical guidance

• In adolescent offspring of parents with mood disorders, levels of family conflict, adolescents' perceptions of criticism, and aggression are intercorrelated over time.• A 12-session course of family-focused therapy may positively impact these family processes.• Mobile applications that emphasize mood tracking and family communication skills may enhance telehealth-based family interventions by encouraging skill practice between sessions.

Clinical trial registration information

Technology Enhanced Family Treatment; https://clinicaltrials.gov/; NCT03913013.

Diversity & Inclusion Statement

We worked to ensure sex and gender balance in the recruitment of human participants. We worked to ensure race, ethnic, and/or other types of diversity in the recruitment of human participants. We worked to ensure that the study questionnaires were prepared in an inclusive way. One or more of the authors of this paper self-identifies as a member of one or more historically underrepresented racial and/or ethnic groups in science. One or more of the authors of this paper self-identifies as a member of one or more historically underrepresented sexual and/or gender groups in science. The author list of this paper includes contributors from the location and/or community where the research was conducted who participated in the data collection, design, analysis, and/or interpretation of the work.

Voice Quality as Digital Biomarker in Bipolar Disorder: A Systematic Review

BACKGROUND

Voice analysis has emerged as a potential biomarker for mood state detection and monitoring in bipolar disorder (BD). The systematic review aimed to summarize the evidence for voice analysis applications in BD, examining (1) the predictive validity of voice quality outcomes for mood state detection, and (2) the correlation between voice parameters and clinical symptom scales.

METHODS

A PubMed, Scopus, and Cochrane Library search was carried out by two investigators for publications investigating voice quality in BD according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statements. Studies were assessed using the modified methodological index for non-randomized studies (MINORS).

RESULTS

Of the 400 identified publications, 16 studies met the inclusion accounting for 575 BD patients. Machine learning approaches were implemented in 87.5% of studies, with classification accuracies ranging from 70.9% to 96.9%. Manic state detection showed the strongest predictive validity [area under the curve (AUC) up to 0.89], while depression detection demonstrated moderate performance (AUC: 0.66-0.78). Individual-specific models outperformed population-level approaches (correlation coefficients: 0.78 versus 0.44). Voice quality showed significant correlations with standardized clinical scales, particularly Young Mania Rating Scale and Hamilton Depression Rating Scale (normalized root mean square errors: 1.985 and 3.945, respectively). Prosodic features were examined in 81.25% of studies, with pitch consistently elevated during manic episodes. MINORS varied from 10 to 14, with notable limitations in sample size calculations and blinding procedures.

CONCLUSIONS

Voice quality is a promising biomarker in BD, particularly for manic state detection and individualized monitoring. While controlled settings showed strong performance, naturalistic applications yielded more modest results. Future research should focus on standardizing protocols across different environments and conducting large-scale longitudinal studies with robust methodological controls.

Machine Learning for Mental Health: Applications, Challenges, and the Clinician's Role

PURPOSE OF REVIEW

This review aims to evaluate the current psychiatric applications and limitations of machine learning (ML), defined as techniques used to train algorithms to improve performance at a task based on data. The review emphasizes the clinician's role in ensuring equitable and effective patient care and seeks to inform mental health providers about the importance of clinician involvement in these technologies.

RECENT FINDINGS

ML in psychiatry has advanced through electronic health record integration, disease phenotyping, and remote monitoring through mobile applications. However, these applications face challenges related to health equity, privacy, translation to practice, and validation. Clinicians play crucial roles in ensuring data quality, mitigating biases, promoting algorithm transparency, guiding clinical implementation, and advocating for ethical and patient-centered use of ML tools. Clinicians are essential in addressing the challenges of ML, ensuring its ethical application, and promoting equitable care, thus improving the effectiveness of ML in practice.

The voice of depression: speech features as biomarkers for major depressive disorder

BACKGROUND

Psychiatry faces a challenge due to the lack of objective biomarkers, as current assessments are based on subjective evaluations. Automated speech analysis shows promise in detecting symptom severity in depressed patients. This project aimed to identify discriminating speech features between patients with major depressive disorder (MDD) and healthy controls (HCs) by examining associations with symptom severity measures.

METHODS

Forty-four MDD patients from the Psychiatry Department, University Hospital Aachen, Germany and fifty-two HCs were recruited. Participants described positive and negative life events, which were recorded for analysis. The Beck Depression Inventory (BDI-II) and the Hamilton Rating Scale for Depression gauged depression severity. Transcribed audio recordings underwent feature extraction, including acoustics, speech rate, and content. Machine learning models including speech features and neuropsychological assessments, were used to differentiate between the MDD patients and HCs.

RESULTS

Acoustic variables such as pitch and loudness differed significantly between the MDD patients and HCs (effect sizes 𝜼2 between 0.183 and 0.3, p < 0.001). Furthermore, variables pertaining to temporality, lexical richness, and speech sentiment displayed moderate to high effect sizes (𝜼2 between 0.062 and 0.143, p < 0.02). A support vector machine (SVM) model based on 10 acoustic features showed a high performance (AUC = 0.93) in differentiating between HCs and patients with MDD, comparable to an SVM based on the BDI-II (AUC = 0.99, p = 0.01).

CONCLUSIONS

This study identified robust speech features associated with MDD. A machine learning model based on speech features yielded similar results to an established pen-and-paper depression assessment. In the future, these findings may shape voice-based biomarkers, enhancing clinical diagnosis and MDD monitoring.

Behavioral skill practice as a predictor of mood and family functioning in adolescents with bipolar and depressive mood disorders: Results of a 6-month randomized trial of family-focused therapy

OBJECTIVE

Behavioral interventions require considerable practice of treatment skills in between therapy sessions. The effects of these treatments may vary with the degree to which patients are able to implement these practices. In offspring of parents with bipolar and major depressive disorders, we examined whether youth who frequently practiced communication and problem-solving skills between family-focused therapy (FFT) sessions had less severe mood symptoms and better psychosocial functioning over 6 months than youth who practiced less frequently.

METHODS

We randomly assigned offspring (ages 12-19) of parents with mood disorders to 12 sessions of FFT plus a mobile app that encouraged the practice of communication, problem-solving and mood management skills (FFT-MyCoachConnect [MCC] condition) or 12 sessions of FFT with an app that only allowed for tracking of symptoms and stress (FFT-Track condition). Independent evaluators assessed youths' mood and psychosocial functioning at 9-week intervals over 27 weeks. Clinicians rated participants' between-session skill practice at each FFT session.

RESULTS

FFT-MCC was associated with more frequent skill practice than FFT-Track over 18 weeks of treatment. Skill practice was associated with reductions in youths' mood instability and perceptions of family conflict over 27 weeks in both app conditions. Skill practice mediated the effects of app condition on youths' mood instability and family functioning.

CONCLUSIONS

Mobile applications as adjuncts to family therapy for youth with mood disorders can help increase skill practice. These findings provide preliminary causal evidence for behavioral skill practice improving mood symptoms and family functioning among youth with mood disorders.

Mental health issues of higher education students reflected in academic research: A text mining study

Objective: This study investigated mental health issues among higher education students to identify key concepts, topics, and trends over three periods of time: Period 1 (2000-2009), Period 2 (2010-2019), and Period 3 (2020-May 2024). Methods: The study collected 11,732 bibliographic records from Scopus and Web of Science, published between January 2000 and May 2024, and employed textual analysis methods, including keyword co-occurrence analysis, cluster analysis, and topic modeling. Results: In Period 1, general topics related to mental health disorders and treatments were identified. Period 2 showed prominence of well-being and help-seeking, as well as the emergence of digital mental health. Period 3 emphasized the impact of COVID-19 and increased technology usage. Conclusions: Based on the findings, we discussed the significance of the study and practical implications for clinicians and policymakers, as well as methodological implications for researchers. Additionally, the limitations of the study and future research were addressed.

Natural Language Processing and Schizophrenia: A Scoping Review of Uses and Challenges

(1) Background: Approximately 1% of the global population is affected by schizophrenia, a disorder marked by cognitive deficits, delusions, hallucinations, and language issues. It is associated with genetic, neurological, and environmental factors, and linked to dopaminergic hyperactivity and neurotransmitter imbalances. Recent research reveals that patients exhibit significant language impairments, such as reduced verbal output and fluency. Advances in machine learning and natural language processing show potential for early diagnosis and personalized treatments, but additional research is required for the practical application and interpretation of such technology. The objective of this study is to explore the applications of natural language processing in patients diagnosed with schizophrenia. (2) Methods: A scoping review was conducted across multiple electronic databases, including Medline, PubMed, Embase, and PsycInfo. The search strategy utilized a combination of text words and subject headings, focusing on schizophrenia and natural language processing. Systematically extracted information included authors, population, primary uses of the natural language processing algorithms, main outcomes, and limitations. The quality of the identified studies was assessed. (3) Results: A total of 516 eligible articles were identified, from which 478 studies were excluded based on the first analysis of titles and abstracts. Of the remaining 38 studies, 18 were selected as part of this scoping review. The following six main uses of natural language processing were identified: diagnostic and predictive modeling, followed by specific linguistic phenomena, speech and communication analysis, social media and online content analysis, clinical and cognitive assessment, and linguistic feature analysis. (4) Conclusions: This review highlights the main uses of natural language processing in the field of schizophrenia and the need for more studies to validate the effectiveness of natural language processing in diagnosing and treating schizophrenia.

Evaluating Atypical Gaze Patterns through Vision Models: The Case of Cortical Visual Impairment

A wide range of neurological and cognitive disorders exhibit distinct behavioral markers aside from their clinical manifestations. Cortical Visual Impairment (CVI) is a prime example of such conditions, resulting from damage to visual pathways in the brain, and adversely impacting low- and high-level visual function. The characteristics impacted by CVI are primarily described qualitatively, challenging the establishment of an objective, evidence-based measure of CVI severity. To study those characteristics, we propose to create visual saliency maps by adequately prompting deep vision models with attributes of clinical interest. After extracting saliency maps for a curated set of stimuli, we evaluate fixation traces on those from children with CVI through eye tracking technology. Our experiments reveal significant gaze markers that verify clinical knowledge and yield nuanced discriminability when compared to those of age-matched control subjects. Using deep learning to unveil atypical visual saliency is an important step toward establishing an eye-tracking signature for severe neurodevelopmental disorders, like CVI.

Validating the efficacy and value proposition of mental fitness vocal biomarkers in a psychiatric population: prospective cohort study

Background

The utility of vocal biomarkers for mental health assessment has gained increasing attention. This study aims to further this line of research by introducing a novel vocal scoring system designed to provide mental fitness tracking insights to users in real-world settings.

Methods

A prospective cohort study with 104 outpatient psychiatric participants was conducted to validate the "Mental Fitness Vocal Biomarker" (MFVB) score. The MFVB score was derived from eight vocal features, selected based on literature review. Participants' mental health symptom severity was assessed using the M3 Checklist, which serves as a transdiagnostic tool for measuring depression, anxiety, post-traumatic stress disorder, and bipolar symptoms.

Results

The MFVB demonstrated an ability to stratify individuals by their risk of elevated mental health symptom severity. Continuous observation enhanced the MFVB's efficacy, with risk ratios improving from 1.53 (1.09-2.14, p=0.0138) for single 30-second voice samples to 2.00 (1.21-3.30, p=0.0068) for data aggregated over two weeks. A higher risk ratio of 8.50 (2.31-31.25, p=0.0013) was observed in participants who used the MFVB 5-6 times per week, underscoring the utility of frequent and continuous observation. Participant feedback confirmed the user-friendliness of the application and its perceived benefits.

Conclusions

The MFVB is a promising tool for objective mental health tracking in real-world conditions, with potential to be a cost-effective, scalable, and privacy-preserving adjunct to traditional psychiatric assessments. User feedback suggests that vocal biomarkers can offer personalized insights and support clinical therapy and other beneficial activities that are associated with improved mental health risks and outcomes.

Natural language processing for mental health interventions: a systematic review and research framework

Neuropsychiatric disorders pose a high societal cost, but their treatment is hindered by lack of objective outcomes and fidelity metrics. AI technologies and specifically Natural Language Processing (NLP) have emerged as tools to study mental health interventions (MHI) at the level of their constituent conversations. However, NLP's potential to address clinical and research challenges remains unclear. We therefore conducted a pre-registered systematic review of NLP-MHI studies using PRISMA guidelines (osf.io/s52jh) to evaluate their models, clinical applications, and to identify biases and gaps. Candidate studies (n = 19,756), including peer-reviewed AI conference manuscripts, were collected up to January 2023 through PubMed, PsycINFO, Scopus, Google Scholar, and ArXiv. A total of 102 articles were included to investigate their computational characteristics (NLP algorithms, audio features, machine learning pipelines, outcome metrics), clinical characteristics (clinical ground truths, study samples, clinical focus), and limitations. Results indicate a rapid growth of NLP MHI studies since 2019, characterized by increased sample sizes and use of large language models. Digital health platforms were the largest providers of MHI data. Ground truth for supervised learning models was based on clinician ratings (n = 31), patient self-report (n = 29) and annotations by raters (n = 26). Text-based features contributed more to model accuracy than audio markers. Patients' clinical presentation (n = 34), response to intervention (n = 11), intervention monitoring (n = 20), providers' characteristics (n = 12), relational dynamics (n = 14), and data preparation (n = 4) were commonly investigated clinical categories. Limitations of reviewed studies included lack of linguistic diversity, limited reproducibility, and population bias. A research framework is developed and validated (NLPxMHI) to assist computational and clinical researchers in addressing the remaining gaps in applying NLP to MHI, with the goal of improving clinical utility, data access, and fairness.

Speech characteristics yield important clues about motor function: Speech variability in individuals at clinical high-risk for psychosis

BACKGROUND AND HYPOTHESIS

Motor abnormalities are predictive of psychosis onset in individuals at clinical high risk (CHR) for psychosis and are tied to its progression. We hypothesize that these motor abnormalities also disrupt their speech production (a highly complex motor behavior) and predict CHR individuals will produce more variable speech than healthy controls, and that this variability will relate to symptom severity, motor measures, and psychosis-risk calculator risk scores.

STUDY DESIGN

We measure variability in speech production (variability in consonants, vowels, speech rate, and pausing/timing) in N = 58 CHR participants and N = 67 healthy controls. Three different tasks are used to elicit speech: diadochokinetic speech (rapidly-repeated syllables e.g., papapa…, pataka…), read speech, and spontaneously-generated speech.

STUDY RESULTS

Individuals in the CHR group produced more variable consonants and exhibited greater speech rate variability than healthy controls in two of the three speech tasks (diadochokinetic and read speech). While there were no significant correlations between speech measures and remotely-obtained motor measures, symptom severity, or conversion risk scores, these comparisons may be under-powered (in part due to challenges of remote data collection during the COVID-19 pandemic).

CONCLUSION

This study provides a thorough and theory-driven first look at how speech production is affected in this at-risk population and speaks to the promise and challenges facing this approach moving forward.

Identifying Medications Underlying Communication Atypicalities in Psychotic and Affective Disorders: A Pharmacovigilance Study Within the FDA Adverse Event Reporting System

PURPOSE

Communication atypicalities are considered promising markers of a broad range of clinical conditions. However, little is known about the mechanisms and confounders underlying them. Medications might have a crucial, relatively unknown role both as potential confounders and offering an insight on the mechanisms at work. The integration of regulatory documents with disproportionality analyses provides a more comprehensive picture to account for in future investigations of communication-related markers. The aim of this study was to identify a list of drugs potentially associated with communicative atypicalities within psychotic and affective disorders.

METHOD

We developed a query using the Medical Dictionary for Regulatory Activities to search for communicative atypicalities within the FDA Adverse Event Reporting System (updated June 2021). A Bonferroni-corrected disproportionality analysis (reporting odds ratio) was separately performed on spontaneous reports involving psychotic, affective, and non-neuropsychiatric disorders, to account for the confounding role of different underlying conditions. Drug-adverse event associations not already reported in the Side Effect Resource database of labeled adverse drug reactions (unexpected) were subjected to further robustness analyses to account for expected biases.

RESULTS

A list of 291 expected and 91 unexpected potential confounding medications was identified, including drugs that may irritate (inhalants) or desiccate (anticholinergics) the larynx, impair speech motor control (antipsychotics), or induce nodules (acitretin) or necrosis (vascular endothelial growth factor receptor inhibitors) on vocal cords; sedatives and stimulants; neurotoxic agents (anti-infectives); and agents acting on neurotransmitter pathways (dopamine agonists).

CONCLUSIONS

We provide a list of medications to account for in future studies of communication-related markers in affective and psychotic disorders. The current test case illustrates rigorous procedures for digital phenotyping, and the methodological tools implemented for large-scale disproportionality analyses can be considered a road map for investigations of communication-related markers in other clinical populations.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.23721345.

Wearable and Mobile Technologies for the Evaluation and Treatment of Obsessive-Compulsive Disorder: Scoping Review

BACKGROUND

Smartphones and wearable biosensors can continuously and passively measure aspects of behavior and physiology while also collecting data that require user input. These devices can potentially be used to monitor symptom burden; estimate diagnosis and risk for relapse; predict treatment response; and deliver digital interventions in patients with obsessive-compulsive disorder (OCD), a prevalent and disabling psychiatric condition that often follows a chronic and fluctuating course and may uniquely benefit from these technologies.

OBJECTIVE

Given the speed at which mobile and wearable technologies are being developed and implemented in clinical settings, a continual reappraisal of this field is needed. In this scoping review, we map the literature on the use of wearable devices and smartphone-based devices or apps in the assessment, monitoring, or treatment of OCD.

METHODS

In July 2022 and April 2023, we conducted an initial search and an updated search, respectively, of multiple databases, including PubMed, Embase, APA PsycINFO, and Web of Science, with no restriction on publication period, using the following search strategy: ("OCD" OR "obsessive" OR "obsessive-compulsive") AND ("smartphone" OR "phone" OR "wearable" OR "sensing" OR "biofeedback" OR "neurofeedback" OR "neuro feedback" OR "digital" OR "phenotyping" OR "mobile" OR "heart rate variability" OR "actigraphy" OR "actimetry" OR "biosignals" OR "biomarker" OR "signals" OR "mobile health").

RESULTS

We analyzed 2748 articles, reviewed the full text of 77 articles, and extracted data from the 25 articles included in this review. We divided our review into the following three parts: studies without digital or mobile intervention and with passive data collection, studies without digital or mobile intervention and with active or mixed data collection, and studies with a digital or mobile intervention.

CONCLUSIONS

Use of mobile and wearable technologies for OCD has developed primarily in the past 15 years, with an increasing pace of related publications. Passive measures from actigraphy generally match subjective reports. Ecological momentary assessment is well tolerated for the naturalistic assessment of symptoms, may capture novel OCD symptoms, and may also document lower symptom burden than retrospective recall. Digital or mobile treatments are diverse; however, they generally provide some improvement in OCD symptom burden. Finally, ongoing work is needed for a safe and trusted uptake of technology by patients and providers.

Reading and lexical-semantic retrieval tasks outperforms single task speech analysis in the screening of mild cognitive impairment and Alzheimer's disease

Age-related cognitive impairment have increased dramatically in recent years, which has risen the interes in developing screening tools for mild cognitive impairment and Alzheimer's disease. Speech analysis allows to exploit the behavioral consequences of cognitive deficits on the patient's vocal performance so that it is possible to identify pathologies affecting speech production such as dementia. Previous studies have further shown that the speech task used determines how the speech parameters are altered. We aim to combine the impairments in several speech production tasks in order to improve the accuracy of screening through speech analysis. The sample consists of 72 participants divided into three equal groups of healthy older adults, people with mild cognitive impairment, or Alzheimer's disease, matched by age and education. A complete neuropsychological assessment and two voice recordings were performed. The tasks required the participants to read a text, and complete a sentence with semantic information. A stepwise linear discriminant analysis was performed to select speech parameters with discriminative power. The discriminative functions obtained an accuracy of 83.3% in simultaneous classifications of several levels of cognitive impairment. It would therefore be a promising screening tool for dementia.

Acoustic Analysis of Speech for Screening for Suicide Risk: Machine Learning Classifiers for Between- and Within-Person Evaluation of Suicidality

BACKGROUND

Assessing a patient's suicide risk is challenging for health professionals because it depends on voluntary disclosure by the patient and often has limited resources. The application of novel machine learning approaches to determine suicide risk has clinical utility.

OBJECTIVE

This study aimed to investigate cross-sectional and longitudinal approaches to assess suicidality based on acoustic voice features of psychiatric patients using artificial intelligence.

METHODS

We collected 348 voice recordings during clinical interviews of 104 patients diagnosed with mood disorders at baseline and 2, 4, 8, and 12 months after recruitment. Suicidality was assessed using the Beck Scale for Suicidal Ideation and suicidal behavior using the Columbia Suicide Severity Rating Scale. The acoustic features of the voice, including temporal, formal, and spectral features, were extracted from the recordings. A between-person classification model that examines the vocal characteristics of individuals cross sectionally to detect individuals at high risk for suicide and a within-person classification model that detects considerable worsening of suicidality based on changes in acoustic features within an individual were developed and compared. Internal validation was performed using 10-fold cross validation of audio data from baseline to 2-month and external validation was performed using data from 2 to 4 months.

RESULTS

A combined set of 12 acoustic features and 3 demographic variables (age, sex, and past suicide attempts) were included in the single-layer artificial neural network for the between-person classification model. Furthermore, 13 acoustic features were included in the extreme gradient boosting machine learning algorithm for the within-person model. The between-person classifier was able to detect high suicidality with 69% accuracy (sensitivity 74%, specificity 62%, area under the receiver operating characteristic curve 0.62), whereas the within-person model was able to predict worsening suicidality over 2 months with 79% accuracy (sensitivity 68%, specificity 84%, area under receiver operating characteristic curve 0.67). The second model showed 62% accuracy in predicting increased suicidality in external sets.

CONCLUSIONS

Within-person analysis using changes in acoustic features within an individual is a promising approach to detect increased suicidality. Automated analysis of voice can be used to support the real-time assessment of suicide risk in primary care or telemedicine.

Voice Patterns as Markers of Schizophrenia: Building a Cumulative Generalizable Approach Via a Cross-Linguistic and Meta-analysis Based Investigation

BACKGROUND AND HYPOTHESIS

Voice atypicalities are potential markers of clinical features of schizophrenia (eg, negative symptoms). A recent meta-analysis identified an acoustic profile associated with schizophrenia (reduced pitch variability and increased pauses), but also highlighted shortcomings in the field: small sample sizes, little attention to the heterogeneity of the disorder, and to generalizing findings to diverse samples and languages.

STUDY DESIGN

We provide a critical cumulative approach to vocal atypicalities in schizophrenia, where we conceptually and statistically build on previous studies. We aim at identifying a cross-linguistically reliable acoustic profile of schizophrenia and assessing sources of heterogeneity (symptomatology, pharmacotherapy, clinical and social characteristics). We relied on previous meta-analysis to build and analyze a large cross-linguistic dataset of audio recordings of 231 patients with schizophrenia and 238 matched controls (>4000 recordings in Danish, German, Mandarin and Japanese). We used multilevel Bayesian modeling, contrasting meta-analytically informed and skeptical inferences.

STUDY RESULTS

We found only a minimal generalizable acoustic profile of schizophrenia (reduced pitch variability), while duration atypicalities replicated only in some languages. We identified reliable associations between acoustic profile and individual differences in clinical ratings of negative symptoms, medication, age and gender. However, these associations vary across languages.

CONCLUSIONS

The findings indicate that a strong cross-linguistically reliable acoustic profile of schizophrenia is unlikely. Rather, if we are to devise effective clinical applications able to target different ranges of patients, we need first to establish larger and more diverse cross-linguistic datasets, focus on individual differences, and build self-critical cumulative approaches.

Acoustic speech markers for schizophrenia-spectrum disorders: a diagnostic and symptom-recognition tool

BACKGROUND

Clinicians routinely use impressions of speech as an element of mental status examination. In schizophrenia-spectrum disorders, descriptions of speech are used to assess the severity of psychotic symptoms. In the current study, we assessed the diagnostic value of acoustic speech parameters in schizophrenia-spectrum disorders, as well as its value in recognizing positive and negative symptoms.

METHODS

Speech was obtained from 142 patients with a schizophrenia-spectrum disorder and 142 matched controls during a semi-structured interview on neutral topics. Patients were categorized as having predominantly positive or negative symptoms using the Positive and Negative Syndrome Scale (PANSS). Acoustic parameters were extracted with OpenSMILE, employing the extended Geneva Acoustic Minimalistic Parameter Set, which includes standardized analyses of pitch (F0), speech quality and pauses. Speech parameters were fed into a random forest algorithm with leave-ten-out cross-validation to assess their value for a schizophrenia-spectrum diagnosis, and PANSS subtype recognition.

RESULTS

The machine-learning speech classifier attained an accuracy of 86.2% in classifying patients with a schizophrenia-spectrum disorder and controls on speech parameters alone. Patients with predominantly positive v. negative symptoms could be classified with an accuracy of 74.2%.

CONCLUSIONS

Our results show that automatically extracted speech parameters can be used to accurately classify patients with a schizophrenia-spectrum disorder and healthy controls, as well as differentiate between patients with predominantly positive v. negatives symptoms. Thus, the field of speech technology has provided a standardized, powerful tool that has high potential for clinical applications in diagnosis and differentiation, given its ease of comparison and replication across samples.

Word usage in spontaneous speech as a predictor of depressive symptoms among youth at high risk for mood disorders

BACKGROUND

We examined whether digital phenotyping of spontaneous speech, such as the use of specific word categories during speech samples, was associated with depressive symptoms in youth who were at familial and clinical risk for mood disorders.

METHODS

Participants (ages 13-19) had active mood symptoms, mood instability, and at least one parent with bipolar or major depressive disorder. During a randomized trial of family-focused therapy, participants were instructed to make weekly calls to a central voice server and leave speech samples in response to automated prompts. We coded youths' speech samples with the Linguistic Inquiry and Word Count system and used machine learning to identify the combination of speech features that were most closely associated with the course of depressive symptoms over 18 weeks.

RESULTS

A total of 253 speech samples were collected from 44 adolescents (mean age = 15.8 years; SD = 1.6) over 18 weeks. Speech containing affective processes, social processes, drives toward risk or reward, nonfluencies, and time orientation words were correlated with depressive symptoms at concurrent time periods (ps < 0.01). Machine learning analyses revealed that affective processes, nonfluencies, drives and risk words combined to most strongly predict changes in depressive symptoms over 18 weeks of treatment.

LIMITATIONS

Study results were limited by the small sample and the exclusion of paralinguistic or contextual variables in analyzing speech samples.

CONCLUSIONS

In youth at high risk for mood disorders, knowledge of speech patterns may inform prognoses during outpatient psychosocial treatment.

Telephone-Delivered Interventions for Suicide Prevention in Schizophrenia and Related Disorders: A Systematic Review

BACKGROUND

Suicide is a health problem among patients diagnosed with schizophrenia. Telehealth technology has become an emerging intervention that may offer opportunities to reach this at-risk group. However, to consider the implementation of telehealth systems in the prevention of suicidal behaviors in patients diagnosed with schizophrenia, a review of the evidence is required. The present aim was to explore the effectiveness of telephone-based suicide prevention programs among patients with schizophrenia and related disorders.

METHODS

A bibliographic search was carried out in the PubMed, PsycInfo, Scopus and Web of Science electronic databases following PRISMA guidelines. Two reviewers performed the selection, data extraction and methodological quality assessment. A total of 352 articles were retrieved, of which five studies met the eligibility criteria.

RESULTS

Globally, an adherence was observed ranging from 78 to 100%. Three studies reported a reduction in suicidal ideation and two studies showed a reduction in the risk of relapse observed in the intervention group compared to a control group.

CONCLUSIONS

In accordance with the limited data available, the use of a telephone contact approach appears to be feasible and effective in schizophrenia patients with suicidal behaviors. The preliminary evidence also suggests that this system appears to reduce suicidal ideation. Further research is required to design evidence-based future interventions and to determine whether this approach can improve patient outcomes.

Digital Phenotyping for Differential Diagnosis of Major Depressive Episode: Narrative Review

BACKGROUND

Major depressive episode (MDE) is a common clinical syndrome. It can be found in different pathologies such as major depressive disorder (MDD), bipolar disorder (BD), posttraumatic stress disorder (PTSD), or even occur in the context of psychological trauma. However, only 1 syndrome is described in international classifications (Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition [DSM-5]/International Classification of Diseases 11th Revision [ICD-11]), which do not take into account the underlying pathology at the origin of the MDE. Clinical interviews are currently the best source of information to obtain the etiological diagnosis of MDE. Nevertheless, it does not allow an early diagnosis and there are no objective measures of extracted clinical information. To remedy this, the use of digital tools and their correlation with clinical symptomatology could be useful.

OBJECTIVE

We aimed to review the current application of digital tools for MDE diagnosis while highlighting shortcomings for further research. In addition, our work was focused on digital devices easy to use during clinical interview and mental health issues where depression is common.

METHODS

We conducted a narrative review of the use of digital tools during clinical interviews for MDE by searching papers published in PubMed/MEDLINE, Web of Science, and Google Scholar databases since February 2010. The search was conducted from June to September 2021. Potentially relevant papers were then compared against a checklist for relevance and reviewed independently for inclusion, with focus on 4 allocated topics of (1) automated voice analysis, behavior analysis by (2) video and physiological measures, (3) heart rate variability (HRV), and (4) electrodermal activity (EDA). For this purpose, we were interested in 4 frequently found clinical conditions in which MDE can occur: (1) MDD, (2) BD, (3) PTSD, and (4) psychological trauma.

RESULTS

A total of 74 relevant papers on the subject were qualitatively analyzed and the information was synthesized. Thus, a digital phenotype of MDE seems to emerge consisting of modifications in speech features (namely, temporal, prosodic, spectral, source, and formants) and in speech content, modifications in nonverbal behavior (head, hand, body and eyes movement, facial expressivity, and gaze), and a decrease in physiological measurements (HRV and EDA). We not only found similarities but also differences when MDE occurs in MDD, BD, PTSD, or psychological trauma. However, comparative studies were rare in BD or PTSD conditions, which does not allow us to identify clear and distinct digital phenotypes.

CONCLUSIONS

Our search identified markers from several modalities that hold promise for helping with a more objective diagnosis of MDE. To validate their potential, further longitudinal and prospective studies are needed.

A measurement method for mental health based on dynamic multimodal feature recognition

INTRODUCTION

The number of college students with mental problems has increased significantly, particularly during COVID-19. However, the clinical features of early-stage psychological problems are subclinical, so the optimal intervention treatment period can easily be missed. Artificial intelligence technology can efficiently assist in assessing mental health problems by mining the deep correlation of multi-dimensional data of patients, providing ideas for solving the screening of normal psychological problems in large-scale college students. Therefore, we propose a mental health assessment method that integrates traditional scales and multimodal intelligent recognition technology to support the large-scale and normalized screening of mental health problems in colleges and universities.

METHODS

Firstly, utilize the psychological assessment scales based on human-computer interaction to conduct health questionnaires based on traditional methods. Secondly, integrate machine learning technology to identify the state of college students and assess the severity of psychological problems. Finally, the experiments showed that the proposed multimodal intelligent recognition method has high accuracy and can better proofread normal scale results. This study recruited 1,500 students for this mental health assessment.

RESULTS

The results showed that the incidence of moderate or higher stress, anxiety, and depression was 36.3, 48.1, and 23.0%, which is consistent with the results of our multiple targeted tests.

CONCLUSION

Therefore, the interactive multimodality emotion recognition method proposed provides an effective way for large-scale mental health screening, monitoring, and intervening in college students' mental health problems.

Automatic rating of therapist facilitative interpersonal skills in text: A natural language processing application

Background

While message-based therapy has been shown to be effective in treating a range of mood disorders, it is critical to ensure that providers are meeting a consistently high standard of care over this medium. One recently developed measure of messaging quality–The Facilitative Interpersonal Skills Task for Text (FIS-T)–provides estimates of therapists’ demonstrated ability to convey psychotherapy's common factors (e.g., hopefulness, warmth, persuasiveness) over text. However, the FIS-T's scoring procedure relies on trained human coders to manually code responses, thereby rendering the FIS-T an unscalable quality control tool for large messaging therapy platforms.

Objective

In the present study, researchers developed two algorithms to automatically score therapist performance on the FIS-T task.

Methods

The FIS-T was administered to 978 messaging therapists, whose responses were then manually scored by a trained team of raters. Two machine learning algorithms were then trained on task-taker messages and coder scores: a support vector regressor (SVR) and a transformer-based neural network (DistilBERT).

Results

The DistilBERT model had superior performance on the prediction task while providing a distribution of ratings that was more closely aligned with those of human raters, versus SVR. Specifically, the DistilBERT model was able to explain 58.8% of the variance (R2 = 0.588) in human-derived ratings and realized a prediction mean absolute error of 0.134 on a 1–5 scale.

Conclusions

Algorithms can be effectively used to ensure that digital providers meet a consistently high standard of interactions in the course of messaging therapy. Natural language processing can be applied to develop new quality assurance systems in message-based digital psychotherapy.

Family-focused therapy for individuals at high clinical risk for psychosis: A confirmatory efficacy trial

AIMS

Young people with attenuated psychotic symptoms (APS), brief intermittent psychosis, and/or genetic risk and functional deterioration are at high risk for developing psychotic disorders. In a prior trial, family-focused therapy for clinical high risk youth (FFT-CHR) was more effective than brief psychoeducation in reducing APS severity over 6 months. This 7-site trial will compare the efficacy of FFT-CHR to a psychoeducational and supportive intervention (enhanced care) on APS and social functioning in CHR individuals over 18 months.

METHODS

Participants (N = 220, ages 13-25 years) with a CHR syndrome will be randomly assigned to FFT-CHR (18 1-h sessions of family psychoeducation and communication/problem-solving skills training) or enhanced care (3 1-h family psychoeducational sessions followed by 5 individual support sessions), both given over 6 months. Participants will rate their weekly progress during treatment using a mobile-enhanced online platform. Family communication will be assessed in a laboratory interactional task at baseline and post-treatment. Independent evaluators will assess APS (primary outcome) and psychosocial functioning (secondary outcome) every 6 months over 18 months.

RESULTS

We hypothesize that, compared to enhanced care, FFT-CHR will be associated with greater improvements in APS and psychosocial functioning over 18 months. Secondarily, improvements in family communication over 6 months will mediate the relationship between treatment condition and primary and secondary outcomes over 18 months. The effects of FFT-CHR are predicted to be greater in individuals with higher baseline risk for psychosis conversion.

CONCLUSIONS

Results of the trial will inform treatment guidelines for individuals at high risk for psychosis.

Monitoring the effects of therapeutic interventions in depression through self-assessments

The treatment of major psychiatric disorders is an arduous and thorny path for the patients concerned, characterized by polypharmacy, massive adverse side effects, modest prospects of success, and constantly declining response rates. The more important is the early detection of psychiatric disorders prior to the development of clinically relevant symptoms, so that people can benefit from early interventions. A well-proven approach to monitoring mental health relies on voice analysis. This method has been successfully used with psychiatric patients to 'objectively' document the progress of improvement or the onset of relapse. The studies with psychiatric patients over 2-4 weeks demonstrated that daily voice assessments have a notable therapeutic effect in themselves. Therefore, daily voice assessments appear to be a lowthreshold form of therapeutic means that may be realized through self-assessments. To evaluate performance and reliability of this approach, we have carried out a longitudinal study on 82 university students in 3 different countries with daily assessments over 2 weeks. The sample included 41 males (mean age 24.2±3.83 years) and 41 females (mean age 21.6±2.05 years). Unlike other research in the field, this study was not concerned with the classification of individuals in terms of diagnostic categories. The focus lay on the monitoring aspect and the extent to which the effects of therapeutic interventions or of behavioural changes are visible in the results of self-assessment voice analyses. The test persons showed an over-proportionally good adherence to the daily voice analysis scheme. The accumulated data were of generally high quality: sufficiently high signal levels, a very limited number of movement artifacts, and little to no interfering background noise. The method was sufficiently sensitive to detect: i) habituation effects when test persons became used to the daily procedure; and ii) short-term fluctuations that exceeded prespecified thresholds and reached significance. Results are directly interpretable and provide information about what is going well, what is going less well, and where there is a need for action. The proposed self-assessment approach was found to be well-suited to serve as a health-monitoring tool for subjects with an elevated vulnerability to psychiatric disorders or to stress-induced mental health problems. Daily voice assessments are in fact a low-threshold form of therapeutic means that can be realized through selfassessments, that requires only little effort, can be carried out in the test person's own home, and has the potential to strengthen resilience and to induce positive behavioural changes.

Voice analyses using smartphone-based data in patients with bipolar disorder, unaffected relatives and healthy control individuals, and during different affective states

BACKGROUND

Voice features have been suggested as objective markers of bipolar disorder (BD).

AIMS

To investigate whether voice features from naturalistic phone calls could discriminate between (1) BD, unaffected first-degree relatives (UR) and healthy control individuals (HC); (2) affective states within BD.

METHODS

Voice features were collected daily during naturalistic phone calls for up to 972 days. A total of 121 patients with BD, 21 UR and 38 HC were included. A total of 107.033 voice data entries were collected [BD (n  = 78.733), UR (n  = 8004), and HC (n  =  20.296)]. Daily, patients evaluated symptoms using a smartphone-based system. Affective states were defined according to these evaluations. Data were analyzed using random forest machine learning algorithms.

RESULTS

Compared to HC, BD was classified with a sensitivity of 0.79 (SD 0.11)/AUC  = 0.76 (SD 0.11) and UR with a sensitivity of 0.53 (SD 0.21)/AUC of 0.72 (SD 0.12). Within BD, compared to euthymia, mania was classified with a specificity of 0.75 (SD 0.16)/AUC  =  0.66 (SD 0.11). Compared to euthymia, depression was classified with a specificity of 0.70 (SD 0.16)/AUC  =  0.66 (SD 0.12). In all models the user dependent models outperformed the user independent models. Models combining increased mood, increased activity and insomnia compared to periods without performed best with a specificity of 0.78 (SD 0.16)/AUC  =  0.67 (SD 0.11).

CONCLUSIONS

Voice features from naturalistic phone calls may represent a supplementary objective marker discriminating BD from HC and a state marker within BD.

Measuring neuropsychiatric symptoms in patients with early cognitive decline using speech analysis

BACKGROUND

Certain neuropsychiatric symptoms (NPS), namely apathy, depression, and anxiety demonstrated great value in predicting dementia progression, representing eventually an opportunity window for timely diagnosis and treatment. However, sensitive and objective markers of these symptoms are still missing. Therefore, the present study aims to investigate the association between automatically extracted speech features and NPS in patients with mild neurocognitive disorders.

METHODS

Speech of 141 patients aged 65 or older with neurocognitive disorder was recorded while performing two short narrative speech tasks. NPS were assessed by the neuropsychiatric inventory. Paralinguistic markers relating to prosodic, formant, source, and temporal qualities of speech were automatically extracted, correlated with NPS. Machine learning experiments were carried out to validate the diagnostic power of extracted markers.

RESULTS

Different speech variables are associated with specific NPS; apathy correlates with temporal aspects, and anxiety with voice quality-and this was mostly consistent between male and female after correction for cognitive impairment. Machine learning regressors are able to extract information from speech features and perform above baseline in predicting anxiety, apathy, and depression scores.

CONCLUSIONS

Different NPS seem to be characterized by distinct speech features, which are easily extractable automatically from short vocal tasks. These findings support the use of speech analysis for detecting subtypes of NPS in patients with cognitive impairment. This could have great implications for the design of future clinical trials as this cost-effective method could allow more continuous and even remote monitoring of symptoms.

Implementation of automatic speech analysis for early detection of psychiatric symptoms: What do patients want?

Psychiatry is in dire need of a method to aid early detection of symptoms. Recent developments in automatic speech analysis prove promising in this regard, and open avenues for implementation of speech-based applications to detect psychiatric symptoms. The current survey was conducted to assess positions with regard to speech recordings among a group (n = 675) of individuals who experience psychiatric symptoms. Overall, respondents are open to the idea of speech recordings in light of their mental welfare. Importantly, concerns with regard to privacy were raised. Given that speech recordings are privacy sensitive, this requires special attention upon implementation of automatic speech analysis techniques. Furthermore, respondents indicated a preference for speech recordings in the presence of a clinician, as opposed to a recording made at home without the clinician present. In developing a speech marker for psychiatry, close collaboration with the intended users is essential to arrive at a truly valid and implementable method.

Natural Language Processing as an Emerging Tool to Detect Late-Life Depression

Late-life depression (LLD) is a major public health concern. Despite the availability of effective treatments for depression, barriers to screening and diagnosis still exist. The use of current standardized depression assessments can lead to underdiagnosis or misdiagnosis due to subjective symptom reporting and the distinct cognitive, psychomotor, and somatic features of LLD. To overcome these limitations, there has been a growing interest in the development of objective measures of depression using artificial intelligence (AI) technologies such as natural language processing (NLP). NLP approaches focus on the analysis of acoustic and linguistic aspects of human language derived from text and speech and can be integrated with machine learning approaches to classify depression and its severity. In this review, we will provide rationale for the use of NLP methods to study depression using speech, summarize previous research using NLP in LLD, compare findings to younger adults with depression and older adults with other clinical conditions, and discuss future directions including the use of complementary AI strategies to fully capture the spectrum of LLD.

Vocal features obtained through automated methods in verbal fluency tasks can aid the identification of mixed episodes in bipolar disorder

There is a lack of consensus on the diagnostic thresholds that could improve the detection accuracy of bipolar mixed episodes in clinical settings. Some studies have shown that voice features could be reliable biomarkers of manic and depressive episodes compared to euthymic states, but none thus far have investigated whether they could aid the distinction between mixed and non-mixed acute bipolar episodes. Here we investigated whether vocal features acquired via verbal fluency tasks could accurately classify mixed states in bipolar disorder using machine learning methods. Fifty-six patients with bipolar disorder were recruited during an acute episode (19 hypomanic, 8 mixed hypomanic, 17 with mixed depression, 12 with depression). Nine different trials belonging to four conditions of verbal fluency tasks-letter, semantic, free word generation, and associational fluency-were administered. Spectral and prosodic features in three conditions were selected for the classification algorithm. Using the leave-one-subject-out (LOSO) strategy to train the classifier, we calculated the accuracy rate, the F1 score, and the Matthews correlation coefficient (MCC). For depression versus mixed depression, the accuracy and F1 scores were high, i.e., respectively 0.83 and 0.86, and the MCC was of 0.64. For hypomania versus mixed hypomania, accuracy and F1 scores were also high, i.e., 0.86 and 0.75, respectively, and the MCC was of 0.57. Given the high rates of correctly classified subjects, vocal features quickly acquired via verbal fluency tasks seem to be reliable biomarkers that could be easily implemented in clinical settings to improve diagnostic accuracy.

A Convenient and Low-Cost Model of Depression Screening and Early Warning Based on Voice Data Using for Public Mental Health

Depression is a common mental health disease, which has great harm to public health. At present, the diagnosis of depression mainly depends on the interviews between doctors and patients, which is subjective, slow and expensive. Voice data are a kind of data that are easy to obtain and have the advantage of low cost. It has been proved that it can be used in the diagnosis of depression. The voice data used for modeling in this study adopted the authoritative public data set, which had passed the ethical review. The features of voice data were extracted by Python programming, and the voice features were stored in the format of CSV files. Through data processing, a big database, containing 1479 voice feature samples, was generated for modeling. Then, the decision tree screening model of depression was established by 10-fold cross validation and algorithm selection. The experiment achieved 83.4% prediction accuracy on voice data set. According to the prediction results of the model, the patients can be given early warning and intervention in time, so as to realize the health management of personal depression.

Perceived Utility and Characterization of Personal Google Search Histories to Detect Data Patterns Proximal to a Suicide Attempt in Individuals Who Previously Attempted Suicide: Pilot Cohort Study

BACKGROUND

Despite decades of research to better understand suicide risk and to develop detection and prevention methods, suicide is still one of the leading causes of death globally. While large-scale studies using real-world evidence from electronic health records can identify who is at risk, they have not been successful at pinpointing when someone is at risk. Personalized social media and online search history data, by contrast, could provide an ongoing real-world datastream revealing internal thoughts and personal states of mind.

OBJECTIVE

We conducted this study to determine the feasibility and acceptability of using personalized online information-seeking behavior in the identification of risk for suicide attempts.

METHODS

This was a cohort survey study to assess attitudes of participants with a prior suicide attempt about using web search data for suicide prevention purposes, dates of lifetime suicide attempts, and an optional one-time download of their past web searches on Google. The study was conducted at the University of Washington School of Medicine Psychiatry Research Offices. The main outcomes were participants' opinions on internet search data for suicide prediction and intervention and any potential change in online information-seeking behavior proximal to a suicide attempt. Individualized nonparametric association analysis was used to assess the magnitude of difference in web search data features derived from time periods proximal (7, 15, 30, and 60 days) to the suicide attempts versus the typical (baseline) search behavior of participants.

RESULTS

A total of 62 participants who had attempted suicide in the past agreed to participate in the study. Internet search activity varied from person to person (median 2-24 searches per day). Changes in online search behavior proximal to suicide attempts were evident up to 60 days before attempt. For a subset of attempts (7/30, 23%) search features showed associations from 2 months to a week before the attempt. The top 3 search constructs associated with attempts were online searching patterns (9/30 attempts, 30%), semantic relatedness of search queries to suicide methods (7/30 attempts, 23%), and anger (7/30 attempts, 23%). Participants (40/59, 68%) indicated that use of this personalized web search data for prevention purposes was acceptable with noninvasive potential interventions such as connection to a real person (eg, friend, family member, or counselor); however, concerns were raised about detection accuracy, privacy, and the potential for overly invasive intervention.

CONCLUSIONS

Changes in online search behavior may be a useful and acceptable means of detecting suicide risk. Personalized analysis of online information-seeking behavior showed notable changes in search behavior and search terms that are tied to early warning signs of suicide and are evident 2 months to 7 days before a suicide attempt.

Using machine learning analyses of speech to classify levels of expressed emotion in parents of youth with mood disorders

Expressed emotion (EE), a measure of attitudes among caregivers towards a patient with a psychiatric disorder, is a robust predictor of relapse across mood and psychotic disorders. Because the measurement of EE is time-intensive and costly, its use in clinical settings has been limited. In an effort to automate EE classification, we evaluated whether machine learning (ML) applied to lexical features of speech samples can accurately categorize parents as high or low in EE and in its subtypes (criticism, overinvolvement, and warmth). The sample was 123 parents of youth who had active mood symptoms and a family history of bipolar disorder. Using ML algorithms, we achieved 75.2-81.8% accuracy (sensitivities of ~0.7 and specificities of ~0.8) in classifying parents as high or low in EE and EE subtypes. Additionally, machine-derived EE classifications and observer-rated EE classifications had simiar relationships with youth mood symptoms, parental distress, and family conflict. Of note, criticism related to greater manic severity, parental distress, and family conflict. Study findings indicate that EE classification can be automated through lexical analysis and suggest potential for facilitating larger-scale applications in clinical settings. The results also provide initial indications of the digital phenotypes that underlie EE and its subtypes.

Development and Open Trial of a Technology-Enhanced Family Intervention for Adolescents at Risk for Mood Disorders

AIM

Integrating psychosocial interventions with mobile apps may increase treatment engagement among adolescents. We examined the user experience, uptake, and clinical effects of a mobile-enhanced family-focused therapy (FFT) among adolescents at risk for mood disorders.

METHOD

We created a mobile app containing 12 lesson plans corresponding to content of weekly FFT sessions, with modules concerning mood management, family communication and problem-solving. We pilot tested the app in an open trial of FFT (12 sessions in 18 weeks) for adolescents who had active depressive or hypomanic symptoms, a parent with mood disorder, and at least one parent who expressed high levels of criticism. Teens and parents made daily and weekly ratings of youths' moods, amount of parent/offspring criticism, and practice of FFT psychoeducational, communication or problem-solving skills. Independent evaluators interviewed adolescents at baseline and every 9 weeks over 27 weeks to measure symptom trajectories.

RESULTS

Participants were adolescents (n=22; mean age 15.4 ± 1.8 years; 45.5% female) and their 34 parents. Completion of requested app assessment and skill practices averaged 46%-65% among adolescents and parents over 18 weeks of treatment. Adolescents showed significant improvement in clinician-rated depression scores over 27 weeks (Cohen's d=1.58, 95% CI, 0.83 to 2.32) and reported reductions in the amount of perceived criticism expressed by parents.

LIMITATIONS

The uncontrolled design limits inferences about whether the mobile app augmented the effects of FFT on moods or family relationships.

CONCLUSIONS

Mobile applications may enhance users' responses to family therapy and provide clinicians with information regarding clinical status. Clinicaltrials.gov NCT03913013.

Blinded Clinical Ratings of Social Media Data are Correlated with In-Person Clinical Ratings in Participants Diagnosed with Either Depression, Schizophrenia, or Healthy Controls

This study investigates clinically valid signals about psychiatric symptoms in social media data, by rating severity of psychiatric symptoms in donated, de-identified Facebook posts and comparing to in-person clinical assessments. Participants with schizophrenia (N=8), depression (N=7), or who were healthy controls (N=8) also consented to the collection of their Facebook activity from three months before the in-person assessments to six weeks after this evaluation. Depressive symptoms were assessed in- person using the Montgomery-Åsberg Depression Rating Scale (MADRS), psychotic symptoms were assessed using the Brief Psychiatric Rating Scale (BPRS), and global functioning was assessed using the Community Assessment of Psychotic Experiences (CAPE-42). Independent raters (psychiatrists, non-psychiatrist mental health clinicians, and two staff members) rated depression, psychosis, and global functioning symptoms from the social media activity of deidentified participants. The correlations between in-person clinical ratings and blinded ratings based on social media data were evaluated. Significant correlations (and trends for significance in the mixed model controlling for multiple raters) were found for psychotic symptoms, global symptom ratings and depressive symptoms. Results like these, indicating the presence of clinically valid signal in social media, are an important step toward developing computational tools that could assist clinicians by providing additional data outside the context of clinical encounters.

Language as a biomarker for psychosis: A natural language processing approach

Human ratings of conceptual disorganization, poverty of content, referential cohesion and illogical thinking have been shown to predict psychosis onset in prospective clinical high risk (CHR) cohort studies. The potential value of linguistic biomarkers has been significantly magnified, however, by recent advances in natural language processing (NLP) and machine learning (ML). Such methodologies allow for the rapid and objective measurement of language features, many of which are not easily recognized by human raters. Here we review the key findings on language production disturbance in psychosis. We also describe recent advances in the computational methods used to analyze language data, including methods for the automatic measurement of discourse coherence, syntactic complexity, poverty of content, referential coherence, and metaphorical language. Linguistic biomarkers of psychosis risk are now undergoing cross-validation, with attention to harmonization of methods. Future directions in extended CHR networks include studies of sources of variance, and combination with other promising biomarkers of psychosis risk, such as cognitive and sensory processing impairments likely to be related to language. Implications for the broader study of social communication, including reciprocal prosody, face expression and gesture, are discussed.

Understanding Language Abnormalities and Associated Clinical Markers in Psychosis: The Promise of Computational Methods

The language and speech of individuals with psychosis reflect their impairments in cognition and motor processes. These language disturbances can be used to identify individuals with and at high risk for psychosis, as well as help track and predict symptom progression, allowing for early intervention and improved outcomes. However, current methods of language assessment-manual annotations and/or clinical rating scales-are time intensive, expensive, subject to bias, and difficult to administer on a wide scale, limiting this area from reaching its full potential. Computational methods that can automatically perform linguistic analysis have started to be applied to this problem and could drastically improve our ability to use linguistic information clinically. In this article, we first review how these automated, computational methods work and how they have been applied to the field of psychosis. We show that across domains, these methods have captured differences between individuals with psychosis and healthy controls and can classify individuals with high accuracies, demonstrating the promise of these methods. We then consider the obstacles that need to be overcome before these methods can play a significant role in the clinical process and provide suggestions for how the field should address them. In particular, while much of the work thus far has focused on demonstrating the successes of these methods, we argue that a better understanding of when and why these models fail will be crucial toward ensuring these methods reach their potential in the field of psychosis.

"How are you?" Estimation of anxiety, sleep quality, and mood using computational voice analysis

We developed a method of estimating impactors of cognitive function (ICF) - such as anxiety, sleep quality, and mood - using computational voice analysis. Clinically validated questionnaires (VQs) were used to score anxiety, sleep and mood while salient voice features were extracted to train regression models with deep neural networks. Experiments with 203 subjects showed promising results with significant concordance correlation coefficients (CCC) between actual VQ scores and the predicted scores (0.46 = anxiety, 0.50 = sleep quality, 0.45 = mood).