Development of an Emotion-Sensitive mHealth Approach for Mood-State Recognition in Bipolar Disorder

Acoustic and Natural Language Markers for Bipolar Disorder: A Pilot, mHealth Cross-Sectional Study

Background

Monitoring symptoms of bipolar disorder (BD) is a challenge faced by mental health services. Speech patterns are crucial in assessing the current experiences, emotions, and thought patterns of people with BD. Natural language processing (NLP) and acoustic signal processing may support ongoing BD assessment within a mobile health (mHealth) framework.

Objective

Using both acoustic and NLP-based features from the speech of people with BD, we built an app-based tool and tested its feasibility and performance to remotely assess the individual clinical status.

Methods

We carried out a pilot, observational study, sampling adults diagnosed with BD from the caseload of the Nord Milano Mental Health Trust (Italy) to explore the relationship between selected speech features and symptom severity and to test their potential to remotely assess mental health status. Symptom severity assessment was based on clinician ratings, using the Young Mania Rating Scale (YMRS) and Montgomery-Åsberg Depression Rating Scale (MADRS) for manic and depressive symptoms, respectively. Leveraging a digital health tool embedded in a mobile app, which records and processes speech, participants self-administered verbal performance tasks. Both NLP-based and acoustic features were extracted, testing associations with mood states and exploiting machine learning approaches based on random forest models.

Results

We included 32 subjects (mean [SD] age 49.6 [14.3] years; 50% [16/32] females) with a MADRS median (IQR) score of 13 (21) and a YMRS median (IQR) score of 5 (16). Participants freely managed the digital environment of the app, without perceiving it as intrusive and reporting an acceptable system usability level (average score 73.5, SD 19.7). Small-to-moderate correlations between speech features and symptom severity were uncovered, with sex-based differences in predictive capability. Higher latency time (ρ=0.152), increased silences (ρ=0.416), and vocal perturbations correlated with depressive symptomatology. Pressure of speech based on the mean intraword time (ρ=-0.343) and lower voice instability based on jitter-related parameters (ρ ranging from -0.19 to -0.27) were detected for manic symptoms. However, a higher contribution of NLP-based and conversational features, rather than acoustic features, was uncovered, especially for predictive models for depressive symptom severity (NLP-based: R2=0.25, mean squared error [MSE]=110.07, mean absolute error [MAE]=8.17; acoustics: R2=0.11, MSE=133.75, MAE=8.86; combined: R2=0.16; MSE=118.53, MAE=8.68).

Conclusions

Remotely collected speech patterns, including both linguistic and acoustic features, are associated with symptom severity levels and may help differentiate clinical conditions in individuals with BD during their mood state assessments. In the future, multimodal, smartphone-integrated digital ecological momentary assessments could serve as a powerful tool for clinical purposes, remotely complementing standard, in-person mental health evaluations.

Digital phenotyping in psychiatry: A scoping review

BACKGROUND

Digital phenotyping has been defined as the moment-by-moment assessment of an illness state through digital means, promising objective, quantifiable data on psychiatric patients' conditions, and could potentially improve diagnosis and management of mental illness. As it is a rapidly growing field, it is to be expected that new literature is being published frequently.

OBJECTIVE

We conducted this scoping review to assess the current state of literature on digital phenotyping and offer some discussion on the current trends and future direction of this area of research.

METHODS

We searched four databases, PubMed, Ovid MEDLINE, PsycINFO and Web of Science, from inception to August 25th, 2021. We included studies written in English that 1) investigated or applied their findings to diagnose psychiatric disorders and 2) utilized passive sensing for management or diagnosis. Protocols were excluded. A narrative synthesis approach was used, due to the heterogeneity and variability in outcomes and outcome types reported.

RESULTS

Of 10506 unique records identified, we included a total of 107 articles. The number of published studies has increased over tenfold from 2 in 2014 to 28 in 2020, illustrating the field's rapid growth. However, a significant proportion of these (49% of all studies and 87% of primary studies) were proof of concept, pilot or correlational studies examining digital phenotyping's potential. Most (62%) of the primary studies published evaluated individuals with depression (21%), BD (18%) and SZ (23%) (Appendix 1).

CONCLUSION

There is promise shown in certain domains of data and their clinical relevance, which have yet to be fully elucidated. A consensus has yet to be reached on the best methods of data collection and processing, and more multidisciplinary collaboration between physicians and other fields is needed to unlock the full potential of digital phenotyping and allow for statistically powerful clinical trials to prove clinical utility.

Wearable, Environmental, and Smartphone-Based Passive Sensing for Mental Health Monitoring

Collecting and analyzing data from sensors embedded in the context of daily life has been widely employed for the monitoring of mental health. Variations in parameters such as movement, sleep duration, heart rate, electrocardiogram, skin temperature, etc., are often associated with psychiatric disorders. Namely, accelerometer data, microphone, and call logs can be utilized to identify voice features and social activities indicative of depressive symptoms, and physiological factors such as heart rate and skin conductance can be used to detect stress and anxiety disorders. Therefore, a wide range of devices comprising a variety of sensors have been developed to capture these physiological and behavioral data and translate them into phenotypes and states related to mental health. Such systems aim to identify behaviors that are the consequence of an underlying physiological alteration, and hence, the raw sensor data are captured and converted into features that are used to define behavioral markers, often through machine learning. However, due to the complexity of passive data, these relationships are not simple and need to be well-established. Furthermore, parameters such as intrapersonal and interpersonal differences need to be considered when interpreting the data. Altogether, combining practical mobile and wearable systems with the right data analysis algorithms can provide a useful tool for the monitoring and management of mental disorders. The current review aims to comprehensively present and critically discuss all available smartphone-based, wearable, and environmental sensors for detecting such parameters in relation to the treatment and/or management of the most common mental health conditions.