Giving Voice to Vulnerable Children: Machine Learning Analysis of Speech Detects Anxiety and Depression in Early Childhood

Using Wearable Digital Devices to Screen Children for Mental Health Conditions: Ethical Promises and Challenges

In response to a burgeoning pediatric mental health epidemic, recent guidelines have instructed pediatricians to regularly screen their patients for mental health disorders with consistency and standardization. Yet, gold-standard screening surveys to evaluate mental health problems in children typically rely solely on reports given by caregivers, who tend to unintentionally under-report, and in some cases over-report, child symptomology. Digital phenotype screening tools (DPSTs), currently being developed in research settings, may help overcome reporting bias by providing objective measures of physiology and behavior to supplement child mental health screening. Prior to their implementation in pediatric practice, however, the ethical dimensions of DPSTs should be explored. Herein, we consider some promises and challenges of DPSTs under three broad categories: accuracy and bias, privacy, and accessibility and implementation. We find that DPSTs have demonstrated accuracy, may eliminate concerns regarding under- and over-reporting, and may be more accessible than gold-standard surveys. However, we also find that if DPSTs are not responsibly developed and deployed, they may be biased, raise privacy concerns, and be cost-prohibitive. To counteract these potential shortcomings, we identify ways to support the responsible and ethical development of DPSTs for clinical practice to improve mental health screening in children.

Exploring the impact of type II diabetes mellitus on voice quality

PURPOSE

This cross-sectional study aimed to investigate the potential of voice analysis as a prescreening tool for type II diabetes mellitus (T2DM) by examining the differences in voice recordings between non-diabetic and T2DM participants.

METHODS

60 participants diagnosed as non-diabetic (n = 30) or T2DM (n = 30) were recruited on the basis of specific inclusion and exclusion criteria in Iran between February 2020 and September 2023. Participants were matched according to their year of birth and then placed into six age categories. Using the WhatsApp application, participants recorded the translated versions of speech elicitation tasks. Seven acoustic features [fundamental frequency, jitter, shimmer, harmonic-to-noise ratio (HNR), cepstral peak prominence (CPP), voice onset time (VOT), and formant (F1-F2)] were extracted from each recording and analyzed using Praat software. Data was analyzed with Kolmogorov-Smirnov, two-way ANOVA, post hoc Tukey, binary logistic regression, and student t tests.

RESULTS

The comparison between groups showed significant differences in fundamental frequency, jitter, shimmer, CPP, and HNR (p < 0.05), while there were no significant differences in formant and VOT (p > 0.05). Binary logistic regression showed that shimmer was the most significant predictor of the disease group. There was also a significant difference between diabetes status and age, in the case of CPP.

CONCLUSIONS

Participants with type II diabetes exhibited significant vocal variations compared to non-diabetic controls.

Exploring a multimodal approach for utilizing digital biomarkers for childhood mental health screening

Background

Depression and anxiety are prevalent mental health concerns among children and adolescents. The application of conventional assessment methods, such as survey questionnaires to children, may lead to self-reporting issues. Digital biomarkers provide extensive data, reducing bias in mental health self-reporting, and significantly influence patient screening. Our primary objectives were to accurately assess children's mental health and to investigate the feasibility of using various digital biomarkers.

Methods

This study included a total of 54 boys and girls aged between 7 to 11 years. Each participant's mental state was assessed using the Depression, Anxiety, and Stress Scale. Subsequently, the subjects participated in digital biomarker collection tasks. Heart rate variability (HRV) data were collected using a camera sensor. Eye-tracking data were collected through tasks displaying emotion-face stimuli. Voice data were obtained by recording the participants' voices while they engaged in free speech and description tasks.

Results

Depressive symptoms were positively correlated with low frequency (LF, 0.04-0.15 Hz of HRV) in HRV and negatively associated with eye-tracking variables. Anxiety symptoms had a negative correlation with high frequency (HF, 0.15-0.40 Hz of HRV) in HRV and a positive association with LF/HF. Regarding stress, eye-tracking variables indicated a positive correlation, while pNN50, which represents the proportion of NN50 (the number of pairs of successive R-R intervals differing by more than 50 milliseconds) divided by the total number of NN (R-R) intervals, exhibited a negative association. Variables identified for childhood depression included LF and the total time spent looking at a sad face. Those variables recognized for anxiety were LF/HF, heart rate (HR), and pNN50. For childhood stress, HF, LF, and Jitter showed different correlation patterns between the two grade groups.

Discussion

We examined the potential of multimodal biomarkers in children, identifying features linked to childhood depression, particularly LF and the Sad.TF:time. Anxiety was most effectively explained by HRV features. To explore reasons for non-replication of previous studies, we categorized participants by elementary school grades into lower grades (1st, 2nd, 3rd) and upper grades (4th, 5th, 6th).

Conclusion

This study confirmed the potential use of multimodal digital biomarkers for children's mental health screening, serving as foundational research.

The ChAMP App: A Scalable mHealth Technology for Detecting Digital Phenotypes of Early Childhood Mental Health

Childhood mental health problems are common, impairing, and can become chronic if left untreated. Children are not reliable reporters of their emotional and behavioral health, and caregivers often unintentionally under- or over-report child symptoms, making assessment challenging. Objective physiological and behavioral measures of emotional and behavioral health are emerging. However, these methods typically require specialized equipment and expertise in data and sensor engineering to administer and analyze. To address this challenge, we have developed the ChAMP (Childhood Assessment and Management of digital Phenotypes) System, which includes a mobile application for collecting movement and audio data during a battery of mood induction tasks and an open-source platform for extracting digital biomarkers. As proof of principle, we present ChAMP System data from 101 children 4-8 years old, with and without diagnosed mental health disorders. Machine learning models trained on these data detect the presence of specific disorders with 70-73% balanced accuracy, with similar results to clinical thresholds on established parent-report measures (63-82% balanced accuracy). Features favored in model architectures are described using Shapley Additive Explanations (SHAP). Canonical Correlation Analysis reveals moderate to strong associations between predictors of each disorder and associated symptom severity (r = .51-.83). The open-source ChAMP System provides clinically-relevant digital biomarkers that may later complement parent-report measures of emotional and behavioral health for detecting kids with underlying mental health conditions and lowers the barrier to entry for researchers interested in exploring digital phenotyping of childhood mental health.

The ChAMP App: A Scalable mHealth Technology for Detecting Digital Phenotypes of Early Childhood Mental Health

Childhood mental health problems are common, impairing, and can become chronic if left untreated. Children are not reliable reporters of their emotional and behavioral health, and caregivers often unintentionally under- or over-report child symptoms, making assessment challenging. Objective physiological and behavioral measures of emotional and behavioral health are emerging. However, these methods typically require specialized equipment and expertise in data and sensor engineering to administer and analyze. To address this challenge, we have developed the ChAMP (Childhood Assessment and Management of digital Phenotypes) System, which includes a mobile application for collecting movement and audio data during a battery of mood induction tasks and an open-source platform for extracting digital biomarkers. As proof of principle, we present ChAMP System data from 101 children 4-8 years old, with and without diagnosed mental health disorders. Machine learning models trained on these data detect the presence of specific disorders with 70-73% balanced accuracy, with similar results to clinical thresholds on established parent-report measures (63-82% balanced accuracy). Features favored in model architectures are described using Shapley Additive Explanations (SHAP). Canonical Correlation Analysis reveals moderate to strong associations between predictors of each disorder and associated symptom severity (r = .51-.83). The open-source ChAMP System provides clinically-relevant digital biomarkers that may later complement parent-report measures of emotional and behavioral health for detecting kids with underlying mental health conditions and lowers the barrier to entry for researchers interested in exploring digital phenotyping of childhood mental health.

A systematic review on automated clinical depression diagnosis

Assessing mental health disorders and determining treatment can be difficult for a number of reasons, including access to healthcare providers. Assessments and treatments may not be continuous and can be limited by the unpredictable nature of psychiatric symptoms. Machine-learning models using data collected in a clinical setting can improve diagnosis and treatment. Studies have used speech, text, and facial expression analysis to identify depression. Still, more research is needed to address challenges such as the need for multimodality machine-learning models for clinical use. We conducted a review of studies from the past decade that utilized speech, text, and facial expression analysis to detect depression, as defined by the Diagnostic and Statistical Manual of Mental Disorders (DSM-5), using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guideline. We provide information on the number of participants, techniques used to assess clinical outcomes, speech-eliciting tasks, machine-learning algorithms, metrics, and other important discoveries for each study. A total of 544 studies were examined, 264 of which satisfied the inclusion criteria. A database has been created containing the query results and a summary of how different features are used to detect depression. While machine learning shows its potential to enhance mental health disorder evaluations, some obstacles must be overcome, especially the requirement for more transparent machine-learning models for clinical purposes. Considering the variety of datasets, feature extraction techniques, and metrics used in this field, guidelines have been provided to collect data and train machine-learning models to guarantee reproducibility and generalizability across different contexts.

Using Wearable Devices and Speech Data for Personalized Machine Learning in Early Detection of Mental Disorders: Protocol for a Participatory Research Study

BACKGROUND

Early identification of mental disorder symptoms is crucial for timely treatment and reduction of recurring symptoms and disabilities. A tool to help individuals recognize warning signs is important. We posit that such a tool would have to rely on longitudinal analysis of patterns and trends in the individual's daily activities and mood, which can now be captured through data from wearable activity trackers, speech recordings from mobile devices, and the individual's own description of their mental state. In this paper, we describe such a tool developed by our team to detect early signs of depression, anxiety, and stress.

OBJECTIVE

This study aims to examine three questions about the effectiveness of machine learning models constructed based on multimodal data from wearables, speech, and self-reports: (1) How does speech about issues of personal context differ from speech while reading a neutral text, what type of speech data are more helpful in detecting mental health indicators, and how is the quality of the machine learning models influenced by multilanguage data? (2) Does accuracy improve with longitudinal data collection and how, and what are the most important features? and (3) How do personalized machine learning models compare against population-level models?

METHODS

We collect longitudinal data to aid machine learning in accurately identifying patterns of mental disorder symptoms. We developed an app that collects voice, physiological, and activity data. Physiological and activity data are provided by a variety of off-the-shelf fitness trackers, that record steps, active minutes, duration of sleeping stages (rapid eye movement, deep, and light sleep), calories consumed, distance walked, heart rate, and speed. We also collect voice recordings of users reading specific texts and answering open-ended questions chosen randomly from a set of questions without repetition. Finally, the app collects users' answers to the Depression, Anxiety, and Stress Scale. The collected data from wearable devices and voice recordings will be used to train machine learning models to predict the levels of anxiety, stress, and depression in participants.

RESULTS

The study is ongoing, and data collection will be completed by November 2023. We expect to recruit at least 50 participants attending 2 major universities (in Canada and Mexico) fluent in English or Spanish. The study will include participants aged between 18 and 35 years, with no communication disorders, acute neurological diseases, or history of brain damage. Data collection complied with ethical and privacy requirements.

CONCLUSIONS

The study aims to advance personalized machine learning for mental health; generate a data set to predict Depression, Anxiety, and Stress Scale results; and deploy a framework for early detection of depression, anxiety, and stress. Our long-term goal is to develop a noninvasive and objective method for collecting mental health data and promptly detecting mental disorder symptoms.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/48210.

A Wearable Artificial Intelligence Feedback Tool (Wrist Angel) for Treatment and Research of Obsessive Compulsive Disorder: Protocol for a Nonrandomized Pilot Study

BACKGROUND

Obsessive compulsive disorder (OCD) in youth is characterized by behaviors, emotions, physiological reactions, and family interaction patterns. An essential component of therapy involves increasing awareness of the links among thoughts, emotions, behaviors, bodily sensations, and family interactions. An automatic assessment tool using physiological signals from a wearable biosensor may enable continuous symptom monitoring inside and outside of the clinic and support cognitive behavioral therapy for OCD.

OBJECTIVE

The primary aim of this study is to evaluate the feasibility and acceptability of using a wearable biosensor to monitor OCD symptoms. The secondary aim is to explore the feasibility of developing clinical and research tools that can detect and predict OCD-relevant internal states and interpersonal processes with the use of speech and behavioral signals.

METHODS

Eligibility criteria for the study include children and adolescents between 8 and 17 years of age diagnosed with OCD, controls with no psychiatric diagnoses, and one parent of the participating youths. Youths and parents wear biosensors on their wrists that measure pulse, electrodermal activity, skin temperature, and acceleration. Patients and their parents mark OCD episodes, while control youths and their parents mark youth fear episodes. Continuous, in-the-wild data collection will last for 8 weeks. Controlled experiments designed to link physiological, speech, behavioral, and biochemical signals to mental states are performed at baseline and after 8 weeks. Interpersonal interactions in the experiments are filmed and coded for behavior. The films are also processed with computer vision and for speech signals. Participants complete clinical interviews and questionnaires at baseline, and at weeks 4, 7, and 8. Feasibility criteria were set for recruitment, retention, biosensor functionality and acceptability, adherence to wearing the biosensor, and safety related to the biosensor. As a first step in learning the associations between signals and OCD-related parameters, we will use paired t tests and mixed effects models with repeated measures to assess associations between oxytocin, individual biosignal features, and outcomes such as stress-rest and case-control comparisons.

RESULTS

The first participant was enrolled on December 3, 2021, and recruitment closed on December 31, 2022. Nine patient dyads and nine control dyads were recruited. Sixteen participating dyads completed follow-up assessments.

CONCLUSIONS

The results of this study will provide preliminary evidence for the extent to which a wearable biosensor that collects physiological signals can be used to monitor OCD severity and events in youths. If we find the study to be feasible, further studies will be conducted to integrate biosensor signals output into machine learning algorithms that can provide patients, parents, and therapists with actionable insights into OCD symptoms and treatment progress. Future definitive studies will be tasked with testing the accuracy of machine learning models to detect and predict OCD episodes and classify clinical severity.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05064527; https://clinicaltrials.gov/ct2/show/NCT05064527.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/45123.

Evaluation of the Michigan Clinical Consultation and Care Program: An Evidence-Based Approach to Perinatal Mental Healthcare

Mood and anxiety disorders affect pregnant individuals and their families at increased rates throughout the perinatal period. Geographic, financial, and social barriers often preclude adequate diagnosis and treatment. The aim of this manuscript is to describe the consultation and care arms of the Michigan Clinical Consultation and Care (MC3) program, a statewide program designed to facilitate access to perinatal mental healthcare for OB/Gyn patients, and to describe the participants engaged in the program, examine the predictors of participant retention, and provide preliminary data regarding participants' mental health outcomes. We enrolled 209 participants to the clinical care arm, of which 48 were lost to follow-up, while 107 remained enrolled at the time of data analysis. A total of 54 participants met their treatment goals. A total of 97% of participants asserted they were satisfied with the services they received. Black race and public insurance predicted faster attrition from the care arm treatment; risks for interpersonal violence exposure and substance use were unrelated to attrition. Preliminary mental health outcomes showed significant decreases in anxiety and depression, with the most dramatic decreases in the first month of treatment. Overall, the MC3 clinical care arm shows promising rates of adherence, excellent program satisfaction, and a positive impact on perinatal mental health, supporting continued program implementation and ongoing evaluation.

Toward Digital Phenotypes of Early Childhood Mental Health via Unsupervised and Supervised Machine Learning

Childhood mental health disorders such as anxiety, depression, and ADHD are commonly-occurring and often go undetected into adolescence or adulthood. This can lead to detrimental impacts on long-term wellbeing and quality of life. Current parent-report assessments for pre-school aged children are often biased, and thus increase the need for objective mental health screening tools. Leveraging digital tools to identify the behavioral signature of childhood mental disorders may enable increased intervention at the time with the highest chance of long-term impact. We present data from 84 participants (4-8 years old, 50% diagnosed with anxiety, depression, and/or ADHD) collected during a battery of mood induction tasks using the ChAMP System. Unsupervised Kohonen Self-Organizing Maps (SOM) constructed from movement and audio features indicate that age did not tend to explain clusters as consistently as gender within task-specific and cross-task SOMs. Symptom prevalence and diagnostic status also showed some evidence of clustering. Case studies suggest that high impairment (>80th percentile symptom counts) and diagnostic subtypes (ADHD-Combined) may account for most behaviorally distinct children. Based on this same dataset, we also present results from supervised modeling for the binary classification of diagnoses. Our top performing models yield moderate but promising results (ROC AUC .6-.82, TPR .36-.71, Accuracy .62-.86) on par with our previous efforts for isolated behavioral tasks. Enhancing features, tuning model parameters, and incorporating additional wearable sensor data will continue to enable the rapid progression towards the discovery of digital phenotypes of childhood mental health.Clinical Relevance- This work advances the use of wearables for detecting childhood mental health disorders.

Fast and accurate assessment of depression based on voice acoustic features: a cross-sectional and longitudinal study

Background

Depression is a widespread mental disorder that affects a significant portion of the population. However, the assessment of depression is often subjective, relying on standard questions or interviews. Acoustic features have been suggested as a reliable and objective alternative for depression assessment. Therefore, in this study, we aim to identify and explore voice acoustic features that can effectively and rapidly predict the severity of depression, as well as investigate the potential correlation between specific treatment options and voice acoustic features.

Methods

We utilized voice acoustic features correlated with depression scores to train a prediction model based on artificial neural network. Leave-one-out cross-validation was performed to evaluate the performance of the model. We also conducted a longitudinal study to analyze the correlation between the improvement of depression and changes in voice acoustic features after an Internet-based cognitive-behavioral therapy (ICBT) program consisting of 12 sessions.

Results

Our study showed that the neural network model trained based on the 30 voice acoustic features significantly correlated with HAMD scores can accurately predict the severity of depression with an absolute mean error of 3.137 and a correlation coefficient of 0.684. Furthermore, four out of the 30 features significantly decreased after ICBT, indicating their potential correlation with specific treatment options and significant improvement in depression (p < 0.05).

Conclusion

Voice acoustic features can effectively and rapidly predict the severity of depression, providing a low-cost and efficient method for screening patients with depression on a large scale. Our study also identified potential acoustic features that may be significantly related to specific treatment options for depression.

Deep learning and machine learning in psychiatry: a survey of current progress in depression detection, diagnosis and treatment

Informatics paradigms for brain and mental health research have seen significant advances in recent years. These developments can largely be attributed to the emergence of new technologies such as machine learning, deep learning, and artificial intelligence. Data-driven methods have the potential to support mental health care by providing more precise and personalised approaches to detection, diagnosis, and treatment of depression. In particular, precision psychiatry is an emerging field that utilises advanced computational techniques to achieve a more individualised approach to mental health care. This survey provides an overview of the ways in which artificial intelligence is currently being used to support precision psychiatry. Advanced algorithms are being used to support all phases of the treatment cycle. These systems have the potential to identify individuals suffering from mental health conditions, allowing them to receive the care they need and tailor treatments to individual patients who are mostly to benefit. Additionally, unsupervised learning techniques are breaking down existing discrete diagnostic categories and highlighting the vast disease heterogeneity observed within depression diagnoses. Artificial intelligence also provides the opportunity to shift towards evidence-based treatment prescription, moving away from existing methods based on group averages. However, our analysis suggests there are several limitations currently inhibiting the progress of data-driven paradigms in care. Significantly, none of the surveyed articles demonstrate empirically improved patient outcomes over existing methods. Furthermore, greater consideration needs to be given to uncertainty quantification, model validation, constructing interdisciplinary teams of researchers, improved access to diverse data and standardised definitions within the field. Empirical validation of computer algorithms via randomised control trials which demonstrate measurable improvement to patient outcomes are the next step in progressing models to clinical implementation.

Functional connectivity analysis of childhood depressive symptoms

BACKGROUND

Childhood depression is a highly distinct and prevalent condition with an unknown neurobiological basis. We wish to explore the resting state fMRI data in children for potential associations between neural connectivity and childhood depressive symptoms.

METHODS

A longitudinal birth cohort study with neuroimaging data obtained at 4.5, 6.0 and 7.5 years of age and the Children Depression Inventory 2 (CDI) administered between 8.5 and 10.5 years was used. The CDI score was used as the dependent variable and tested for correlation, both simple Pearson and network based statistic, with the functional connectivity values obtained from the resting state fMRI. Cross-validated permutation testing with a general linear model was used to validate that the identified functional connections were indeed implicated in childhood depression.

RESULTS

Ten functional connections and four brain regions (Somatomotor Area B, Temporoparietal Junction, Orbitofrontal Cortex and Insula) were identified as significantly associated with childhood depressive symptoms for girls at 6.0 and 7.5 years. No significant functional connections were found in girls at 4.5 years or for boys at any timepoint. Network based statistic and permutation testing confirmed these findings.

CONCLUSIONS

This study revealed significant sex-dependent associations of neural connectivity and childhood depressive symptoms. The regions identified are implicated in speech/language, social cognition and information integration and suggest unique pathways to childhood depressive symptoms.

Screening Children's Intellectual Disabilities with Phonetic Features, Facial Phenotype and Craniofacial Variability Index

BACKGROUND

Intellectual Disability (ID) is a kind of developmental deficiency syndrome caused by congenital diseases or postnatal events. This syndrome could be intervened as soon as possible if its early screening was efficient, which may improve the condition of patients and enhance their self-care ability. The early screening of ID is always achieved by clinical interview, which needs in-depth participation of medical professionals and related medical resources.

METHODS

A new method for screening ID has been proposed by analyzing the facial phenotype and phonetic characteristic of young subjects. First, the geometric features of subjects' faces and phonetic features of subjects' voice are extracted from interview videos, then craniofacial variability index (CVI) is calculated with the geometric features and the risk of ID is given with the measure of CVI. Furthermore, machine learning algorithms are utilized to establish a method for further screening ID based on facial features and phonetic features.

RESULTS

The proposed method using three feature sets, including geometric features, CVI features and phonetic features was evaluated. The best performance of accuracy was closer to 80%.

CONCLUSIONS

The results using the three feature sets revealed that the proposed method may be applied in a clinical setting in the future after continuous improvement.

Applications of Speech Analysis in Psychiatry

ABSTRACT

The need for objective measurement in psychiatry has stimulated interest in alternative indicators of the presence and severity of illness. Speech may offer a source of information that bridges the subjective and objective in the assessment of mental disorders. We systematically reviewed the literature for articles exploring speech analysis for psychiatric applications. The utility of speech analysis depends on how accurately speech features represent clinical symptoms within and across disorders. We identified four domains of the application of speech analysis in the literature: diagnostic classification, assessment of illness severity, prediction of onset of illness, and prognosis and treatment outcomes. We discuss the findings in each of these domains, with a focus on how types of speech features characterize different aspects of psychopathology. Models that bring together multiple speech features can distinguish speakers with psychiatric disorders from healthy controls with high accuracy. Differentiating between types of mental disorders and symptom dimensions are more complex problems that expose the transdiagnostic nature of speech features. Convergent progress in speech research and computer sciences opens avenues for implementing speech analysis to enhance objectivity of assessment in clinical practice. Application of speech analysis will need to address issues of ethics and equity, including the potential to perpetuate discriminatory bias through models that learn from clinical assessment data. Methods that mitigate bias are available and should play a key role in the implementation of speech analysis.

Speech as a Biomarker for Depression

BACKGROUND

Depression is a debilitating disorder that at present lacks a reliable biomarker to aid in diagnosis and early detection. Recent advances in computational analytic approaches have opened up new avenues in developing such a biomarker by taking advantage of the wealth of information that can be extracted from a person's speech.

OBJECTIVE

The current review provides an overview of the latest findings in the rapidly evolving field of computational language analysis for the detection of depression. We cover a wide range of both acoustic and content-related linguistic features, data types (i.e., spoken and written language), and data sources (i.e., lab settings, social media, and smartphone-based). We put special focus on the current methodological advances with regard to feature extraction and computational modeling techniques. Furthermore, we pay attention to potential hurdles in the implementation of automatic speech analysis.

CONCLUSION

Depressive speech is characterized by several anomalies, such as lower speech rate, less pitch variability and more self-referential speech. With current computational modeling techniques, such features can be used to detect depression with an accuracy of up to 91%. The performance of the models is optimized when machine learning techniques are implemented that suit the type and amount of data. Recent studies now work towards further optimization and generalizability of the computational language models to detect depression. Finally, privacy and ethical issues are of paramount importance to be addressed when automatic speech analysis techniques are further implemented in, for example, smartphones. Altogether, computational speech analysis is well underway towards becoming an effective diagnostic aid for depression.

Evaluating Web-Based Automatic Transcription for Alzheimer Speech Data: Transcript Comparison and Machine Learning Analysis

Background

Speech data for medical research can be collected noninvasively and in large volumes. Speech analysis has shown promise in diagnosing neurodegenerative disease. To effectively leverage speech data, transcription is important, as there is valuable information contained in lexical content. Manual transcription, while highly accurate, limits the potential scalability and cost savings associated with language-based screening.

Objective

To better understand the use of automatic transcription for classification of neurodegenerative disease, namely, Alzheimer disease (AD), mild cognitive impairment (MCI), or subjective memory complaints (SMC) versus healthy controls, we compared automatically generated transcripts against transcripts that went through manual correction.

Methods

We recruited individuals from a memory clinic (“patients”) with a diagnosis of mild-to-moderate AD, (n=44, 30%), MCI (n=20, 13%), SMC (n=8, 5%), as well as healthy controls (n=77, 52%) living in the community. Participants were asked to describe a standardized picture, read a paragraph, and recall a pleasant life experience. We compared transcripts generated using Google speech-to-text software to manually verified transcripts by examining transcription confidence scores, transcription error rates, and machine learning classification accuracy. For the classification tasks, logistic regression, Gaussian naive Bayes, and random forests were used.

Results

The transcription software showed higher confidence scores (P<.001) and lower error rates (P>.05) for speech from healthy controls compared with patients. Classification models using human-verified transcripts significantly (P<.001) outperformed automatically generated transcript models for both spontaneous speech tasks. This comparison showed no difference in the reading task. Manually adding pauses to transcripts had no impact on classification performance. However, manually correcting both spontaneous speech tasks led to significantly higher performances in the machine learning models.

Conclusions

We found that automatically transcribed speech data could be used to distinguish patients with a diagnosis of AD, MCI, or SMC from controls. We recommend a human verification step to improve the performance of automatic transcripts, especially for spontaneous tasks. Moreover, human verification can focus on correcting errors and adding punctuation to transcripts. However, manual addition of pauses is not needed, which can simplify the human verification step to more efficiently process large volumes of speech data.

Acoustic and Linguistic Features of Impromptu Speech and Their Association With Anxiety: Validation Study

BACKGROUND

The measurement and monitoring of generalized anxiety disorder requires frequent interaction with psychiatrists or psychologists. Access to mental health professionals is often difficult because of high costs or insufficient availability. The ability to assess generalized anxiety disorder passively and at frequent intervals could be a useful complement to conventional treatment and help with relapse monitoring. Prior work suggests that higher anxiety levels are associated with features of human speech. As such, monitoring speech using personal smartphones or other wearable devices may be a means to achieve passive anxiety monitoring.

OBJECTIVE

This study aims to validate the association of previously suggested acoustic and linguistic features of speech with anxiety severity.

METHODS

A large number of participants (n=2000) were recruited and participated in a single web-based study session. Participants completed the Generalized Anxiety Disorder 7-item scale assessment and provided an impromptu speech sample in response to a modified version of the Trier Social Stress Test. Acoustic and linguistic speech features were a priori selected based on the existing speech and anxiety literature, along with related features. Associations between speech features and anxiety levels were assessed using age and personal income as covariates.

RESULTS

Word count and speaking duration were negatively correlated with anxiety scores (r=-0.12; P<.001), indicating that participants with higher anxiety scores spoke less. Several acoustic features were also significantly (P<.05) associated with anxiety, including the mel-frequency cepstral coefficients, linear prediction cepstral coefficients, shimmer, fundamental frequency, and first formant. In contrast to previous literature, second and third formant, jitter, and zero crossing rate for the z score of the power spectral density acoustic features were not significantly associated with anxiety. Linguistic features, including negative-emotion words, were also associated with anxiety (r=0.10; P<.001). In addition, some linguistic relationships were sex dependent. For example, the count of words related to power was positively associated with anxiety in women (r=0.07; P=.03), whereas it was negatively associated with anxiety in men (r=-0.09; P=.01).

CONCLUSIONS

Both acoustic and linguistic speech measures are associated with anxiety scores. The amount of speech, acoustic quality of speech, and gender-specific linguistic characteristics of speech may be useful as part of a system to screen for anxiety, detect relapse, or monitor treatment.

UVM KID Study: Identifying Multimodal Features and Optimizing Wearable Instrumentation to Detect Child Anxiety

Anxiety and depression, collectively known as internalizing disorders, begin as early as the preschool years and impact nearly 1 out of every 5 children. Left undiagnosed and untreated, childhood internalizing disorders predict later health problems including substance abuse, development of comorbid psychopathology, increased risk for suicide, and substantial functional impairment. Current diagnostic procedures require access to clinical experts, take considerable time to complete, and inherently assume that child symptoms are observable by caregivers. Multi-modal wearable sensors may enable development of rapid point-of-care diagnostics that address these challenges. Building on our prior work, here we present an assessment battery for the development of a digital phenotype for internalizing disorders in young children and an early feasibility case study of multi-modal wearable sensor data from two participants, one of whom has been clinically diagnosed with an internalizing disorder. Results lend support that sacral movement responses and R-R interval during a short stress-induction task may facilitate child diagnosis. Multi-modal sensors measuring movement and surface biopotentials of the chest and trapezius are also shown to have significant redundancy, introducing the potential for sensor optimization moving forward. Future work aims to further optimize sensor placement, signals, features, and assessments to enable deployment in clinical practice. Clinical Relevance- This work considers the development and optimization of technologies for improving the identification of children with internalizing disorders.

Polysomnographic identification of anxiety and depression using deep learning

Anxiety and depression are common psychiatric conditions associated with significant morbidity and healthcare costs. Sleep is an evolutionarily conserved health state. Anxiety and depression have a bidirectional relationship with sleep. This study reports on the use of analysis of polysomnographic data using deep learning methods to detect the presence of anxiety and depression. Polysomnography data on 940 patients performed at an academic sleep center during the 3-year period from 01/01/2016 to 12/31/2018 were identified for analysis. The data were divided into 3 subgroups: 205 patients with Anxiety/Depression, 349 patients with no Anxiety/Depression, and 386 patients with likely Anxiety/Depression. The first two subgroups were used for training and testing of the deep learning algorithm, and the third subgroup was used for external validation of the resulting model. Hypnograms were constructed via automatic sleep staging, with the 12-channel PSG data being transformed into three-channel RGB (red, green, blue channels) images for analysis. Composite patient images were generated and utilized for training the Xception model, which provided a validation set accuracy of 0.9782 on the ninth training epoch. In the independent test set, the model achieved a high accuracy (0.9688), precision (0.9533), recall (0.9630), and F1-score (0.9581). Classification performance of most other mainstream deep learning models was comparable. These findings suggest that machine learning techniques have the potential to accurately detect the presence of anxiety and depression from analysis of sleep study data. Further studies are needed to explore the utility of these techniques in the field of psychiatry.

Screening for Generalized Anxiety Disorder from Acoustic and Linguistic Features of Impromptu Speech: Prediction Model Evaluation Study (Preprint)

Background

Frequent interaction with mental health professionals is required to screen, diagnose, and track mental health disorders. However, high costs and insufficient access can make frequent interactions difficult. The ability to assess a mental health disorder passively and at frequent intervals could be a useful complement to the conventional treatment. It may be possible to passively assess clinical symptoms with high frequency by characterizing speech alterations collected using personal smartphones or other wearable devices. The association between speech features and mental health disorders can be leveraged as an objective screening tool.

Objective

This study aimed to evaluate the performance of a model that predicts the presence of generalized anxiety disorder (GAD) from acoustic and linguistic features of impromptu speech on a larger and more generalizable scale than prior studies did.

Methods

A total of 2000 participants were recruited, and they participated in a single web-based session. They completed the Generalized Anxiety Disorder-7 item scale assessment and provided an impromptu speech sample in response to a modified version of the Trier Social Stress Test. We used the linguistic and acoustic features that were found to be associated with anxiety disorders in previous studies along with demographic information to predict whether participants fell above or below the screening threshold for GAD based on the Generalized Anxiety Disorder-7 item scale threshold of 10. Separate models for each sex were also evaluated. We reported the mean area under the receiver operating characteristic (AUROC) from a repeated 5-fold cross-validation to evaluate the performance of the models.

Results

A logistic regression model using only acoustic and linguistic speech features achieved a significantly greater prediction accuracy than a random model did (mean AUROC 0.57, SD 0.03; P<.001). When separately assessing samples from female participants, we observed a mean AUROC of 0.55 (SD 0.05; P=.01). The model constructed from the samples from male participants achieved a mean AUROC of 0.57 (SD 0.07; P=.002). The mean AUROC increased to 0.62 (SD 0.03; P<.001) on the all-sample data set when demographic information (age, sex, and income) was included, indicating the importance of demographics when screening for anxiety disorders. The performance also increased for the female sample to a mean of 0.62 (SD 0.04; P<.001) when using demographic information (age and income). An increase in performance was not observed when demographic information was added to the model constructed from the male samples.

Conclusions

A logistic regression model using acoustic and linguistic speech features, which have been suggested to be associated with anxiety disorders in prior studies, can achieve above-random accuracy for predicting GAD. Importantly, the addition of basic demographic variables further improves model performance, suggesting a role for speech and demographic information to be used as automated, objective screeners of GAD.

An insight into diagnosis of depression using machine learning techniques: a systematic review

BACKGROUND

In this modern era, depression is one of the most prevalent mental disorders from which millions of individuals are affected today. The symptoms of depression are heterogeneous and often coincide with other disorders such as bipolar disorder, Parkinson's, schizophrenia, etc. It is a serious mental illness that may lead to other health problems if left untreated. Currently, identifying individuals with depression is totally based on the expertise of the clinician's experience. In order to assist clinicians in identifying the characteristics and classifying depressed people, different types of data modalities and machine learning techniques have been incorporated by researchers in this field. This study aims to find the answers to some important questions related to the trend of publications, data modality, machine learning models, dataset usage, pre-processing techniques and feature extraction and selection techniques that are prevalent and guide the direction of future research on depression diagnosis.

METHODS

This systematic review was conducted using a broad range of articles from two major databases: IEEE Xplore and PubMed. Studies ranging from the years 2011 to April 2021 were retrieved from the databases resulting in a total of 590 articles (53 articles from the IEEE Xplore database and 537 articles from the PubMed database). Out of those, the articles which satisfied the defined inclusion criteria were investigated for further analysis.

RESULTS

A total of 135 articles were identified and analysed for this review. High growth in the number of publications has been observed in recent years. Furthermore, significant diversity in the use of data modalities and machine learning classifiers has also been noted in this study. fMRI data with an SVM classifier was found to be the most popular choice among researchers. In most of the studies, data scarcity and small sample size, particularly for neuroimaging data are major concerns. The use of identical data pre-processing tools for similar data modalities can be seen. This study also provides statistical analysis of the current framework with respect to the modality, machine learning classifier, sample size and accuracy by applying one-way ANOVA and the Tukey - Kramer test.

CONCLUSION

The results indicate that an effective fusion of machine learning techniques with a potential data modality has a promising future for assisting clinicians in automatic depression diagnosis.

Monitoring the effects of therapeutic interventions in depression through self-assessments

The treatment of major psychiatric disorders is an arduous and thorny path for the patients concerned, characterized by polypharmacy, massive adverse side effects, modest prospects of success, and constantly declining response rates. The more important is the early detection of psychiatric disorders prior to the development of clinically relevant symptoms, so that people can benefit from early interventions. A well-proven approach to monitoring mental health relies on voice analysis. This method has been successfully used with psychiatric patients to 'objectively' document the progress of improvement or the onset of relapse. The studies with psychiatric patients over 2-4 weeks demonstrated that daily voice assessments have a notable therapeutic effect in themselves. Therefore, daily voice assessments appear to be a lowthreshold form of therapeutic means that may be realized through self-assessments. To evaluate performance and reliability of this approach, we have carried out a longitudinal study on 82 university students in 3 different countries with daily assessments over 2 weeks. The sample included 41 males (mean age 24.2±3.83 years) and 41 females (mean age 21.6±2.05 years). Unlike other research in the field, this study was not concerned with the classification of individuals in terms of diagnostic categories. The focus lay on the monitoring aspect and the extent to which the effects of therapeutic interventions or of behavioural changes are visible in the results of self-assessment voice analyses. The test persons showed an over-proportionally good adherence to the daily voice analysis scheme. The accumulated data were of generally high quality: sufficiently high signal levels, a very limited number of movement artifacts, and little to no interfering background noise. The method was sufficiently sensitive to detect: i) habituation effects when test persons became used to the daily procedure; and ii) short-term fluctuations that exceeded prespecified thresholds and reached significance. Results are directly interpretable and provide information about what is going well, what is going less well, and where there is a need for action. The proposed self-assessment approach was found to be well-suited to serve as a health-monitoring tool for subjects with an elevated vulnerability to psychiatric disorders or to stress-induced mental health problems. Daily voice assessments are in fact a low-threshold form of therapeutic means that can be realized through selfassessments, that requires only little effort, can be carried out in the test person's own home, and has the potential to strengthen resilience and to induce positive behavioural changes.

Classification of Depression and Other Psychiatric Conditions Using Speech Features Extracted from a Thai Psychiatric and Verbal Screening Test

Depression is a common and serious mental illness which negatively affects daily functioning. To prevent the progression of the illness into severe or long-term consequences, early diagnosis is crucial. We developed an automated speech feature analysis application for depression and other psychiatric disorders derived from a developed Thai psychiatric and verbal screening test. The screening test includes Thai's version of Patient Health Questionnaire-9 (PHQ-9) and Hamilton Depression Rating Scale (HAM-D), and 32 additional emotion-induced questions. Case-control study was conducted on speech features from 66 participants. Twenty seven of those had depression (DP), 12 had other psychiatric disorders (OP), and 27 were normal controls (NC). The five-fold cross-validation from 6 settings of 5 classifiers with the combination of PHQ-9 and HAM-D scores, and speech features were examined. Results showed highest performance from the multilayer perceptron (MLP) classifier which yielded 83.33% sensitivity, 91.67% specificity, and 83.33% accuracy, where negative-emotional questions were most effective in classification. The automated speech feature analysis showed promising results for screening patients with depression or other psychiatric disorders. The current application is accessible through smartphone, making it a feasible and intuitive setup for low-resource countries such as Thailand.

Machine Learning-Based Behavioral Diagnostic Tools for Depression: Advances, Challenges, and Future Directions

The psychiatric diagnostic procedure is currently based on self-reports that are subject to personal biases. Therefore, the diagnostic process would benefit greatly from data-driven tools that can enhance accuracy and specificity. In recent years, many studies have achieved promising results in detecting and diagnosing depression based on machine learning (ML) analysis. Despite these favorable results in depression diagnosis, which are primarily based on ML analysis of neuroimaging data, most patients do not have access to neuroimaging tools. Hence, objective assessment tools are needed that can be easily integrated into the routine psychiatric diagnostic process. One solution is to use behavioral data, which can be easily collected while still maintaining objectivity. The current paper summarizes the main ML-based approaches that use behavioral data in diagnosing depression and other psychiatric disorders. We classified these studies into two main categories: (a) laboratory-based assessments and (b) data mining, the latter of which we further divided into two sub-groups: (i) social media usage and movement sensors data and (ii) demographic and clinical information. The paper discusses the advantages and challenges in this field and suggests future research directions and implementations. The paper's overarching aim is to serve as a first step in synthetizing existing knowledge about ML-based behavioral diagnosis studies in order to develop interventions and individually tailored treatments in the future.

Digital Phenotype for Childhood Internalizing Disorders: Less Positive Play and Promise for a Brief Assessment Battery

Childhood internalizing disorders, like anxiety and depression, are common, impairing, and difficult to detect. Universal childhood mental health screening has been recommended, but new technologies are needed to provide objective detection. Instrumented mood induction tasks, designed to press children for specific behavioral responses, have emerged as means for detecting childhood internalizing psychopathology. In our previous work, we leveraged machine learning to identify digital phenotypes of childhood internalizing psychopathology from movement and voice data collected during negative valence tasks (pressing for anxiety and fear). In this work, we develop a digital phenotype for childhood internalizing disorders based on wearable inertial sensor data recorded from a Positive Valence task during which a child plays with bubbles. We find that a phenotype derived from features that capture reward responsiveness is able to accurately detect children with underlying internalizing psychopathology (AUC = 0.81). In so doing, we explore the impact of a variety of feature sets computed from wearable sensors deployed to two body locations on phenotype performance across two phases of the task. We further consider this novel digital phenotype in the context of our previous Negative Valence digital phenotypes and find that each task brings unique information to the problem of detecting childhood internalizing psychopathology, capturing different problems and disorder subtypes. Collectively, these results provide preliminary evidence for a mood induction task battery to develop a novel diagnostic for childhood internalizing disorders.

A Convenient and Low-Cost Model of Depression Screening and Early Warning Based on Voice Data Using for Public Mental Health

Depression is a common mental health disease, which has great harm to public health. At present, the diagnosis of depression mainly depends on the interviews between doctors and patients, which is subjective, slow and expensive. Voice data are a kind of data that are easy to obtain and have the advantage of low cost. It has been proved that it can be used in the diagnosis of depression. The voice data used for modeling in this study adopted the authoritative public data set, which had passed the ethical review. The features of voice data were extracted by Python programming, and the voice features were stored in the format of CSV files. Through data processing, a big database, containing 1479 voice feature samples, was generated for modeling. Then, the decision tree screening model of depression was established by 10-fold cross validation and algorithm selection. The experiment achieved 83.4% prediction accuracy on voice data set. According to the prediction results of the model, the patients can be given early warning and intervention in time, so as to realize the health management of personal depression.