Selection of OSA-specific pronunciations and assessment of disease severity assisted by machine learning

STUDY OBJECTIVES

To screen all of the obstructive sleep apnea (OSA)-characteristic pronunciations, explore the pronunciations which provide a better OSA classification effect than those used previously, and further clarify the correlation between speech signals and OSA.

METHODS

A total of 158 adult male Mandarin native speakers who completed polysomnography at the Sleep Medicine Center of Beijing Tongren Hospital from November 15, 2019, to January 19, 2020, were enrolled in this study. All Chinese syllables were collected from each participant in the sitting position. The syllables, vowels, consonants, and tones were screened to identify the pronunciations that were most effective for OSA classification.

RESULTS

The linear prediction coefficients of Chinese syllables were extracted as features and mathematically modeled using a decision tree model to dichotomize participants with apnea-hypopnea index thresholds of 10 and 30 events/h, and the leave-one-out method was used to verify the classification performance of Chinese syllables for OSA. Chinese syllables such as [leng] and [jue], consonant pronunciations such as [zh] and [f], and vowel pronunciations such as [ing] and [ai] were the most suitable pronunciations for classification of OSA. An OSA classification model consisting of several syllable combinations was constructed, with areas under curve of 0.83 (threshold of apnea-hypopnea index = 10) and 0.87 (threshold of apnea-hypopnea index = 30), respectively.

CONCLUSIONS

This study is the first comprehensive screening of OSA-characteristic pronunciations and can act as a guideline for the construction of OSA speech corpora in other languages.

CITATION

Ding Y, Sun Y, Li Y, et al. Selection of OSA-specific pronunciations and assessment of disease severity assisted by machine learning. J Clin Sleep Med. 2022;18(11):2663-2672.

Artificial Intelligence for Detecting COVID-19 With the Aid of Human Cough, Breathing and Speech Signals: Scoping Review

Goal: Official tests for COVID-19 are time consuming, costly, can produce high false negatives, use up vital chemicals and may violate social distancing laws. Therefore, a fast and reliable additional solution using recordings of cough, breathing and speech data for preliminary screening may help alleviate these issues. Objective: This scoping review explores how Artificial Intelligence (AI) technology aims to detect COVID-19 disease by using cough, breathing and speech recordings, as reported in the literature. Here, we describe and summarize attributes of the identified AI techniques and datasets used for their implementation. Methods: A scoping review was conducted following the guidelines of PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews). Electronic databases (Google Scholar, Science Direct, and IEEE Xplore) were searched between 1st April 2020 and 15th August 2021. Terms were selected based on the target intervention (i.e., AI), the target disease (i.e., COVID-19) and acoustic correlates of the disease (i.e., speech, breathing and cough). A narrative approach was used to summarize the extracted data. Results: 24 studies and 8 Apps out of the 86 retrieved studies met the inclusion criteria. Half of the publications and Apps were from the USA. The most prominent AI architecture used was a convolutional neural network, followed by a recurrent neural network. AI models were mainly trained, tested and run-on websites and personal computers, rather than on phone apps. More than half of the included studies reported area-under-the-curve performance of greater than 0.90 on symptomatic and negative datasets while one study achieved 100% sensitivity in predicting asymptomatic COVID-19 from cough-, breathing- or speech-based acoustic features. Conclusions: The included studies show that AI has the potential to help detect COVID-19 using cough, breathing and speech samples. The proposed methods (with some time and appropriate clinical testing) could prove to be an effective method in detecting various diseases related to respiratory and neurophysiological changes in the human body.

Age and Gender Recognition Using a Convolutional Neural Network with a Specially Designed Multi-Attention Module through Speech Spectrograms

Speech signals are being used as a primary input source in human-computer interaction (HCI) to develop several applications, such as automatic speech recognition (ASR), speech emotion recognition (SER), gender, and age recognition. Classifying speakers according to their age and gender is a challenging task in speech processing owing to the disability of the current methods of extracting salient high-level speech features and classification models. To address these problems, we introduce a novel end-to-end age and gender recognition convolutional neural network (CNN) with a specially designed multi-attention module (MAM) from speech signals. Our proposed model uses MAM to extract spatial and temporal salient features from the input data effectively. The MAM mechanism uses a rectangular shape filter as a kernel in convolution layers and comprises two separate time and frequency attention mechanisms. The time attention branch learns to detect temporal cues, whereas the frequency attention module extracts the most relevant features to the target by focusing on the spatial frequency features. The combination of the two extracted spatial and temporal features complements one another and provide high performance in terms of age and gender classification. The proposed age and gender classification system was tested using the Common Voice and locally developed Korean speech recognition datasets. Our suggested model achieved 96%, 73%, and 76% accuracy scores for gender, age, and age-gender classification, respectively, using the Common Voice dataset. The Korean speech recognition dataset results were 97%, 97%, and 90% for gender, age, and age-gender recognition, respectively. The prediction performance of our proposed model, which was obtained in the experiments, demonstrated the superiority and robustness of the tasks regarding age, gender, and age-gender recognition from speech signals.

Personal Resilience Can Be Well Estimated from Heart Rate Variability and Paralinguistic Features during Human-Robot Conversations

Mental health is as crucial as physical health, but it is underappreciated by mainstream biomedical research and the public. Compared to the use of AI or robots in physical healthcare, the use of AI or robots in mental healthcare is much more limited in number and scope. To date, psychological resilience—the ability to cope with a crisis and quickly return to the pre-crisis state—has been identified as an important predictor of psychological well-being but has not been commonly considered by AI systems (e.g., smart wearable devices) or social robots to personalize services such as emotion coaching. To address the dearth of investigations, the present study explores the possibility of estimating personal resilience using physiological and speech signals measured during human–robot conversations. Specifically, the physiological and speech signals of 32 research participants were recorded while the participants answered a humanoid social robot’s questions about their positive and negative memories about three periods of their lives. The results from machine learning models showed that heart rate variability and paralinguistic features were the overall best predictors of personal resilience. Such predictability of personal resilience can be leveraged by AI and social robots to improve user understanding and has great potential for various mental healthcare applications in the future.