The COUGHVID crowdsourcing dataset, a corpus for the study of large-scale cough analysis algorithms

Artificial intelligence-based framework to identify the abnormalities in the COVID-19 disease and other common respiratory diseases from digital stethoscope data using deep CNN

The utilization of lung sounds to diagnose lung diseases using respiratory sound features has significantly increased in the past few years. The Digital Stethoscope data has been examined extensively by medical researchers and technical scientists to diagnose the symptoms of respiratory diseases. Artificial intelligence-based approaches are applied in the real universe to distinguish respiratory disease signs from human pulmonary auscultation sounds. The Deep CNN model is implemented with combined multi-feature channels (Modified MFCC, Log Mel, and Soft Mel) to obtain the sound parameters from lung-based Digital Stethoscope data. The model analysis is observed with max-pooling and without max-pool operations using multi-feature channels on respiratory digital stethoscope data. In addition, COVID-19 sound data and enriched data, which are recently acquired data to enhance model performance using a combination of L2 regularization to overcome the risk of overfitting because of less respiratory sound data, are included in the work. The suggested DCNN with Max-Pooling on the improved dataset demonstrates cutting-edge performance employing a multi-feature channels spectrogram. The model has been developed with different convolutional filter sizes (1 × 12 , 1 × 24 , 1 × 36 , 1 × 48 , and 1 × 60 ) that helped to test the proposed neural network. According to the experimental findings, the suggested DCNN architecture with a max-pooling function performs better to identify respiratory disease symptoms than DCNN without max-pooling. In order to demonstrate the model's effectiveness in categorization, it is trained and tested with the DCNN model that extract several modalities of respiratory sound data.

Respiratory Diseases Diagnosis Using Audio Analysis and Artificial Intelligence: A Systematic Review

Respiratory diseases represent a significant global burden, necessitating efficient diagnostic methods for timely intervention. Digital biomarkers based on audio, acoustics, and sound from the upper and lower respiratory system, as well as the voice, have emerged as valuable indicators of respiratory functionality. Recent advancements in machine learning (ML) algorithms offer promising avenues for the identification and diagnosis of respiratory diseases through the analysis and processing of such audio-based biomarkers. An ever-increasing number of studies employ ML techniques to extract meaningful information from audio biomarkers. Beyond disease identification, these studies explore diverse aspects such as the recognition of cough sounds amidst environmental noise, the analysis of respiratory sounds to detect respiratory symptoms like wheezes and crackles, as well as the analysis of the voice/speech for the evaluation of human voice abnormalities. To provide a more in-depth analysis, this review examines 75 relevant audio analysis studies across three distinct areas of concern based on respiratory diseases' symptoms: (a) cough detection, (b) lower respiratory symptoms identification, and (c) diagnostics from the voice and speech. Furthermore, publicly available datasets commonly utilized in this domain are presented. It is observed that research trends are influenced by the pandemic, with a surge in studies on COVID-19 diagnosis, mobile data acquisition, and remote diagnosis systems.

Limitations of the Cough Sound-Based COVID-19 Diagnosis Artificial Intelligence Model and its Future Direction: Longitudinal Observation Study

BACKGROUND

The outbreak of SARS-CoV-2 in 2019 has necessitated the rapid and accurate detection of COVID-19 to manage patients effectively and implement public health measures. Artificial intelligence (AI) models analyzing cough sounds have emerged as promising tools for large-scale screening and early identification of potential cases.

OBJECTIVE

This study aimed to investigate the efficacy of using cough sounds as a diagnostic tool for COVID-19, considering the unique acoustic features that differentiate positive and negative cases. We investigated whether an AI model trained on cough sound recordings from specific periods, especially the early stages of the COVID-19 pandemic, were applicable to the ongoing situation with persistent variants.

METHODS

We used cough sound recordings from 3 data sets (Cambridge, Coswara, and Virufy) representing different stages of the pandemic and variants. Our AI model was trained using the Cambridge data set with subsequent evaluation against all data sets. The performance was analyzed based on the area under the receiver operating curve (AUC) across different data measurement periods and COVID-19 variants.

RESULTS

The AI model demonstrated a high AUC when tested with the Cambridge data set, indicative of its initial effectiveness. However, the performance varied significantly with other data sets, particularly in detecting later variants such as Delta and Omicron, with a marked decline in AUC observed for the latter. These results highlight the challenges in maintaining the efficacy of AI models against the backdrop of an evolving virus.

CONCLUSIONS

While AI models analyzing cough sounds offer a promising noninvasive and rapid screening method for COVID-19, their effectiveness is challenged by the emergence of new virus variants. Ongoing research and adaptations in AI methodologies are crucial to address these limitations. The adaptability of AI models to evolve with the virus underscores their potential as a foundational technology for not only the current pandemic but also future outbreaks, contributing to a more agile and resilient global health infrastructure.

An AI-Enabled Bias-Free Respiratory Disease Diagnosis Model Using Cough Audio

Cough-based diagnosis for respiratory diseases (RDs) using artificial intelligence (AI) has attracted considerable attention, yet many existing studies overlook confounding variables in their predictive models. These variables can distort the relationship between cough recordings (input data) and RD status (output variable), leading to biased associations and unrealistic model performance. To address this gap, we propose the Bias-Free Network (RBF-Net), an end-to-end solution that effectively mitigates the impact of confounders in the training data distribution. RBF-Net ensures accurate and unbiased RD diagnosis features, emphasizing its relevance by incorporating a COVID-19 dataset in this study. This approach aims to enhance the reliability of AI-based RD diagnosis models by navigating the challenges posed by confounding variables. A hybrid of a Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks is proposed for the feature encoder module of RBF-Net. An additional bias predictor is incorporated in the classification scheme to formulate a conditional Generative Adversarial Network (c-GAN) that helps in decorrelating the impact of confounding variables from RD prediction. The merit of RBF-Net is demonstrated by comparing classification performance with a State-of-The-Art (SoTA) Deep Learning (DL) model (CNN-LSTM) after training on different unbalanced COVID-19 data sets, created by using a large-scale proprietary cough data set. RBF-Net proved its robustness against extremely biased training scenarios by achieving test set accuracies of 84.1%, 84.6%, and 80.5% for the following confounding variables-gender, age, and smoking status, respectively. RBF-Net outperforms the CNN-LSTM model test set accuracies by 5.5%, 7.7%, and 8.2%, respectively.

The smarty4covid dataset and knowledge base as a framework for interpretable physiological audio data analysis

Harnessing the power of Artificial Intelligence (AI) and m-health towards detecting new bio-markers indicative of the onset and progress of respiratory abnormalities/conditions has greatly attracted the scientific and research interest especially during COVID-19 pandemic. The smarty4covid dataset contains audio signals of cough (4,676), regular breathing (4,665), deep breathing (4,695) and voice (4,291) as recorded by means of mobile devices following a crowd-sourcing approach. Other self reported information is also included (e.g. COVID-19 virus tests), thus providing a comprehensive dataset for the development of COVID-19 risk detection models. The smarty4covid dataset is released in the form of a web-ontology language (OWL) knowledge base enabling data consolidation from other relevant datasets, complex queries and reasoning. It has been utilized towards the development of models able to: (i) extract clinically informative respiratory indicators from regular breathing records, and (ii) identify cough, breath and voice segments in crowd-sourced audio recordings. A new framework utilizing the smarty4covid OWL knowledge base towards generating counterfactual explanations in opaque AI-based COVID-19 risk detection models is proposed and validated.

A semi-supervised algorithm for improving the consistency of crowdsourced datasets: The COVID-19 case study on respiratory disorder classification

BACKGROUND AND OBJECTIVE

Cough audio signal classification is a potentially useful tool in screening for respiratory disorders, such as COVID-19. Since it is dangerous to collect data from patients with contagious diseases, many research teams have turned to crowdsourcing to quickly gather cough sound data. The COUGHVID dataset enlisted expert physicians to annotate and diagnose the underlying diseases present in a limited number of recordings. However, this approach suffers from potential cough mislabeling, as well as disagreement between experts.

METHODS

In this work, we use a semi-supervised learning (SSL) approach - based on audio signal processing tools and interpretable machine learning models - to improve the labeling consistency of the COUGHVID dataset for 1) COVID-19 versus healthy cough sound classification 2) distinguishing wet from dry coughs, and 3) assessing cough severity. First, we leverage SSL expert knowledge aggregation techniques to overcome the labeling inconsistencies and label sparsity in the dataset. Next, our SSL approach is used to identify a subsample of re-labeled COUGHVID audio samples that can be used to train or augment future cough classifiers.

RESULTS

The consistency of the re-labeled COVID-19 and healthy data is demonstrated in that it exhibits a high degree of inter-class feature separability: 3x higher than that of the user-labeled data. Similarly, the SSL method increases this separability by 11.3x for cough type and 5.1x for severity classifications. Furthermore, the spectral differences in the user-labeled audio segments are amplified in the re-labeled data, resulting in significantly different power spectral densities between healthy and COVID-19 coughs in the 1-1.5 kHz range (p=1.2×10-64), which demonstrates both the increased consistency of the new dataset and its explainability from an acoustic perspective. Finally, we demonstrate how the re-labeled dataset can be used to train a COVID-19 classifier, achieving an AUC of 0.797.

CONCLUSIONS

We propose a SSL expert knowledge aggregation technique for the field of cough sound classification for the first time, and demonstrate how it can be used to combine the medical knowledge of multiple experts in an explainable fashion, thus providing abundant, consistent data for cough classification tasks.

CovidCoughNet: A new method based on convolutional neural networks and deep feature extraction using pitch-shifting data augmentation for covid-19 detection from cough, breath, and voice signals

This study proposes a new deep learning-based method that demonstrates high performance in detecting Covid-19 disease from cough, breath, and voice signals. This impressive method, named CovidCoughNet, consists of a deep feature extraction network (InceptionFireNet) and a prediction network (DeepConvNet). The InceptionFireNet architecture, based on Inception and Fire modules, was designed to extract important feature maps. The DeepConvNet architecture, which is made up of convolutional neural network blocks, was developed to predict the feature vectors obtained from the InceptionFireNet architecture. The COUGHVID dataset containing cough data and the Coswara dataset containing cough, breath, and voice signals were used as the data sets. The pitch-shifting technique was used to data augmentation the signal data, which significantly contributed to improving performance. Additionally, Chroma features (CF), Root mean square energy (RMSE), Spectral centroid (SC), Spectral bandwidth (SB), Spectral rolloff (SR), Zero crossing rate (ZCR), and Mel frequency cepstral coefficients (MFCC) feature extraction techniques were used to extract important features from voice signals. Experimental studies have shown that using the pitch-shifting technique improved performance by around 3% compared to raw signals. When the proposed model was used with the COUGHVID dataset (Healthy, Covid-19, and Symptomatic), a high performance of 99.19% accuracy, 0.99 precision, 0.98 recall, 0.98 F1-Score, 97.77% specificity, and 98.44% AUC was achieved. Similarly, when the voice data in the Coswara dataset was used, higher performance was achieved compared to the cough and breath studies, with 99.63% accuracy, 100% precision, 0.99 recall, 0.99 F1-Score, 99.24% specificity, and 99.24% AUC. Moreover, when compared with current studies in the literature, the proposed model was observed to exhibit highly successful performance. The codes and details of the experimental studies can be accessed from the relevant Github page: (https://github.com/GaffariCelik/CovidCoughNet).

Automatic Detection of Dyspnea in Real Human-Robot Interaction Scenarios

A respiratory distress estimation technique for telephony previously proposed by the authors is adapted and evaluated in real static and dynamic HRI scenarios. The system is evaluated with a telephone dataset re-recorded using the robotic platform designed and implemented for this study. In addition, the original telephone training data are modified using an environmental model that incorporates natural robot-generated and external noise sources and reverberant effects using room impulse responses (RIRs). The results indicate that the average accuracy and AUC are just 0.4% less than those obtained with matched training/testing conditions with simulated data. Quite surprisingly, there is not much difference in accuracy and AUC between static and dynamic HRI conditions. Moreover, the beamforming methods delay-and-sum and MVDR lead to average improvement in accuracy and AUC equal to 8% and 2%, respectively, when applied to training and testing data. Regarding the complementarity of time-dependent and time-independent features, the combination of both types of classifiers provides the best joint accuracy and AUC score.

A Multimodal Dataset for Automatic Edge-AI Cough Detection

Counting the number of times a patient coughs per day is an essential biomarker in determining treatment efficacy for novel antitussive therapies and personalizing patient care. Automatic cough counting tools must provide accurate information, while running on a lightweight, portable device that protects the patient's privacy. Several devices and algorithms have been developed for cough counting, but many use only error-prone audio signals, rely on offline processing that compromises data privacy, or utilize processing and memory-intensive neural networks that require more hardware resources than can fit on a wearable device. Therefore, there is a need for wearable devices that employ multimodal sensors to perform accurate, privacy-preserving, automatic cough counting algorithms directly on the device in an edge Artificial Intelligence (edge-AI) fashion. To advance this research field, we contribute the first publicly accessible cough counting dataset of multimodal biosignals. The database contains nearly 4 hours of biosignal data, with both acoustic and kinematic modalities, covering 4,300 annotated cough events from 15 subjects. Furthermore, a variety of non-cough sounds and motion scenarios mimicking daily life activities are also present, which the research community can use to accelerate machine learning (ML) algorithm development. A technical validation of the dataset reveals that it represents a wide variety of signal-to-noise ratios, which can be expected in a real-life use case, as well as consistency across experimental trials. Finally, to demonstrate the usability of the dataset, we train a simple cough vs non-cough signal classifier that obtains a 91% sensitivity, 92% specificity, and 80% precision on unseen test subject data. Such edge-friendly AI algorithms have the potential to provide continuous ambulatory monitoring of the numerous chronic cough patients.

Adolescent Asthma Monitoring: A Preliminary Study of Audio and Spirometry Modalities

Asthma patients' sleep quality is correlated with how well their asthma symptoms are controlled. In this paper, deep learning techniques are explored to improve forecasting of forced expiratory volume in one second (FEV1) by using audio data from participants and test whether auditory sleep disturbances are correlated with poorer asthma outcomes. These are applied to a representative data set of FEV1 collected from a commercially available sprirometer and audio spectrograms collected overnight using a smartphone. A model for detecting nonverbal vocalizations including coughs, sneezes, sighs, snoring, throat clearing, sniffs, and breathing sounds was trained and used to capture nightly sleep disturbances. Our preliminary analysis found significant improvement in FEV1 forecasting when using overnight nonverbal vocalization detections as an additional feature for regression using XGBoost over using only spirometry data.Clinical relevance- This preliminary study establishes up to 30% improvement of FEV1 forecasting using features generated by deep learning techniques over only spirometry-based features.

Coswara: A respiratory sounds and symptoms dataset for remote screening of SARS-CoV-2 infection

This paper presents the Coswara dataset, a dataset containing diverse set of respiratory sounds and rich meta-data, recorded between April-2020 and February-2022 from 2635 individuals (1819 SARS-CoV-2 negative, 674 positive, and 142 recovered subjects). The respiratory sounds contained nine sound categories associated with variants of breathing, cough and speech. The rich metadata contained demographic information associated with age, gender and geographic location, as well as the health information relating to the symptoms, pre-existing respiratory ailments, comorbidity and SARS-CoV-2 test status. Our study is the first of its kind to manually annotate the audio quality of the entire dataset (amounting to 65 hours) through manual listening. The paper summarizes the data collection procedure, demographic, symptoms and audio data information. A COVID-19 classifier based on bi-directional long short-term (BLSTM) architecture, is trained and evaluated on the different population sub-groups contained in the dataset to understand the bias/fairness of the model. This enabled the analysis of the impact of gender, geographic location, date of recording, and language proficiency on the COVID-19 detection performance.

COVID-19 Detection from Cough Recordings Using Bag-of-Words Classifiers

Reliable detection of COVID-19 from cough recordings is evaluated using bag-of-words classifiers. The effect of using four distinct feature extraction procedures and four different encoding strategies is evaluated in terms of the Area Under Curve (AUC), accuracy, sensitivity, and F1-score. Additional studies include assessing the effect of both input and output fusion approaches and a comparative analysis against 2D solutions using Convolutional Neural Networks. Extensive experiments conducted on the COUGHVID and COVID-19 Sounds datasets indicate that sparse encoding yields the best performances, showing robustness against various combinations of feature type, encoding strategy, and codebook dimension parameters.

The voice of COVID-19: Breath and cough recording classification with temporal decision trees and random forests

Symbolic learning is the logic-based approach to machine learning, and its mission is to provide algorithms and methodologies to extract logical information from data and express it in an interpretable way. Interval temporal logic has been recently proposed as a suitable tool for symbolic learning, specifically via the design of an interval temporal logic decision tree extraction algorithm. In order to improve their performances, interval temporal decision trees can be embedded into interval temporal random forests, mimicking the corresponding schema at the propositional level. In this article we consider a dataset of cough and breath sample recordings of volunteer subjects, labeled with their COVID-19 status, originally collected by the University of Cambridge. By interpreting such recordings as multivariate time series, we study the problem of their automated classification using interval temporal decision trees and forests. While this problem has been approached with the same dataset as well as with other datasets, in all cases, non-symbolic learning methods (usually, deep learning-based) have been applied to solve it; in this article we apply a symbolic approach, and show that it does not only outperform the state-of-the-art obtained with the same dataset, but its results are also superior to those of most non-symbolic techniques applied on other datasets. As an added bonus, thanks to the symbolic nature of our approach, we are also able to extract explicit knowledge to help physicians characterize typical COVID-positive cough and breath.

Fruit-CoV: An efficient vision-based framework for speedy detection and diagnosis of SARS-CoV-2 infections through recorded cough sounds

COVID-19 is an infectious disease caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). This deadly virus has spread worldwide, leading to a global pandemic since March 2020. A recent variant of SARS-CoV-2 named Delta is intractably contagious and responsible for more than four million deaths globally. Therefore, developing an efficient self-testing service for SARS-CoV-2 at home is vital. In this study, a two-stage vision-based framework, namely Fruit-CoV, is introduced for detecting SARS-CoV-2 infections through recorded cough sounds. Specifically, audio signals are converted into Log-Mel spectrograms, and the EfficientNet-V2 network is used to extract their visual features in the first stage. In the second stage, 14 convolutional layers extracted from the large-scale Pretrained Audio Neural Networks for audio pattern recognition (PANNs) and the Wavegram-Log-Mel-CNN are employed to aggregate feature representations of the Log-Mel spectrograms and the waveform. Finally, the combined features are used to train a binary classifier. In this study, a dataset provided by the AICovidVN 115M Challenge is employed for evaluation. It includes 7,371 recorded cough sounds collected throughout Vietnam, India, and Switzerland. Experimental results indicate that the proposed model achieves an Area Under the Receiver Operating Characteristic Curve (AUC) score of 92.8% and ranks first on the final leaderboard of the AICovidVN 115M Challenge. Our code is publicly available.

MSCCov19Net: multi-branch deep learning model for COVID-19 detection from cough sounds

Coronavirus has an impact on millions of lives and has been added to the important pandemics that continue to affect with its variants. Since it is transmitted through the respiratory tract, it has had significant effects on public health and social relations. Isolating people who are COVID positive can minimize the transmission, therefore several exams are proposed to detect the virus such as reverse transcription-polymerase chain reaction (RT-PCR), chest X-Ray, and computed tomography (CT). However, these methods suffer from either a low detection rate or high radiation dosage, along with being expensive. In this study, deep neural network-based model capable of detecting coronavirus from only coughing sound, which is fast, remotely operable and has no harmful side effects, has been proposed. The proposed multi-branch model takes M el Frequency Cepstral Coefficients (MFCC), S pectrogram, and C hromagram as inputs and is abbreviated as MSCCov19Net. The system is trained on publicly available crowdsourced datasets, and tested on two unseen (used only for testing) clinical and non-clinical datasets. Experimental outcomes represent that the proposed system outperforms the 6 popular deep learning architectures on four datasets by representing a better generalization ability. The proposed system has reached an accuracy of 61.5 % in Virufy and 90.4 % in NoCoCoDa for unseen test datasets.

Dyspnea Severity Assessment Based on Vocalization Behavior with Deep Learning on the Telephone

In this paper, a system to assess dyspnea with the mMRC scale, on the phone, via deep learning, is proposed. The method is based on modeling the spontaneous behavior of subjects while pronouncing controlled phonetization. These vocalizations were designed, or chosen, to deal with the stationary noise suppression of cellular handsets, to provoke different rates of exhaled air, and to stimulate different levels of fluency. Time-independent and time-dependent engineered features were proposed and selected, and a k-fold scheme with double validation was adopted to select the models with the greatest potential for generalization. Moreover, score fusion methods were also investigated to optimize the complementarity of the controlled phonetizations and features that were engineered and selected. The results reported here were obtained from 104 participants, where 34 corresponded to healthy individuals and 70 were patients with respiratory conditions. The subjects' vocalizations were recorded with a telephone call (i.e., with an IVR server). The system provided an accuracy of 59% (i.e., estimating the correct mMRC), a root mean square error equal to 0.98, false positive rate of 6%, false negative rate of 11%, and an area under the ROC curve equal to 0.97. Finally, a prototype was developed and implemented, with an ASR-based automatic segmentation scheme, to estimate dyspnea on line.

A Performance Study of CNN Architectures for the Autonomous Detection of COVID-19 Symptoms Using Cough and Breathing

Deep learning (DL) methods have the potential to be used for detecting COVID-19 symptoms. However, the rationale for which DL method to use and which symptoms to detect has not yet been explored. In this paper, we present the first performance study which compares various convolutional neural network (CNN) architectures for the autonomous preliminary COVID-19 detection of cough and/or breathing symptoms. We compare and analyze residual networks (ResNets), visual geometry Groups (VGGs), Alex neural networks (AlexNet), densely connected networks (DenseNet), squeeze neural networks (SqueezeNet), and COVID-19 identification ResNet (CIdeR) architectures to investigate their classification performance. We uniquely train and validate both unimodal and multimodal CNN architectures using the EPFL and Cambridge datasets. Performance comparison across all modes and datasets showed that the VGG19 and DenseNet-201 achieved the highest unimodal and multimodal classification performance. VGG19 and DensNet-201 had high F1 scores (0.94 and 0.92) for unimodal cough classification on the Cambridge dataset, compared to the next highest F1 score for ResNet (0.79), with comparable F1 scores to ResNet for the larger EPFL cough dataset. They also had consistently high accuracy, recall, and precision. For multimodal detection, VGG19 and DenseNet-201 had the highest F1 scores (0.91) compared to the other CNN structures (≤0.90), with VGG19 also having the highest accuracy and recall. Our investigation provides the foundation needed to select the appropriate deep CNN method to utilize for non-contact early COVID-19 detection.

Transfer learning for the efficient detection of COVID-19 from smartphone audio data

Disease detection from smartphone data represents an open research challenge in mobile health (m-health) systems. COVID-19 and its respiratory symptoms are an important case study in this area and their early detection is a potential real instrument to counteract the pandemic situation. The efficacy of this solution mainly depends on the performances of AI algorithms applied to the collected data and their possible implementation directly on the users' mobile devices. Considering these issues, and the limited amount of available data, in this paper we present the experimental evaluation of 3 different deep learning models, compared also with hand-crafted features, and of two main approaches of transfer learning in the considered scenario: both feature extraction and fine-tuning. Specifically, we considered VGGish, YAMNET, and L³-Net (including 12 different configurations) evaluated through user-independent experiments on 4 different datasets (13,447 samples in total). Results clearly show the advantages of L³-Net in all the experimental settings as it overcomes the other solutions by 12.3% in terms of Precision-Recall AUC as features extractor, and by 10% when the model is fine-tuned. Moreover, we note that to fine-tune only the fully-connected layers of the pre-trained models generally leads to worse performances, with an average drop of 6.6% with respect to feature extraction. Finally, we evaluate the memory footprints of the different models for their possible applications on commercial mobile devices.

Intelligent speech technologies for transcription, disease diagnosis, and medical equipment interactive control in smart hospitals: A review

The growing and aging of the world population have driven the shortage of medical resources in recent years, especially during the COVID-19 pandemic. Fortunately, the rapid development of robotics and artificial intelligence technologies help to adapt to the challenges in the healthcare field. Among them, intelligent speech technology (IST) has served doctors and patients to improve the efficiency of medical behavior and alleviate the medical burden. However, problems like noise interference in complex medical scenarios and pronunciation differences between patients and healthy people hamper the broad application of IST in hospitals. In recent years, technologies such as machine learning have developed rapidly in intelligent speech recognition, which is expected to solve these problems. This paper first introduces IST's procedure and system architecture and analyzes its application in medical scenarios. Secondly, we review existing IST applications in smart hospitals in detail, including electronic medical documentation, disease diagnosis and evaluation, and human-medical equipment interaction. In addition, we elaborate on an application case of IST in the early recognition, diagnosis, rehabilitation training, evaluation, and daily care of stroke patients. Finally, we discuss IST's limitations, challenges, and future directions in the medical field. Furthermore, we propose a novel medical voice analysis system architecture that employs active hardware, active software, and human-computer interaction to realize intelligent and evolvable speech recognition. This comprehensive review and the proposed architecture offer directions for future studies on IST and its applications in smart hospitals.

An evolutionary ensemble learning for diagnosing COVID-19 via cough signals

Objective The spread of the COVID-19 disease has caused great concern around the world and detecting the positive cases is crucial in curbing the pandemic. One of the symptoms of the disease is the dry cough it causes. It has previously been shown that cough signals can be used to identify a variety of diseases including tuberculosis, asthma, etc. In this paper, we proposed an algorithm to diagnose via cough signals the COVID-19 disease. Methods The proposed algorithm is an ensemble scheme that consists of a number of base learners, where each base learner uses a different feature extractor method, including statistical approaches and convolutional neural networks (CNN) for automatic feature extraction. Features are extracted from the raw signal and some transforms performed it, including Fourier, wavelet, Hilbert-Huang, and short-term Fourier transforms. The outputs of these base-learners are aggregated via a weighted voting scheme, with the weights optimised via an evolutionary paradigm. This paper also proposes a memetic algorithm for training the CNNs in the base-learners, which combines the speed of gradient descent (GD) algorithms and global search space coverage of the evolutionary algorithms. Results Experiments were performed on the proposed algorithm and different rival algorithms which included a number of CNN architectures in the literature and generic machine learning algorithms. The results suggested that the proposed algorithm achieves better performance compared to the existing algorithms in diagnosing COVID-19 via cough signals. Conclusion This research showed that COVID-19 could be diagnosed via cough signals and CNNs could be employed to process these signals and it may be further improved by the optimization of CNN architecture.

Validation of a small cough detector

Research question

The assessment of cough frequency in clinical practice relies predominantly on the patient's history. Currently, objective evaluation of cough is feasible with bulky equipment during a brief time (i.e. hours up to 1 day). Thus, monitoring of cough has been rarely performed outside clinical studies. We developed a small wearable cough detector (SIVA-P3) that uses deep neural networks for the automatic counting of coughs. This study examined the performance of the SIVA-P3 in an outpatient setting.

Methods

We recorded cough epochs with SIVA-P3 over eight consecutive days in patients suffering from chronic cough. During the first 24 h, the detector was validated against cough events counted by trained human listeners. The wearing comfort and the device usage were assessed using a questionnaire.

Results

In total, 27 participants (mean±sd age 50±14 years) with either chronic unexplained cough (n=12), COPD (n=4), asthma (n=5) or interstitial lung disease (n=6) were studied. During the daytime, the sensitivity of SIVA-P3 cough detection was 88.5±2.49% and the specificity was 99.97±0.01%. During the night-time, the sensitivity was 84.15±5.04% and the specificity was 99.97±0.02%. The wearing comfort and usage of the device was rated as very high by most participants.

Conclusion

SIVA-P3 enables automatic continuous cough monitoring in an outpatient setting for objective assessment of cough over days and weeks. It shows comparable sensitivity or higher sensitivity than other devices with fully automatic cough counting. Thanks to its wearing comfort and the high performance for cough detection, it has the potential for being used in routine clinical practice.

Systematic Review of Advanced AI Methods for Improving Healthcare Data Quality in Post COVID-19 Era

At the beginning of the COVID-19 pandemic, there was significant hype about the potential impact of artificial intelligence (AI) tools in combatting COVID-19 on diagnosis, prognosis, or surveillance. However, AI tools have not yet been widely successful. One of the key reason is the COVID-19 pandemic has demanded faster real-time development of AI-driven clinical and health support tools, including rapid data collection, algorithm development, validation, and deployment. However, there was not enough time for proper data quality control. Learning from the hard lessons in COVID-19, we summarize the important health data quality challenges during COVID-19 pandemic such as lack of data standardization, missing data, tabulation errors, and noise and artifact. Then we conduct a systematic investigation of computational methods that address these issues, including emerging novel advanced AI data quality control methods that achieve better data quality outcomes and, in some cases, simplify or automate the data cleaning process. We hope this article can assist healthcare community to improve health data quality going forward with novel AI development.

A summary of the ComParE COVID-19 challenges

The COVID-19 pandemic has caused massive humanitarian and economic damage. Teams of scientists from a broad range of disciplines have searched for methods to help governments and communities combat the disease. One avenue from the machine learning field which has been explored is the prospect of a digital mass test which can detect COVID-19 from infected individuals' respiratory sounds. We present a summary of the results from the INTERSPEECH 2021 Computational Paralinguistics Challenges: COVID-19 Cough, (CCS) and COVID-19 Speech, (CSS).

Reliability of crowdsourced data and patient-reported outcome measures in cough-based COVID-19 screening

Mass community testing is a critical means for monitoring the spread of the COVID-19 pandemic. Polymerase chain reaction (PCR) is the gold standard for detecting the causative coronavirus 2 (SARS-CoV-2) but the test is invasive, test centers may not be readily available, and the wait for laboratory results can take several days. Various machine learning based alternatives to PCR screening for SARS-CoV-2 have been proposed, including cough sound analysis. Cough classification models appear to be a robust means to predict infective status, but collecting reliable PCR confirmed data for their development is challenging and recent work using unverified crowdsourced data is seen as a viable alternative. In this study, we report experiments that assess cough classification models trained (i) using data from PCR-confirmed COVID subjects and (ii) using data of individuals self-reporting their infective status. We compare performance using PCR-confirmed data. Models trained on PCR-confirmed data perform better than those trained on patient-reported data. Models using PCR-confirmed data also exploit more stable predictive features and converge faster. Crowd-sourced cough data is less reliable than PCR-confirmed data for developing predictive models for COVID-19, and raises concerns about the utility of patient reported outcome data in developing other clinical predictive models when better gold-standard data are available.

Considerations and Challenges for Real-World Deployment of an Acoustic-Based COVID-19 Screening System

Coronavirus disease 2019 (COVID-19) has led to countless deaths and widespread global disruptions. Acoustic-based artificial intelligence (AI) tools could provide a simple, scalable, and prompt method to screen for COVID-19 using easily acquirable physiological sounds. These systems have been demonstrated previously and have shown promise but lack robust analysis of their deployment in real-world settings when faced with diverse recording equipment, noise environments, and test subjects. The primary aim of this work is to begin to understand the impacts of these real-world deployment challenges on the system performance. Using Mel-Frequency Cepstral Coefficients (MFCC) and RelAtive SpecTrAl-Perceptual Linear Prediction (RASTA-PLP) features extracted from cough, speech, and breathing sounds in a crowdsourced dataset, we present a baseline classification system that obtains an average receiver operating characteristic area under the curve (AUC-ROC) of 0.77 when discriminating between COVID-19 and non-COVID subjects. The classifier performance is then evaluated on four additional datasets, resulting in performance variations between 0.64 and 0.87 AUC-ROC, depending on the sound type. By analyzing subsets of the available recordings, it is noted that the system performance degrades with certain recording devices, noise contamination, and with symptom status. Furthermore, performance degrades when a uniform classification threshold from the training data is subsequently used across all datasets. However, the system performance is robust to confounding factors, such as gender, age group, and the presence of other respiratory conditions. Finally, when analyzing multiple speech recordings from the same subjects, the system achieves promising performance with an AUC-ROC of 0.78, though the classification does appear to be impacted by natural speech variations. Overall, the proposed system, and by extension other acoustic-based diagnostic aids in the literature, could provide comparable accuracy to rapid antigen testing but significant deployment challenges need to be understood and addressed prior to clinical use.

A survey of COVID-19 detection and prediction approaches using mobile devices, AI, and telemedicine

Since 2019, the COVID-19 pandemic has had an extremely high impact on all facets of the society and will potentially have an everlasting impact for years to come. In response to this, over the past years, there have been a significant number of research efforts on exploring approaches to combat COVID-19. In this paper, we present a survey of the current research efforts on using mobile Internet of Thing (IoT) devices, Artificial Intelligence (AI), and telemedicine for COVID-19 detection and prediction. We first present the background and then present current research in this field. Specifically, we present the research on COVID-19 monitoring and detection, contact tracing, machine learning based approaches, telemedicine, and security. We finally discuss the challenges and the future work that lay ahead in this field before concluding this paper.

Multi-modal approach for COVID-19 detection using coughs and self-reported symptoms

 COVID-19 (Coronavirus Disease of 2019) is one of the most challenging healthcare crises of the twenty-first century. The pandemic causes many negative impacts on all aspects of life and livelihoods. Although recent developments of relevant vaccines, such as Pfizer/BioNTech mRNA, AstraZeneca, or Moderna, the emergence of new virus mutations and their fast infection rate yet pose significant threats to public health. In this context, early detection of the disease is an important factor to reduce its effect and quickly control the spread of pandemic. Nevertheless, many countries still rely on methods that are either expensive and time-consuming (i.e., Reverse-transcription polymerase chain reaction) or uncomfortable and difficult for self-testing (i.e., Rapid Antigen Test Nasal). Recently, deep learning methods have been proposed as a potential solution for COVID-19 analysis. However, previous works usually focus on a single symptom, which can omit critical information for disease diagnosis. Therefore, in this study, we propose a multi-modal method to detect COVID-19 using cough sounds and self-reported symptoms. The proposed method consists of five neural networks to deal with different input features, including CNN-biLSTM for MFCC features, EfficientNetV2 for Mel spectrogram images, MLP for self-reported symptoms, C-YAMNet for cough detection, and RNNoise for noise-canceling. Experimental results demonstrated that our method outperformed the other state-of-the-art methods with a high AUC, accuracy, and F1-score of 98.6%, 96.9%, and 96.9% on the testing set.

Vocal biomarker predicts fatigue in people with COVID-19: results from the prospective Predi-COVID cohort study

OBJECTIVE

To develop a vocal biomarker for fatigue monitoring in people with COVID-19.

DESIGN

Prospective cohort study.

SETTING

Predi-COVID data between May 2020 and May 2021.

PARTICIPANTS

A total of 1772 voice recordings were used to train an AI-based algorithm to predict fatigue, stratified by gender and smartphone's operating system (Android/iOS). The recordings were collected from 296 participants tracked for 2 weeks following SARS-CoV-2 infection.

PRIMARY AND SECONDARY OUTCOME MEASURES

Four machine learning algorithms (logistic regression, k-nearest neighbours, support vector machine and soft voting classifier) were used to train and derive the fatigue vocal biomarker. The models were evaluated based on the following metrics: area under the curve (AUC), accuracy, F1-score, precision and recall. The Brier score was also used to evaluate the models' calibrations.

RESULTS

The final study population included 56% of women and had a mean (±SD) age of 40 (±13) years. Women were more likely to report fatigue (p<0.001). We developed four models for Android female, Android male, iOS female and iOS male users with a weighted AUC of 86%, 82%, 79%, 85% and a mean Brier Score of 0.15, 0.12, 0.17, 0.12, respectively. The vocal biomarker derived from the prediction models successfully discriminated COVID-19 participants with and without fatigue.

CONCLUSIONS

This study demonstrates the feasibility of identifying and remotely monitoring fatigue thanks to voice. Vocal biomarkers, digitally integrated into telemedicine technologies, are expected to improve the monitoring of people with COVID-19 or Long-COVID.

TRIAL REGISTRATION NUMBER

NCT04380987.

Exploring machine learning for audio-based respiratory condition screening: A concise review of databases, methods, and open issues

Auscultation plays an important role in the clinic, and the research community has been exploring machine learning (ML) to enable remote and automatic auscultation for respiratory condition screening via sounds. To give the big picture of what is going on in this field, in this narrative review, we describe publicly available audio databases that can be used for experiments, illustrate the developed ML methods proposed to date, and flag some under-considered issues which still need attention. Compared to existing surveys on the topic, we cover the latest literature, especially those audio-based COVID-19 detection studies which have gained extensive attention in the last two years. This work can help to facilitate the application of artificial intelligence in the respiratory auscultation field.

The Use of Audio Signals for Detecting COVID-19: A Systematic Review

A systematic review on the topic of automatic detection of COVID-19 using audio signals was performed. A total of 48 papers were obtained after screening 659 records identified in the PubMed, IEEE Xplore, Embase, and Google Scholar databases. The reviewed studies employ a mixture of open-access and self-collected datasets. Because COVID-19 has only recently been investigated, there is a limited amount of available data. Most of the data are crowdsourced, which motivated a detailed study of the various pre-processing techniques used by the reviewed studies. Although 13 of the 48 identified papers show promising results, several have been performed with small-scale datasets (<200). Among those papers, convolutional neural networks and support vector machine algorithms were the best-performing methods. The analysis of the extracted features showed that Mel-frequency cepstral coefficients and zero-crossing rate continue to be the most popular choices. Less common alternatives, such as non-linear features, have also been proven to be effective. The reported values for sensitivity range from 65.0% to 99.8% and those for accuracy from 59.0% to 99.8%.

A voice-based biomarker for monitoring symptom resolution in adults with COVID-19: Findings from the prospective Predi-COVID cohort study

People with COVID-19 can experience impairing symptoms that require enhanced surveillance. Our objective was to train an artificial intelligence-based model to predict the presence of COVID-19 symptoms and derive a digital vocal biomarker for easily and quantitatively monitoring symptom resolution. We used data from 272 participants in the prospective Predi-COVID cohort study recruited between May 2020 and May 2021. A total of 6473 voice features were derived from recordings of participants reading a standardized pre-specified text. Models were trained separately for Android devices and iOS devices. A binary outcome (symptomatic versus asymptomatic) was considered, based on a list of 14 frequent COVID-19 related symptoms. A total of 1775 audio recordings were analyzed (6.5 recordings per participant on average), including 1049 corresponding to symptomatic cases and 726 to asymptomatic ones. The best performances were obtained from Support Vector Machine models for both audio formats. We observed an elevated predictive capacity for both Android (AUC = 0.92, balanced accuracy = 0.83) and iOS (AUC = 0.85, balanced accuracy = 0.77) as well as low Brier scores (0.11 and 0.16 respectively for Android and iOS when assessing calibration. The vocal biomarker derived from the predictive models accurately discriminated asymptomatic from symptomatic individuals with COVID-19 (t-test P-values<0.001). In this prospective cohort study, we have demonstrated that using a simple, reproducible task of reading a standardized pre-specified text of 25 seconds enabled us to derive a vocal biomarker for monitoring the resolution of COVID-19 related symptoms with high accuracy and calibration.

Challenges and Opportunities of Deep Learning for Cough-Based COVID-19 Diagnosis: A Scoping Review

In the past two years, medical researchers and data scientists worldwide have focused their efforts on containing the pandemic of coronavirus disease 2019 (COVID-19). Deep learning models have been proven to be capable of efficient medical diagnosis and prognosis in cancer, common lung diseases, and COVID-19. On the other hand, artificial neural networks have demonstrated their potential in pattern recognition and classification in various domains, including healthcare. This literature review aims to report the state of research on developing neural network models to diagnose COVID-19 from cough sounds to create a cost-efficient and accessible testing tool in the fight against the pandemic. A total of 35 papers were included in this review following a screening of the 161 outputs of the literature search. We extracted information from articles on data resources, model structures, and evaluation metrics and then explored the scope of experimental studies and methodologies and analyzed their outcomes and limitations. We found that cough is a biomarker, and its associated information can determine an individual’s health status. Convolutional neural networks were predominantly used, suggesting they are particularly suitable for feature extraction and classification. The reported accuracy values ranged from 73.1% to 98.5%. Moreover, the dataset sizes ranged from 16 to over 30,000 cough audio samples. Although deep learning is a promising prospect in identifying COVID-19, we identified a gap in the literature on research conducted over large and diversified data sets.

A comparative analysis of deep neural network architectures for the dynamic diagnosis of COVID-19 based on acoustic cough features

The study aims to assess the detection performance of a rapid primary screening technique for COVID-19 that is purely based on the cough sound extracted from 2200 clinically validated samples using laboratory molecular testing (1100 COVID-19 negative and 1100 COVID-19 positive). Results and severity of samples based on quantitative RT-PCR (qRT-PCR), cycle threshold, and patient lymphocyte numbers were clinically labeled. Our suggested general methods consist of a tensor based on audio characteristics and deep-artificial neural network classification with deep cough convolutional layers, based on the dilated temporal convolution neural network (DTCN). DTCN has approximately 76% accuracy, 73.12% in TCN, and 72.11% in CNN-LSTM which have been trained at a learning rate of 0.2%, respectively. In our scenario, CNN-LSTM can no longer be employed for COVID-19 predictions, as they would generally offer questionable forecasts. In the previous stage, we discussed the exactness of the total cases of TCN, dilated TCN, and CNN-LSTM models which were truly predicted. Our proposed technique to identify COVID-19 can be considered as a robust and in-demand technique to rapidly detect the infection. We believe it can considerably hinder the COVID-19 pandemic worldwide.

COVID-19: respiratory disease diagnosis with regularized deep convolutional neural network using human respiratory sounds

Human respiratory sound auscultation (HRSA) parameters have been the real choice for detecting human respiratory diseases in the last few years. It is a challenging task to extract the respiratory sound features from the breath, voice, and cough sounds. The existing methods failed to extract the sound features to diagnose respiratory diseases. We proposed and evaluated a new regularized deep convolutional neural network (RDCNN) architecture to accept COVID-19 sound data and essential sound features. The proposed architecture is trained with the COVID-19 sound data sets and gives a better learning curve than any other state-of-the-art model. We examine the performance of RDCNN with Max-Pooling (Model-1) and without Max-Pooling (Model-2) functions. In this work, we observed that RDCNN model performance with three sound feature extraction methods [Soft-Mel frequency channel, Log-Mel frequency spectrum, and Modified Mel-frequency Cepstral Coefficient (MMFCC) spectrum] for COVID-19 sound data sets (KDD-data, ComParE2021-CCS-CSS-Data, and NeurlPs2021-data). To amplify the models' performance, we applied the augmentation technique along with regularization. We have also carried out this work to estimate the mutation of SARS-CoV-2 in the five waves using prognostic models (fractal-based). The proposed model achieves state-of-the-art performance on the COVID-19 sound data set to identify COVID-19 disease symptoms. The model's learnable parameter gradients have vanished in the intermediate layers while optimizing the prediction error which is addressed with our proposed RDCNN model. Our experiments suggested that 3 × 3 kernel size for regularized deep CNN (without max-pooling) shows 2-3% better classification accuracy compared to RDCNN with max-pooling. The experimental results suggest that this new approach may achieve the finest results on respiratory diseases.

Mobile Sensing in the COVID-19 Era: A Review

Background

During the COVID-19 pandemic, mobile sensing and data analytics techniques have demonstrated their capabilities in monitoring the trajectories of the pandemic, by collecting behavioral, physiological, and mobility data on individual, neighborhood, city, and national scales. Notably, mobile sensing has become a promising way to detect individuals' infectious status, track the change in long-term health, trace the epidemics in communities, and monitor the evolution of viruses and subspecies.

Methods

We followed the PRISMA practice and reviewed 60 eligible papers on mobile sensing for monitoring COVID-19. We proposed a taxonomy system to summarize literature by the time duration and population scale under mobile sensing studies.

Results

We found that existing literature can be naturally grouped in four clusters, including remote detection, long-term tracking, contact tracing, and epidemiological study. We summarized each group and analyzed representative works with regard to the system design, health outcomes, and limitations on techniques and societal factors. We further discussed the implications and future directions of mobile sensing in communicable diseases from the perspectives of technology and applications.

Conclusion

Mobile sensing techniques are effective, efficient, and flexible to surveil COVID-19 in scales of time and populations. In the post-COVID era, technical and societal issues in mobile sensing are expected to be addressed to improve healthcare and social outcomes.

Visual Detection of COVID-19 from Materials Aspect

ABSTRACT

In the recent COVID-19 pandemic, World Health Organization emphasized that early detection is an effective strategy to reduce the spread of SARS-CoV-2 viruses. Several diagnostic methods, such as reverse transcription-polymerase chain reaction (RT-PCR) and lateral flow immunoassay (LFIA), have been applied based on the mechanism of specific recognition and binding of the probes to viruses or viral antigens. Although the remarkable progress, these methods still suffer from inadequate cellular materials or errors in the detection and sampling procedure of nasopharyngeal/oropharyngeal swab collection. Therefore, developing accurate, ultrafast, and visualized detection calls for more advanced materials and technology urgently to fight against the epidemic. In this review, we first summarize the current methodologies for SARS-CoV-2 diagnosis. Then, recent representative examples are introduced based on various output signals (e.g., colorimetric, fluorometric, electronic, acoustic). Finally, we discuss the limitations of the methods and provide our perspectives on priorities for future test development.

Evaluating the COVID-19 Identification ResNet (CIdeR) on the INTERSPEECH COVID-19 From Audio Challenges

Several machine learning-based COVID-19 classifiers exploiting vocal biomarkers of COVID-19 has been proposed recently as digital mass testing methods. Although these classifiers have shown strong performances on the datasets on which they are trained, their methodological adaptation to new datasets with different modalities has not been explored. We report on cross-running the modified version of recent COVID-19 Identification ResNet (CIdeR) on the two Interspeech 2021 COVID-19 diagnosis from cough and speech audio challenges: ComParE and DiCOVA. CIdeR is an end-to-end deep learning neural network originally designed to classify whether an individual is COVID-19-positive or COVID-19-negative based on coughing and breathing audio recordings from a published crowdsourced dataset. In the current study, we demonstrate the potential of CIdeR at binary COVID-19 diagnosis from both the COVID-19 Cough and Speech Sub-Challenges of INTERSPEECH 2021, ComParE and DiCOVA. CIdeR achieves significant improvements over several baselines. We also present the results of the cross dataset experiments with CIdeR that show the limitations of using the current COVID-19 datasets jointly to build a collective COVID-19 classifier.

Audio texture analysis of COVID-19 cough, breath, and speech sounds

The coronavirus disease (COVID-19) first appeared at the end of December 2019 and is still spreading in most countries. To diagnose COVID-19 using reverse transcription - Polymerase chain reaction (RT-PCR), one has to go to a dedicated center, which requires significant cost and human resources. Hence, there is a requirement for a remote monitoring tool that can perform the preliminary screening of COVID-19. In this paper, we propose that a detailed audio texture analysis of COVID-19 sounds may help in performing the initial screening of COVID-19. The texture analysis is done on three different signal modalities of COVID-19, i.e. cough, breath, and speech signals. In this work, we have used 1141 samples of cough signals, 392 samples of breath signals, and 893 samples of speech signals. To analyze the audio textural behavior of COVID-19 sounds, the local binary patterns LBP) and Haralick’s features were extracted from the spectrogram of the signals. The textural analysis on cough and breath sounds was done on the following 5 classes for the first time: COVID-19 positive with cough, COVID-19 positive without cough, healthy person with cough, healthy person without cough, and an asthmatic cough. For speech sounds there were only two classes: COVID-19 positive, and COVID-19 negative. During experiments, 71.7% of the cough samples and 72.2% of breath samples were classified into 5 classes. Also, 79.7% of speech samples are classified into 2 classes. The highest accuracy rate of 98.9% was obtained when binary classification between COVID-19 cough and non-COVID-19 cough was done.

Explainable COVID-19 detection using fractal dimension and vision transformer with Grad-CAM on cough sounds

The polymerase chain reaction (PCR) test is not only time-intensive but also a contact method that puts healthcare personnel at risk. Thus, contactless and fast detection tests are more valuable. Cough sound is an important indicator of COVID-19, and in this paper, a novel explainable scheme is developed for cough sound-based COVID-19 detection. In the presented work, the cough sound is initially segmented into overlapping parts, and each segment is labeled as the input audio, which may contain other sounds. The deep Yet Another Mobile Network (YAMNet) model is considered in this work. After labeling, the segments labeled as cough are cropped and concatenated to reconstruct the pure cough sounds. Then, four fractal dimensions (FD) calculation methods are employed to acquire the FD coefficients on the cough sound with an overlapped sliding window that forms a matrix. The constructed matrixes are then used to form the fractal dimension images. Finally, a pretrained vision transformer (ViT) model is used to classify the constructed images into COVID-19, healthy and symptomatic classes. In this work, we demonstrate the performance of the ViT on cough sound-based COVID-19, and a visual explainability of the inner workings of the ViT model is shown. Three publically available cough sound datasets, namely COUGHVID, VIRUFY, and COSWARA, are used in this study. We have obtained 98.45%, 98.15%, and 97.59% accuracy for COUGHVID, VIRUFY, and COSWARA datasets, respectively. Our developed model obtained the highest performance compared to the state-of-the-art methods and is ready to be tested in real-world applications.

The Acoustic Dissection of Cough: Diving Into Machine Listening-based COVID-19 Analysis and Detection

OBJECTIVES

The coronavirus disease 2019 (COVID-19) has caused a crisis worldwide. Amounts of efforts have been made to prevent and control COVID-19's transmission, from early screenings to vaccinations and treatments. Recently, due to the spring up of many automatic disease recognition applications based on machine listening techniques, it would be fast and cheap to detect COVID-19 from recordings of cough, a key symptom of COVID-19. To date, knowledge of the acoustic characteristics of COVID-19 cough sounds is limited but would be essential for structuring effective and robust machine learning models. The present study aims to explore acoustic features for distinguishing COVID-19 positive individuals from COVID-19 negative ones based on their cough sounds.

METHODS

By applying conventional inferential statistics, we analyze the acoustic correlates of COVID-19 cough sounds based on the ComParE feature set, i.e., a standardized set of 6,373 acoustic higher-level features. Furthermore, we train automatic COVID-19 detection models with machine learning methods and explore the latent features by evaluating the contribution of all features to the COVID-19 status predictions.

RESULTS

The experimental results demonstrate that a set of acoustic parameters of cough sounds, e.g., statistical functionals of the root mean square energy and Mel-frequency cepstral coefficients, bear essential acoustic information in terms of effect sizes for the differentiation between COVID-19 positive and COVID-19 negative cough samples. Our general automatic COVID-19 detection model performs significantly above chance level, i.e., at an unweighted average recall (UAR) of 0.632, on a data set consisting of 1,411 cough samples (COVID-19 positive/negative: 210/1,201).

CONCLUSIONS

Based on the acoustic correlates analysis on the ComParE feature set and the feature analysis in the effective COVID-19 detection approach, we find that several acoustic features that show higher effects in conventional group difference testing are also higher weighted in the machine learning models.

Towards Multimodal Equipment to Help in the Diagnosis of COVID-19 Using Machine Learning Algorithms

COVID-19 occurs due to infection through respiratory droplets containing the SARS-CoV-2 virus, which are released when someone sneezes, coughs, or talks. The gold-standard exam to detect the virus is Real-Time Polymerase Chain Reaction (RT-PCR); however, this is an expensive test and may require up to 3 days after infection for a reliable result, and if there is high demand, the labs could be overwhelmed, which can cause significant delays in providing results. Biomedical data (oxygen saturation level—SpO2, body temperature, heart rate, and cough) are acquired from individuals and are used to help infer infection by COVID-19, using machine learning algorithms. The goal of this study is to introduce the Integrated Portable Medical Assistant (IPMA), which is a multimodal piece of equipment that can collect biomedical data, such as oxygen saturation level, body temperature, heart rate, and cough sound, and helps infer the diagnosis of COVID-19 through machine learning algorithms. The IPMA has the capacity to store the biomedical data for continuous studies and can be used to infer other respiratory diseases. Quadratic kernel-free non-linear Support Vector Machine (QSVM) and Decision Tree (DT) were applied on three datasets with data of cough, speech, body temperature, heart rate, and SpO2, obtaining an Accuracy rate (ACC) and Area Under the Curve (AUC) of approximately up to 88.0% and 0.85, respectively, as well as an ACC up to 99% and AUC = 0.94, respectively, for COVID-19 infection inference. When applied to the data acquired with the IMPA, these algorithms achieved 100% accuracy. Regarding the easiness of using the equipment, 36 volunteers reported that the IPMA has a high usability, according to results from two metrics used for evaluation: System Usability Scale (SUS) and Post Study System Usability Questionnaire (PSSUQ), with scores of 85.5 and 1.41, respectively. In light of the worldwide needs for smart equipment to help fight the COVID-19 pandemic, this new equipment may help with the screening of COVID-19 through data collected from biomedical signals and cough sounds, as well as the use of machine learning algorithms.

Machine learning for detecting COVID-19 from cough sounds: An ensemble-based MCDM method

This research aims to analyze the performance of state-of-the-art machine learning techniques for classifying COVID-19 from cough sounds and to identify the model(s) that consistently perform well across different cough datasets. Different performance evaluation metrics (precision, sensitivity, specificity, AUC, accuracy, etc.) make selecting the best performance model difficult. To address this issue, in this paper, we propose an ensemble-based multi-criteria decision making (MCDM) method for selecting top performance machine learning technique(s) for COVID-19 cough classification. We use four cough datasets, namely Cambridge, Coswara, Virufy, and NoCoCoDa to verify the proposed method. At first, our proposed method uses the audio features of cough samples and then applies machine learning (ML) techniques to classify them as COVID-19 or non-COVID-19. Then, we consider a multi-criteria decision-making (MCDM) method that combines ensemble technologies (i.e., soft and hard) to select the best model. In MCDM, we use the technique for order preference by similarity to ideal solution (TOPSIS) for ranking purposes, while entropy is applied to calculate evaluation criteria weights. In addition, we apply the feature reduction process through recursive feature elimination with cross-validation under different estimators. The results of our empirical evaluations show that the proposed method outperforms the state-of-the-art models. We see that when the proposed method is used for analysis using the Extra-Trees classifier, it has achieved promising results (AUC: 0.95, Precision: 1, Recall: 0.97).

Pay attention to the speech: COVID-19 diagnosis using machine learning and crowdsourced respiratory and speech recordings

Since the outbreak of COVID-19, many efforts have been made to utilize the respiratory sounds and coughs collected by smartphones for training Machine Learning models to classify and distinguish COVID-19 sounds from healthy ones. Embedding those models into mobile applications or Internet of things devices can make effective COVID-19 pre-screening tools afforded by anyone anywhere. Most of the previous researchers trained their classifiers with respiratory sounds such as breathing or coughs, and they achieved promising results. We claim that using special voice patterns besides other respiratory sounds can achieve better performance. In this study, we used the Coswara dataset where each user has recorded 9 different types of sounds as cough, breathing, and speech labeled with COVID-19 status. A combination of models trained on different sounds can diagnose COVID-19 more accurately than a single model trained on cough or breathing only. Our results show that using simple binary classifiers can achieve an AUC of 96.4% and an accuracy of 96% by averaging the predictions of multiple models trained and evaluated separately on different sound types. Finally, this study aims to draw attention to the importance of the human voice alongside other respiratory sounds for the sound-based COVID-19 diagnosis.

Graphene-based temperature, humidity, and strain sensor: A review on progress, characterization, and potential applications during Covid-19 pandemic

Graphene's potential as material for wearable, highly sensitive and robust sensor in various fields of technology has been widely investigated until now in order to capitalize on its unique intrinsic physical and chemical properties. In the wake of Covid-19 pandemic, it has been noticed that there are various potentials roles that can be fulfilled by graphene-based temperature, humidity and strain sensor, whose roles has not been widely explored to date. This paper takes the liberty to mainly highlight the progress layout and characterization technique for graphene-based sensor while including a brief discussion on the possible strategy of sensing data analysis that can be employed to minimize and prevent the risk of Covid-19 infection within a living community. While majority of the reported sensor is still in the in-progress status, its highlighted role in this work may provide a brief idea on how the ongoing research in graphene-based sensor may lead to the future implementation of the device for routine healthcare check-up and diagnostic point-care during and post-pandemic era. On the other hand, the sensitivity and response time data against working temperature, humidity and strain range that are provided could serve as a reference for benchmarking purpose, which certainly would help enthusiast in the development of a graphene-based sensor with a better performance for the future.

Attention-based hybrid CNN-LSTM and spectral data augmentation for COVID-19 diagnosis from cough sound

COVID-19 pandemic has fueled the interest in artificial intelligence tools for quick diagnosis to limit virus spreading. Over 60% of people who are infected complain of a dry cough. Cough and other respiratory sounds were used to build diagnosis models in much recent research. We propose in this work, an augmentation pipeline which is applied on the pre-filtered data and uses i) pitch-shifting technique to augment the raw signal and, ii) spectral data augmentation technique SpecAugment to augment the computed mel-spectrograms. A deep learning based architecture that hybridizes convolution neural networks and long-short term memory with an attention mechanism is proposed for building the classification model. The feasibility of the proposed is demonstrated through a set of testing scenarios using the large-scale COUGHVID cough dataset and through a comparison with three baselines models. We have shown that our classification model achieved 91.13% of testing accuracy, 90.93% of sensitivity and an area under the curve of receiver operating characteristic of 91.13%.

Past and Trends in Cough Sound Acquisition, Automatic Detection and Automatic Classification: A Comparative Review

Cough is a very common symptom and the most frequent reason for seeking medical advice. Optimized care goes inevitably through an adapted recording of this symptom and automatic processing. This study provides an updated exhaustive quantitative review of the field of cough sound acquisition, automatic detection in longer audio sequences and automatic classification of the nature or disease. Related studies were analyzed and metrics extracted and processed to create a quantitative characterization of the state-of-the-art and trends. A list of objective criteria was established to select a subset of the most complete detection studies in the perspective of deployment in clinical practice. One hundred and forty-four studies were short-listed, and a picture of the state-of-the-art technology is drawn. The trend shows an increasing number of classification studies, an increase of the dataset size, in part from crowdsourcing, a rapid increase of COVID-19 studies, the prevalence of smartphones and wearable sensors for the acquisition, and a rapid expansion of deep learning. Finally, a subset of 12 detection studies is identified as the most complete ones. An unequaled quantitative overview is presented. The field shows a remarkable dynamic, boosted by the research on COVID-19 diagnosis, and a perfect adaptation to mobile health.

QUCoughScope: An Intelligent Application to Detect COVID-19 Patients Using Cough and Breath Sounds

Problem—Since the outbreak of the COVID-19 pandemic, mass testing has become essential to reduce the spread of the virus. Several recent studies suggest that a significant number of COVID-19 patients display no physical symptoms whatsoever. Therefore, it is unlikely that these patients will undergo COVID-19 testing, which increases their chances of unintentionally spreading the virus. Currently, the primary diagnostic tool to detect COVID-19 is a reverse-transcription polymerase chain reaction (RT-PCR) test from the respiratory specimens of the suspected patient, which is invasive and a resource-dependent technique. It is evident from recent researches that asymptomatic COVID-19 patients cough and breathe in a different way than healthy people. Aim—This paper aims to use a novel machine learning approach to detect COVID-19 (symptomatic and asymptomatic) patients from the convenience of their homes so that they do not overburden the healthcare system and also do not spread the virus unknowingly by continuously monitoring themselves. Method—A Cambridge University research group shared such a dataset of cough and breath sound samples from 582 healthy and 141 COVID-19 patients. Among the COVID-19 patients, 87 were asymptomatic while 54 were symptomatic (had a dry or wet cough). In addition to the available dataset, the proposed work deployed a real-time deep learning-based backend server with a web application to crowdsource cough and breath datasets and also screen for COVID-19 infection from the comfort of the user’s home. The collected dataset includes data from 245 healthy individuals and 78 asymptomatic and 18 symptomatic COVID-19 patients. Users can simply use the application from any web browser without installation and enter their symptoms, record audio clips of their cough and breath sounds, and upload the data anonymously. Two different pipelines for screening were developed based on the symptoms reported by the users: asymptomatic and symptomatic. An innovative and novel stacking CNN model was developed using three base learners from of eight state-of-the-art deep learning CNN algorithms. The stacking CNN model is based on a logistic regression classifier meta-learner that uses the spectrograms generated from the breath and cough sounds of symptomatic and asymptomatic patients as input using the combined (Cambridge and collected) dataset. Results—The stacking model outperformed the other eight CNN networks with the best classification performance for binary classification using cough sound spectrogram images. The accuracy, sensitivity, and specificity for symptomatic and asymptomatic patients were 96.5%, 96.42%, and 95.47% and 98.85%, 97.01%, and 99.6%, respectively. For breath sound spectrogram images, the metrics for binary classification of symptomatic and asymptomatic patients were 91.03%, 88.9%, and 91.5% and 80.01%, 72.04%, and 82.67%, respectively. Conclusion—The web-application QUCoughScope records coughing and breathing sounds, converts them to a spectrogram, and applies the best-performing machine learning model to classify the COVID-19 patients and healthy subjects. The result is then reported back to the test user in the application interface. Therefore, this novel system can be used by patients in their premises as a pre-screening method to aid COVID-19 diagnosis by prioritizing the patients for RT-PCR testing and thereby reducing the risk of spreading of the disease.

Artificial Intelligence for Detecting COVID-19 With the Aid of Human Cough, Breathing and Speech Signals: Scoping Review

Goal: Official tests for COVID-19 are time consuming, costly, can produce high false negatives, use up vital chemicals and may violate social distancing laws. Therefore, a fast and reliable additional solution using recordings of cough, breathing and speech data for preliminary screening may help alleviate these issues. Objective: This scoping review explores how Artificial Intelligence (AI) technology aims to detect COVID-19 disease by using cough, breathing and speech recordings, as reported in the literature. Here, we describe and summarize attributes of the identified AI techniques and datasets used for their implementation. Methods: A scoping review was conducted following the guidelines of PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews). Electronic databases (Google Scholar, Science Direct, and IEEE Xplore) were searched between 1st April 2020 and 15th August 2021. Terms were selected based on the target intervention (i.e., AI), the target disease (i.e., COVID-19) and acoustic correlates of the disease (i.e., speech, breathing and cough). A narrative approach was used to summarize the extracted data. Results: 24 studies and 8 Apps out of the 86 retrieved studies met the inclusion criteria. Half of the publications and Apps were from the USA. The most prominent AI architecture used was a convolutional neural network, followed by a recurrent neural network. AI models were mainly trained, tested and run-on websites and personal computers, rather than on phone apps. More than half of the included studies reported area-under-the-curve performance of greater than 0.90 on symptomatic and negative datasets while one study achieved 100% sensitivity in predicting asymptomatic COVID-19 from cough-, breathing- or speech-based acoustic features. Conclusions: The included studies show that AI has the potential to help detect COVID-19 using cough, breathing and speech samples. The proposed methods (with some time and appropriate clinical testing) could prove to be an effective method in detecting various diseases related to respiratory and neurophysiological changes in the human body.

COVID-19 Diagnosis from Crowdsourced Cough Sound Data

The highly contagious and rapidly mutating COVID-19 virus is affecting individuals worldwide. A rapid and large-scale method for COVID-19 testing is needed to prevent infection. Cough testing using AI has been shown to be potentially valuable. In this paper, we propose a COVID-19 diagnostic method based on an AI cough test. We used only crowdsourced cough sound data to distinguish between the cough sound of COVID-19-positive people and that of healthy people. First, we used the COUGHVID cough database to segment only the cough sound from the original cough data. An effective audio feature set was then extracted from the segmented cough sounds. A deep learning model was trained on the extracted feature set. The COVID-19 diagnostic system constructed using this method had a sensitivity of 93% and a specificity of 94%, and achieved better results than models trained by other existing methods.