Adaptive automatic sleep stage classification under covariate shift

The future of sleep health: a data-driven revolution in sleep science and medicine

In recent years, there has been a significant expansion in the development and use of multi-modal sensors and technologies to monitor physical activity, sleep and circadian rhythms. These developments make accurate sleep monitoring at scale a possibility for the first time. Vast amounts of multi-sensor data are being generated with potential applications ranging from large-scale epidemiological research linking sleep patterns to disease, to wellness applications, including the sleep coaching of individuals with chronic conditions. However, in order to realise the full potential of these technologies for individuals, medicine and research, several significant challenges must be overcome. There are important outstanding questions regarding performance evaluation, as well as data storage, curation, processing, integration, modelling and interpretation. Here, we leverage expertise across neuroscience, clinical medicine, bioengineering, electrical engineering, epidemiology, computer science, mHealth and human-computer interaction to discuss the digitisation of sleep from a inter-disciplinary perspective. We introduce the state-of-the-art in sleep-monitoring technologies, and discuss the opportunities and challenges from data acquisition to the eventual application of insights in clinical and consumer settings. Further, we explore the strengths and limitations of current and emerging sensing methods with a particular focus on novel data-driven technologies, such as Artificial Intelligence.

Validation of 'Somnivore', a Machine Learning Algorithm for Automated Scoring and Analysis of Polysomnography Data

Manual scoring of polysomnography data is labor-intensive and time-consuming, and most existing software does not account for subjective differences and user variability. Therefore, we evaluated a supervised machine learning algorithm, Somnivore^TM, for automated wake–sleep stage classification. We designed an algorithm that extracts features from various input channels, following a brief session of manual scoring, and provides automated wake-sleep stage classification for each recording. For algorithm validation, polysomnography data was obtained from independent laboratories, and include normal, cognitively-impaired, and alcohol-treated human subjects (total n = 52), narcoleptic mice and drug-treated rats (total n = 56), and pigeons (n = 5). Training and testing sets for validation were previously scored manually by 1–2 trained sleep technologists from each laboratory. F-measure was used to assess precision and sensitivity for statistical analysis of classifier output and human scorer agreement. The algorithm gave high concordance with manual visual scoring across all human data (wake 0.91 ± 0.01; N1 0.57 ± 0.01; N2 0.81 ± 0.01; N3 0.86 ± 0.01; REM 0.87 ± 0.01), which was comparable to manual inter-scorer agreement on all stages. Similarly, high concordance was observed across all rodent (wake 0.95 ± 0.01; NREM 0.94 ± 0.01; REM 0.91 ± 0.01) and pigeon (wake 0.96 ± 0.006; NREM 0.97 ± 0.01; REM 0.86 ± 0.02) data. Effects of classifier learning from single signal inputs, simple stage reclassification, automated removal of transition epochs, and training set size were also examined. In summary, we have developed a polysomnography analysis program for automated sleep-stage classification of data from diverse species. Somnivore enables flexible, accurate, and high-throughput analysis of experimental and clinical sleep studies.

Promises and Challenges in the Use of Consumer-Grade Devices for Sleep Monitoring

The market for smartphones, smartwatches, and wearable devices is booming. In recent years, individuals and researchers have used these devices as additional tools to monitor and track sleep, physical activity, and behavior. Their use in sleep research and clinical applications could address the difficulties in scaling up studies that rely on polysomnography, the gold-standard. However, the use of commercial devices for large-scale sleep studies is not without challenges. With this in mind, this paper presents an extensive review of sleep monitoring systems and the techniques used in their development. We also discuss their performance in terms of reliability and validity, and consider the needs and expectations of users, whether they are experts, patients, or the general public. Through this review, we highlight a number of challenges with current studies: a lack of standard evaluation methods for consumer-grade devices (e.g., reliability and validity assessment); limitations in the populations studied; consumer expectations of monitoring devices; constraints on the resources of consumer-grade devices (e.g., power consumption).

Importance Weighted Import Vector Machine for Unsupervised Domain Adaptation

In real-world applications, the assumption of independent and identical distribution is no longer consistent. To alleviate the significant mismatch between source and target domains, importance weighting import vector machine, which is an adaptive classifier, is proposed. This adaptive probabilistic classification method, which is sparse and computationally efficient, can be used for unsupervised domain adaptation (DA). The effectiveness of the proposed approach is demonstrated via a toy problem, and a real-world cross-domain object recognition task. Even though the sparseness, the proposed method outperforms the state-of-the-art in both unsupervised and semisupervised DA scenarios. We also introduce a reliable importance weighted cross validation (RIWCV), which is an improvement of importance weighted cross validation, for parameter and model selection. The RIWCV avoid falling down in local minimum, by selecting a more reliable combination of the parameters instead of the best parameters.

Feature ranking and rank aggregation for automatic sleep stage classification: a comparative study

BACKGROUND

Nowadays, sleep quality is one of the most important measures of healthy life, especially considering the huge number of sleep-related disorders. Identifying sleep stages using polysomnographic (PSG) signals is the traditional way of assessing sleep quality. However, the manual process of sleep stage classification is time-consuming, subjective and costly. Therefore, in order to improve the accuracy and efficiency of the sleep stage classification, researchers have been trying to develop automatic classification algorithms. Automatic sleep stage classification mainly consists of three steps: pre-processing, feature extraction and classification. Since classification accuracy is deeply affected by the extracted features, a poor feature vector will adversely affect the classifier and eventually lead to low classification accuracy. Therefore, special attention should be given to the feature extraction and selection process.

METHODS

In this paper the performance of seven feature selection methods, as well as two feature rank aggregation methods, were compared. Pz-Oz EEG, horizontal EOG and submental chin EMG recordings of 22 healthy males and females were used. A comprehensive feature set including 49 features was extracted from these recordings. The extracted features are among the most common and effective features used in sleep stage classification from temporal, spectral, entropy-based and nonlinear categories. The feature selection methods were evaluated and compared using three criteria: classification accuracy, stability, and similarity.

RESULTS

Simulation results show that MRMR-MID achieves the highest classification performance while Fisher method provides the most stable ranking. In our simulations, the performance of the aggregation methods was in the average level, although they are known to generate more stable results and better accuracy.

CONCLUSIONS

The Borda and RRA rank aggregation methods could not outperform significantly the conventional feature ranking methods. Among conventional methods, some of them slightly performed better than others, although the choice of a suitable technique is dependent on the computational complexity and accuracy requirements of the user.

Automatic sleep stage classification based on sparse deep belief net and combination of multiple classifiers

This paper presents an automatic sleep stage method combining a sparse deep belief net and combination of multiple classifiers for electroencephalogram, electrooculogram and electromyogram. The sparse deep belief net was applied to extract features from these signals automatically, and the combination of multiple classifiers, utilizing the extracted features, assigned each 30-s epoch to one of the five possible sleep stages. More importantly, we proposed a new voting principle based on classification entropy to enhance the classification performance further by harnessing the complementary information provided by the individual classifier. Differently from existing methods, our method used unsupervised feature learning to extract features automatically from raw sleep data and classification based on the learned features. The results of automatic and manual scorings were compared on an epoch-by-epoch basis. The accuracies for wake, S1, S2, SWS and REM were 98.49%, 80.05%, 91.2%, 98.22% and 95.31%, respectively, and the total accuracy of sleep stage was 91.31%. The results demonstrated that the sparse deep belief net was an efficient feature extraction method for sleep data, and the combination of multiple classifiers based on classification entropy performed well on sleep stages.

A two-step automatic sleep stage classification method with dubious range detection

BACKGROUND

The limitations of the current systems of automatic sleep stage classification (ASSC) are essentially related to the similarities between epochs from different sleep stages and the subjects' variability. Several studies have already identified the situations with the highest likelihood of misclassification in sleep scoring. Here, we took advantage of such information to develop an ASSC system based on knowledge of subjects' variability of some indicators that characterize sleep stages and on the American Academy of Sleep Medicine (AASM) rules.

METHODS

An ASSC system consisting of a two-step classifier is proposed. In the first step, epochs are classified using support vector machines (SVMs) spread into different nodes of a decision tree. In the post-processing step, the epochs suspected of misclassification (dubious classification) are tagged, and a new classification is suggested. Identification and correction are based on the AASM rules, and on misclassifications most commonly found/reported in automatic sleep staging. Six electroencephalographic and two electrooculographic channels were used to classify wake, non-rapid eye movement (NREM) sleep--N1, N2 and N3, and rapid eye movement (REM) sleep.

RESULTS

The proposed system was tested in a dataset of 14 clinical polysomnographic records of subjects suspected of apnea disorders. Wake and REM epochs not falling in the dubious range, are classified with accuracy levels compatible with the requirements for clinical applications. The suggested correction assigned to the epochs that are tagged as dubious enhances the global results of all sleep stages.

CONCLUSIONS

This approach provides reliable sleep staging results for non-dubious epochs.