Towards More Accurate Automatic Sleep Staging via Deep Transfer Learning

Efficient Neural Network Classification of Parkinson's Disease and Schizophrenia Using Resting-State EEG Data

Timely identification of Parkinson's disease and schizophrenia is crucial for the effective management and enhancement of patients' quality of life. The utilization of electroencephalogram (EEG) monitoring applications has proven instrumental in diagnosing various brain disorders. Prior research has predominantly relied on predefined knowledge of physiological alterations associated with different diseases, employing feature extraction to discern brain conditions. This study introduces SwiftBrainNet, a neural network designed for the classification of Parkinson's disease and schizophrenia using short resting-state EEG segments. SwiftBrainNet aims to minimize reliance on manual feature extraction, relying solely on short EEG segments. Functioning as a single-input, dual-output neural network, SwiftBrainNet incorporates a deep supervisory mechanism facilitated by an auxiliary decoder, which enhances its classification performance by guiding the network in extracting shallow features. Our study conducts a clinical application-oriented experiment that uses continuous multi-segment EEG voting classification. This experiment demonstrates a noticeable improvement in accuracy compared to leave-one-out cross-validation (LOOCV), especially when combined with our data augmentation techniques. These findings underscore the method's practical value in clinical settings, where continuous data frames and enhanced generalization across subjects can significantly improve diagnostic accuracy. Additionally, the high accuracy observed in subject-dependent classification with very short data segments suggests that SwiftBrainNet might capture subject-specific EEG patterns, which could be further explored to enhance disease-related feature learning. This paper provides new evidence supporting the use of short-term EEG data for neurodiagnostic applications, making SwiftBrainNet a promising tool for the early detection of neurological disorders.

Optimizing wearable single-channel electroencephalography sleep staging in a heterogeneous sleep-disordered population using transfer learning

STUDY OBJECTIVES

Although various wearable electroencephalography devices have been developed, performance evaluation of the devices and their associated automated sleep stage classification models is mostly limited to healthy participants. A major barrier for applying automated wearable electroencephalography sleep staging in clinical populations is the need for large-scale data for model training. We therefore investigated transfer learning as a strategy to overcome limited data availability and optimize automated single-channel electroencephalography sleep staging in people with sleep disorders.

METHODS

We acquired 52 single-channel frontopolar headband electroencephalography recordings from a heterogeneous sleep-disordered population with concurrent polysomnography (PSG). We compared 3 model training strategies: "pretraining" (ie, training on a larger dataset of 901 conventional PSGs), "training-from-scratch" (ie, training on wearable headband recordings), and "fine-tuning" (ie, training on conventional PSGs, followed by training on headband recordings). Performance was evaluated on all headband recordings using 10-fold cross-validation.

RESULTS

Highest performance for 5-stage classification was achieved with fine-tuning (κ = .778), significantly higher than with pretraining (κ = .769) and with training-from-scratch (κ = .733). No significant differences or systematic biases were observed with clinically relevant sleep parameters derived from PSG. All sleep disorder categories showed comparable performance.

CONCLUSIONS

This study emphasizes the importance of leveraging larger available datasets through deep transfer learning to optimize performance with limited data availability. Findings indicate strong similarity in data characteristics between conventional PSG and headband recordings. Altogether, results suggest the combination of the headband, classification model, and training methodology can be viable for sleep monitoring in the heterogeneous clinical population.

CITATION

van der Aar JF, van Gilst MM, van den Ende DA, et al. Optimizing wearable single-channel electroencephalography sleep staging in a heterogeneous sleep-disordered population using transfer learning. J Clin Sleep Med. 2025;21(2):315-323.

Automated remote sleep monitoring needs uncertainty quantification

Wearable electroencephalography devices emerge as a cost-effective and ergonomic alternative to gold-standard polysomnography, paving the way for better health monitoring and sleep disorder screening. Machine learning allows to automate sleep stage classification, but trust and reliability issues have hampered its adoption in clinical applications. Estimating uncertainty is a crucial factor in enhancing reliability by identifying regions of heightened and diminished confidence. In this study, we used an uncertainty-centred machine learning pipeline, U-PASS, to automate sleep staging in a challenging real-world dataset of single-channel electroencephalography and accelerometry collected with a wearable device from an elderly population. We were able to effectively limit the uncertainty of our machine learning model and to reliably inform clinical experts of which predictions were uncertain to improve the machine learning model's reliability. This increased the five-stage sleep-scoring accuracy of a state-of-the-art machine learning model from 63.9% to 71.2% on our dataset. Remarkably, the machine learning approach outperformed the human expert in interpreting these wearable data. Manual review by sleep specialists, without specific training for sleep staging on wearable electroencephalography, proved ineffective. The clinical utility of this automated remote monitoring system was also demonstrated, establishing a strong correlation between the predicted sleep parameters and the reference polysomnography parameters, and reproducing known correlations with the apnea-hypopnea index. In essence, this work presents a promising avenue to revolutionize remote patient care through the power of machine learning by the use of an automated data-processing pipeline enhanced with uncertainty estimation.

Outlier Handling Strategy of Ensembled-Based Sequential Convolutional Neural Networks for Sleep Stage Classification

The pivotal role of sleep has led to extensive research endeavors aimed at automatic sleep stage classification. However, existing methods perform poorly when classifying small groups or individuals, and these results are often considered outliers in terms of overall performance. These outliers may introduce bias during model training, adversely affecting feature selection and diminishing model performance. To address the above issues, this paper proposes an ensemble-based sequential convolutional neural network (E-SCNN) that incorporates a clustering module and neural networks. E-SCNN effectively ensembles machine learning and deep learning techniques to minimize outliers, thereby enhancing model robustness at the individual level. Specifically, the clustering module categorizes individuals based on similarities in feature distribution and assigns personalized weights accordingly. Subsequently, by combining these tailored weights with the robust feature extraction capabilities of convolutional neural networks, the model generates more accurate sleep stage classifications. The proposed model was verified on two public datasets, and experimental results demonstrate that the proposed method obtains overall accuracies of 84.8% on the Sleep-EDF Expanded dataset and 85.5% on the MASS dataset. E-SCNN can alleviate the outlier problem, which is important for improving sleep quality monitoring for individuals.

An enzyme-inspired specificity in deep learning model for sleep stage classification using multi-channel PSG signals input: Separating training approach and its performance on cross-dataset validation for generalizability

Numerous automatic sleep stage classification systems have been developed, but none have become effective assistive tools for sleep technicians due to issues with generalization. Four key factors hinder the generalization of these models are instruments, montage of recording, subject type, and scoring manual factors. This study aimed to develop a deep learning model that addresses generalization problems by integrating enzyme-inspired specificity and employing separating training approaches. Subject type and scoring manual factors were controlled, while the focus was on instruments and montage of recording factors. The proposed model consists of three sets of signal-specific models including EEG-, EOG-, and EMG-specific model. The EEG-specific models further include three sets of channel-specific models. All signal-specific and channel-specific models were established with data manipulation and weighted loss strategies, resulting in three sets of data manipulation models and class-specific models, respectively. These models were CNNs. Additionally, BiLSTM models were applied to EEG- and EOG-specific models to obtain temporal information. Finally, classification task for sleep stage was handled by 'the-last-dense' layer. The optimal sampling frequency for each physiological signal was identified and used during the training process. The proposed model was trained on MGH dataset and evaluated using both within dataset and cross-dataset. For MGH dataset, overall accuracy of 81.05 %, MF1 of 79.05 %, Kappa of 0.7408, and per-class F1-scores: W (84.98 %), N1 (58.06 %), N2 (84.82 %), N3 (79.20 %), and REM (88.17 %) can be achieved. Performances on cross-datasets are as follows: SHHS1 200 records reached 79.54 %, 70.56 %, and 0.7078; SHHS2 200 records achieved 76.77 %, 66.30 %, and 0.6632; Sleep-EDF 153 records gained 78.52 %, 72.13 %, and 0.7031; and BCI-MU (local dataset) 94 records achieved 83.57 %, 82.17 %, and 0.7769 for overall accuracy, MF1, and Kappa respectively. Additionally, the proposed model has approximately 9.3 M trainable parameters and takes around 26 s to process one PSG record. The results indicate that the proposed model demonstrates generalizability in sleep stage classification and shows potential as a feasibility tool for real-world applications. Additionally, enzyme-inspired specificity effectively addresses the challenges posed by varying montage of recording, while the identified optimal frequencies mitigate instrument-related issues.

FlexSleepTransformer: a transformer-based sleep staging model with flexible input channel configurations

Clinical sleep diagnosis traditionally relies on polysomnography (PSG) and expert manual classification of sleep stages. Recent advancements in deep learning have shown promise in automating sleep stage classification using a single PSG channel. However, variations in PSG acquisition devices and environments mean that the number of PSG channels can differ across sleep centers. To integrate a sleep staging method into clinical practice effectively, it must accommodate a flexible number of PSG channels. In this paper, we proposed FlexSleepTransformer, a transformer-based model designed to handle varying number of input channels, making it adaptable to diverse sleep staging datasets. We evaluated FlexSleepTransformer using two distinct datasets: the public SleepEDF-78 dataset and the local SleepUHS dataset. Notably, FlexSleepTransformer is the first model capable of simultaneously training on datasets with differing number of PSG channels. Our experiments showed that FlexSleepTransformer trained on both datasets together achieved 98% of the accuracy compared to models trained on each dataset individually. Furthermore, it outperformed models trained exclusively on one dataset when tested on the other dataset. Additionally, FlexSleepTransformer surpassed state-of-the-art CNN and RNN-based models on both datasets. Due to its adaptability with varying channels numbers, FlexSleepTransformer holds significant potential for clinical adoption, especially when trained with data from a wide range of sleep centers.

Retrospective validation of automatic sleep analysis with grey areas model for human-in-the-loop scoring approach

State-of-the-art automatic sleep staging methods have demonstrated comparable reliability and superior time efficiency to manual sleep staging. However, fully automatic black-box solutions are difficult to adapt into clinical workflow due to the lack of transparency in decision-making processes. Transparency would be crucial for interaction between automatic methods and the work of sleep experts, i.e., in human-in-the-loop applications. To address these challenges, we propose an automatic sleep staging model (aSAGA) that effectively utilises both electroencephalography and electro-oculography channels while incorporating transparency of uncertainty in the decision-making process. We validated the model through extensive retrospective testing using a range of datasets, including open-access, clinical, and research-driven sources. Our channel-wise ensemble model, trained on both electroencephalography and electro-oculography signals, demonstrated robustness and the ability to generalise across various types of sleep recordings, including novel self-applied home polysomnography. Additionally, we compared model uncertainty with human uncertainty in sleep staging and studied various uncertainty mapping metrics to identify ambiguous regions, or "grey areas", that may require manual re-evaluation. The validation of this grey area concept revealed its potential to enhance sleep staging accuracy and to highlight regions in the recordings where sleep experts may struggle to reach a consensus. In conclusion, this study provides a technical basis and understanding of automatic sleep staging uncertainty. Our approach has the potential to improve the integration of automatic sleep staging into clinical practice; however, further studies are needed to test the model prospectively in real-world clinical settings and human-in-the-loop scoring applications.

Personalization of Automatic Sleep Scoring: How Best to Adapt Models to Personal Domains in Wearable EEG

Wearable EEG enables us to capture large amounts of high-quality sleep data for diagnostic purposes. To make full use of this capacity we need high-performance automatic sleep scoring models. To this end, it has been noted that domain mismatch between recording equipment can be considerable, e.g. PSG to wearable EEG, but a previously observed benefit from personalizing models to individual subjects further indicates a personal domain in sleep EEG. In this work, we have investigated the extent of such a personal domain in wearable EEG, and review supervised and unsupervised approaches to personalization as found in the literature. We investigated the personalization effect of the unsupervised Adversarial Domain Adaptation and implemented an unsupervised method based on statistics alignment. No beneficial personalization effect was observed using these unsupervised methods. We find that supervised personalization leads to a substantial performance improvement on the target subject ranging from 15% Cohen's Kappa for subjects with poor performance ( ) to roughly 2% on subjects with high performance ( ). This improvement was present for models trained on both small and large data sets, indicating that even high-performance models benefit from supervised personalization. We found that this personalization can be beneficially regularized using Kullback-Leibler regularization, leading to lower variance with negligible cost to improvement. Based on the experiments, we recommend model personalization using Kullback-Leibler regularization.

SleepBoost: a multi-level tree-based ensemble model for automatic sleep stage classification

Neurodegenerative diseases often exhibit a strong link with sleep disruption, highlighting the importance of effective sleep stage monitoring. In this light, automatic sleep stage classification (ASSC) plays a pivotal role, now more streamlined than ever due to the advancements in deep learning (DL). However, the opaque nature of DL models can be a barrier in their clinical adoption, due to trust concerns among medical practitioners. To bridge this gap, we introduce SleepBoost, a transparent multi-level tree-based ensemble model specifically designed for ASSC. Our approach includes a crafted feature engineering block (FEB) that extracts 41 time and frequency domain features, out of which 23 are selected based on their high mutual information score (> 0.23). Uniquely, SleepBoost integrates three fundamental linear models into a cohesive multi-level tree structure, further enhanced by a novel reward-based adaptive weight allocation mechanism. Tested on the Sleep-EDF-20 dataset, SleepBoost demonstrates superior performance with an accuracy of 86.3%, F1-score of 80.9%, and Cohen kappa score of 0.807, outperforming leading DL models in ASSC. An ablation study underscores the critical role of our selective feature extraction in enhancing model accuracy and interpretability, crucial for clinical settings. This innovative approach not only offers a more transparent alternative to traditional DL models but also extends potential implications for monitoring and understanding sleep patterns in the context of neurodegenerative disorders. The open-source availability of SleepBoost's implementation at https://github.com/akibzaman/SleepBoost can further facilitate its accessibility and potential for widespread clinical adoption.

PSEENet: A Pseudo-Siamese Neural Network Incorporating Electroencephalography and Electrooculography Characteristics for Heterogeneous Sleep Staging

Sleep staging plays a critical role in evaluating the quality of sleep. Currently, most studies are either suffering from dramatic performance drops when coping with varying input modalities or unable to handle heterogeneous signals. To handle heterogeneous signals and guarantee favorable sleep staging performance when a single modality is available, a pseudo-siamese neural network (PSN) to incorporate electroencephalography (EEG), electrooculography (EOG) characteristics is proposed (PSEENet). PSEENet consists of two parts, spatial mapping modules (SMMs) and a weight-shared classifier. SMMs are used to extract high-dimensional features. Meanwhile, joint linkages among multi-modalities are provided by quantifying the similarity of features. Finally, with the cooperation of heterogeneous characteristics, associations within various sleep stages can be established by the classifier. The evaluation of the model is validated on two public datasets, namely, Montreal Archive of Sleep Studies (MASS) and SleepEDFX, and one clinical dataset from Huashan Hospital of Fudan University (HSFU). Experimental results show that the model can handle heterogeneous signals, provide superior results under multimodal signals and show good performance with single modality. PSEENet obtains accuracy of 79.1%, 82.1% with EEG, EEG and EOG on Sleep-EDFX, and significantly improves the accuracy with EOG from 73.7% to 76% by introducing similarity information.

Toward Interpretable Sleep Stage Classification Using Cross-Modal Transformers

Accurate sleep stage classification is significant for sleep health assessment. In recent years, several machine-learning based sleep staging algorithms have been developed, and in particular, deep-learning based algorithms have achieved performance on par with human annotation. Despite improved performance, a limitation of most deep-learning based algorithms is their black-box behavior, which have limited their use in clinical settings. Here, we propose a cross-modal transformer, which is a transformer-based method for sleep stage classification. The proposed cross-modal transformer consists of a cross-modal transformer encoder architecture along with a multi-scale one-dimensional convolutional neural network for automatic representation learning. The performance of our method is on-par with the state-of-the-art methods and eliminates the black-box behavior of deep-learning models by utilizing the interpretability aspect of the attention modules. Furthermore, our method provides considerable reductions in the number of parameters and training time compared to the state-of-the-art methods. Our code is available at https://github.com/Jathurshan0330/Cross-Modal-Transformer. A demo of our work can be found at https://bit.ly/Cross_modal_transformer_demo.

Validation of a sleep staging classification model for healthy adults based on two combinations of a single-channel EEG headband and wrist actigraphy

STUDY OBJECTIVES

The aim of this study was to develop a sleep staging classification model capable of accurately performing on different wearable devices.

METHODS

Twenty-three healthy participants underwent a full-night type I polysomnography and used two device combinations: (A) flexible single-channel electroencephalogram (EEG) headband + actigraphy (n = 12) and (B) rigid single-channel EEG headband + actigraphy (n = 11). The signals were segmented into 30-second epochs according to polysomnographic stages (scored by a board-certified sleep technologist; model ground truth) and 18 frequency and time features were extracted. The model consisted of an ensemble of bagged decision trees. Bagging refers to bootstrap aggregation to reduce overfitting and improve generalization. To evaluate the model, a training dataset under 5-fold cross-validation and an 80-20% dataset split was used. The headbands were also evaluated without the actigraphy feature. Participants also completed a usability evaluation (comfort, pain while sleeping, and sleep disturbance).

RESULTS

Combination A had an F1-score of 98.4% and the flexible headband alone of 97.7% (error rate for N1: combination A = 9.8%; flexible headband alone = 15.7%). Combination B had an F1-score of 96.9% and the rigid headband alone of 95.3% (error rate for N1: combination B = 17.0%; rigid headband alone = 27.7%); in both, N1 was more confounded with N2.

CONCLUSIONS

We developed an accurate sleep classification model based on a single-channel EEG device, and actigraphy was not an important feature of the model. Both headbands were found to be useful, with the rigid one being more disruptive to sleep. Future research can improve our results by applying the developed model in a population with sleep disorders.

CLINICAL TRIAL REGISTRATION

Registry: ClinicalTrials.gov; Name: Actigraphy, Wearable EEG Band and Smartphone for Sleep Staging; URL: https://clinicaltrials.gov/study/NCT04943562; Identifier: NCT04943562.

CITATION

Melo MC, Vallim JRS, Garbuio S, et al. Validation of a sleep staging classification model for healthy adults based on 2 combinations of a single-channel EEG headband and wrist actigraphy. J Clin Sleep Med. 2024;20(6):983-990.

SensorNet: An Adaptive Attention Convolutional Neural Network for Sensor Feature Learning

This work develops a generalizable neural network, SENSORNET, for sensor feature learning across various applications. The primary challenge addressed is the poor portability of pretrained neural networks to new applications with limited sensor data. To solve this challenge, we design SensorNet, which integrates the flexibility of self-attention with multi-scale feature locality of convolution. Moreover, we invent patch-wise self-attention with stacked multi-heads to enrich the sensor feature representation. SensorNet is generalizable to pervasive applications with any number of sensor inputs, and is much smaller than the state-of-the-art self-attention and convolution hybrid baseline (0.83 M vs. 3.87 M parameters) with similar performance. The experimental results show that SensorNet is able to achieve state-of-the-art performance compared with the top five models on a competition activity recognition dataset (SHL'18). Moreover, pretrained SensorNet in a large inertial measurement unit (IMU) dataset can be fine-tuned to achieve the best accuracy on a much smaller IMU dataset (up to 5% improvement in WISDM) and to achieve the state-of-the-art performance on an EEG dataset (SLEEP-EDF-20), showing the strong generalizability of our approach.

ZleepAnlystNet: a novel deep learning model for automatic sleep stage scoring based on single-channel raw EEG data using separating training

Numerous models for sleep stage scoring utilizing single-channel raw EEG signal have typically employed CNN and BiLSTM architectures. While these models, incorporating temporal information for sequence classification, demonstrate superior overall performance, they often exhibit low per-class performance for N1-stage, necessitating an adjustment of loss function. However, the efficacy of such adjustment is constrained by the training process. In this study, a pioneering training approach called separating training is introduced, alongside a novel model, to enhance performance. The developed model comprises 15 CNN models with varying loss function weights for feature extraction and 1 BiLSTM for sequence classification. Due to its architecture, this model cannot be trained using an end-to-end approach, necessitating separate training for each component using the Sleep-EDF dataset. Achieving an overall accuracy of 87.02%, MF1 of 82.09%, Kappa of 0.8221, and per-class F1-socres (W 90.34%, N1 54.23%, N2 89.53%, N3 88.96%, and REM 87.40%), our model demonstrates promising performance. Comparison with sleep technicians reveals a Kappa of 0.7015, indicating alignment with reference sleep stags. Additionally, cross-dataset validation and adaptation through training with the SHHS dataset yield an overall accuracy of 84.40%, MF1 of 74.96% and Kappa of 0.7785 when tested with the Sleep-EDF-13 dataset. These findings underscore the generalization potential in model architecture design facilitated by our novel training approach.

Transfer Learning for Automatic Sleep Staging Using a Pre-Gelled Electrode Grid

Novel sensor solutions for sleep monitoring at home could alleviate bottlenecks in sleep medical care as well as enable selective or continuous observation over long periods of time and contribute to new insights in sleep medicine and beyond. Since especially in the latter case the sensor data differ strongly in signal, number and extent of sensors from the classical polysomnography (PSG) sensor technology, an automatic evaluation is essential for the application. However, the training of an automatic algorithm is complicated by the fact that the development phase of the new sensor technology, extensive comparative measurements with standardized reference systems, is often not possible and therefore only small datasets are available. In order to circumvent high system-specific training data requirements, we employ pre-training on large datasets with finetuning on small datasets of new sensor technology to enable automatic sleep phase detection for small test series. By pre-training on publicly available PSG datasets and finetuning on 12 nights recorded with new sensor technology based on a pre-gelled electrode grid to capture electroencephalography (EEG), electrooculography (EOG) and electromyography (EMG), an F1 score across all sleep phases of 0.81 is achieved (wake 0.84, N1 0.62, N2 0.81, N3 0.87, REM 0.88), using only EEG and EOG. The analysis additionally considers the spatial distribution of the channels and an approach to approximate classical electrode positions based on specific linear combinations of the new sensor grid channels.

Research and application of deep learning-based sleep staging: Data, modeling, validation, and clinical practice

Over the past few decades, researchers have attempted to simplify and accelerate the process of sleep stage classification through various approaches; however, only a few such approaches have gained widespread acceptance. Artificial intelligence technology, particularly deep learning, is promising for earning the trust of the sleep medicine community in automated sleep-staging systems, thus facilitating its application in clinical practice and integration into daily life. We aimed to comprehensively review the latest methods that are applying deep learning for enhancing sleep staging efficiency and accuracy. Starting from the requisite "data" for constructing deep learning algorithms, we elucidated the current landscape of this domain and summarized the fundamental modeling process, encompassing signal selection, data pre-processing, model architecture, classification tasks, and performance metrics. Furthermore, we reviewed the applications of automated sleep staging in scenarios such as sleep-disorder screening, diagnostic procedures, and health monitoring and management. Finally, we conducted an in-depth analysis and discussion of the challenges and future in intelligent sleep staging, particularly focusing on large-scale sleep datasets, interdisciplinary collaborations, and human-computer interactions.

U-PASS: An uncertainty-guided deep learning pipeline for automated sleep staging

With the increasing prevalence of machine learning in critical fields like healthcare, ensuring the safety and reliability of these systems is crucial. Estimating uncertainty plays a vital role in enhancing reliability by identifying areas of high and low confidence and reducing the risk of errors. This study introduces U-PASS, a specialized human-centered machine learning pipeline tailored for clinical applications, which effectively communicates uncertainty to clinical experts and collaborates with them to improve predictions. U-PASS incorporates uncertainty estimation at every stage of the process, including data acquisition, training, and model deployment. Training is divided into a supervised pre-training step and a semi-supervised recording-wise finetuning step. We apply U-PASS to the challenging task of sleep staging and demonstrate that it systematically improves performance at every stage. By optimizing the training dataset, actively seeking feedback from domain experts for informative samples, and deferring the most uncertain samples to experts, U-PASS achieves an impressive expert-level accuracy of 85% on a challenging clinical dataset of elderly sleep apnea patients. This represents a significant improvement over the starting point at 75% accuracy. The largest improvement gain is due to the deferral of uncertain epochs to a sleep expert. U-PASS presents a promising AI approach to incorporating uncertainty estimation in machine learning pipelines, improving their reliability and unlocking their potential in clinical settings.

Dual-Teacher Feature Distillation: A Transfer Learning Method for Insomniac PSG Staging

Insomnia is the most common sleep disorder linked with adverse long-term medical and psychiatric outcomes. Automatic sleep staging plays a crucial role in aiding doctors to diagnose insomnia disorder. Only a few studies have been conducted to develop automatic sleep staging methods for insomniacs, and most of them have utilized transfer learning methods, which involve pre-training models on healthy individuals and then fine-tuning them on insomniacs. Unfortunately, significant differences in feature distribution between the two subject groups impede the transfer performance, highlighting the need to effectively integrate the features of healthy subjects and insomniacs. In this paper, we propose a dual-teacher cross-domain knowledge transfer method based on the feature-based knowledge distillation to improve the performance of sleep staging for insomniacs. Specifically, the insomnia teacher directly learns from insomniacs and feeds the corresponding domain-specific features into the student network, while the health domain teacher guide the student network to learn domain-generic features. During the training process, we adopt the OFD (Overhaul of Feature Distillation) method to build the health domain teacher. We conducted the experiments to validate the proposed method, using the Sleep-EDF database as the source domain and the CAP-Database as the target domain. The results demonstrate that our method surpasses advanced techniques, achieving an average sleep staging accuracy of 80.56% on the CAP-Database. Furthermore, our method exhibits promising performance on the private dataset.

Modality-Specific Feature Selection, Data Augmentation and Temporal Context for Improved Performance in Sleep Staging

This work attempts to design an effective sleep staging system, making the best use of the available signals, strategies, and features in the literature. It must not only perform well on different datasets comprising healthy and clinical populations but also achieve good accuracy in cross-dataset experiments. Toward this end, we propose a model comprising multiple binary classifiers in a hierarchical fashion, where, at each level, one or more of EEG, EOG, and EMG are selected to best differentiate between two sleep stages. The best set of 100 features is chosen out of all the features derived from selected signals. The class imbalance in data is addressed by random undersampling and boosting techniques with decision trees as weak learners. Temporal context and data augmentation are used to improve the performance. We also evaluate the performance of our model by training and testing on different datasets. We compare the results of five approaches: using only EEG, EEG+EOG, EEG+EMG+EOG, EEG+EMG, and selective modality with a specific combination of EEG, EMG, and/or EOG at each level. The best results are obtained by considering features from EEG+EMG+EOG at each hierarchical level. The proposed model achieves average accuracies of 83.1%, 90.0%, 84.4%, 82.1%, 81.5%, 79.9%, and 73.7% on Sleep-EDF, Exp Sleep-EDF, ISRUC-S1, S2 and S3, DRMS-SUB, and DRMS-PAT datasets, respectively. For all the datasets except DRMS-SUB, the proposed method outperforms all the state-of-the-art approaches. Cross-dataset performance exceeds 80% for all datasets except DRMS-PAT; independent of whether the test data is from normal subjects or patients.

Deep transfer learning for automated single-lead EEG sleep staging with channel and population mismatches

Introduction: Automated sleep staging using deep learning models typically requires training on hundreds of sleep recordings, and pre-training on public databases is therefore common practice. However, suboptimal sleep stage performance may occur from mismatches between source and target datasets, such as differences in population characteristics (e.g., an unrepresented sleep disorder) or sensors (e.g., alternative channel locations for wearable EEG). Methods: We investigated three strategies for training an automated single-channel EEG sleep stager: pre-training (i.e., training on the original source dataset), training-from-scratch (i.e., training on the new target dataset), and fine-tuning (i.e., training on the original source dataset, fine-tuning on the new target dataset). As source dataset, we used the F3-M2 channel of healthy subjects (N = 94). Performance of the different training strategies was evaluated using Cohen's Kappa (κ) in eight smaller target datasets consisting of healthy subjects (N = 60), patients with obstructive sleep apnea (OSA, N = 60), insomnia (N = 60), and REM sleep behavioral disorder (RBD, N = 22), combined with two EEG channels, F3-M2 and F3-F4. Results: No differences in performance between the training strategies was observed in the age-matched F3-M2 datasets, with an average performance across strategies of κ = .83 in healthy, κ = .77 in insomnia, and κ = .74 in OSA subjects. However, in the RBD set, where data availability was limited, fine-tuning was the preferred method (κ = .67), with an average increase in κ of .15 to pre-training and training-from-scratch. In the presence of channel mismatches, targeted training is required, either through training-from-scratch or fine-tuning, increasing performance with κ = .17 on average. Discussion: We found that, when channel and/or population mismatches cause suboptimal sleep staging performance, a fine-tuning approach can yield similar to superior performance compared to building a model from scratch, while requiring a smaller sample size. In contrast to insomnia and OSA, RBD data contains characteristics, either inherent to the pathology or age-related, which apparently demand targeted training.

Decentralized Data-Privacy Preserving Deep-Learning Approaches for Enhancing Inter-Database Generalization in Automatic Sleep Staging

Automatic sleep staging has been an active field of development. Despite multiple efforts, the area remains a focus of research interest. Indeed, while promising results have reported in past literature, uptake of automatic sleep scoring in the clinical setting remains low. One of the current issues regards the difficulty to generalization performance results beyond the local testing scenario, i.e. across data from different clinics. Issues derived from data-privacy restrictions, that generally apply in the medical domain, pose additional difficulties in the successful development of these methods. We propose the use of several decentralized deep-learning approaches, namely ensemble models and federated learning, for robust inter-database performance generalization and data-privacy preservation in automatic sleep staging scenario. Specifically, we explore four ensemble combination strategies (max-voting, output averaging, size-proportional weighting, and Nelder-Mead) and present a new federated learning algorithm, so-called sub-sampled federated stochastic gradient descent (ssFedSGD). To evaluate generalization capabilities of such approaches, experimental procedures are carried out using a leaving-one-database-out direct-transfer scenario on six independent and heterogeneous public sleep staging databases. The resulting performance is compared with respect to two baseline approaches involving single-database and centralized multiple-database derived models. Our results show that proposed decentralized learning methods outperform baseline local approaches, and provide similar generalization results to centralized database-combined approaches. We conclude that these methods are more preferable choices, as they come with additional advantages concerning improved scalability, flexible design, and data-privacy preservation.

DenSleepNet: DenseNet based model for sleep staging with two-frequency feature fusion and coordinate attention

Sleep staging is often applied to assess the quality of sleep and also be used to prevent and monitor psychiatric disorders caused by sleep. However, it remains a challenge to extract the discriminative features of salient waveforms in sleep EEG and enable the network to effectively classify sleep stages by emphasizing these crucial features, thus achieving higher accuracy. In this study, an end-to-end deep learning model based on DenseNet for automatic sleep staging is designed and constructed. In the framework, two convolutional branches are devised to extract the underlying features (Two-Frequency Feature) at various frequencies, which are then fused and input into the DenseNet module to extract salient waveform features. After that, the Coordinate Attention mechanism is employed to enhance the localization of salient waveform features by emphasizing the position of salient waveforms and the spatial relationship across the entire frequency spectrum. Finally, the obtained features are accessed to the fully connected for sleep staging. The model was validated with a 20-fold cross-validation procedure on two public available datasets, and the overall accuracy, kappa coefficient, and MF1 score reached 92.9%, 78.7, 0.86 and 90.0%, 75.8, 0.80 on Sleep-EDF-20 and Sleep-EDFx, respectively. Experimental results show that the proposed model achieves competitive performance for sleep staging compared with the reported approaches under the identical conditions.

Optimizing sleep staging on multimodal time series: Leveraging borderline synthetic minority oversampling technique and supervised convolutional contrastive learning

Sleep is an important research area in nutritional medicine that plays a crucial role in human physical and mental health restoration. It can influence diet, metabolism, and hormone regulation, which can affect overall health and well-being. As an essential tool in the sleep study, the sleep stage classification provides a parsing of sleep architecture and a comprehensive understanding of sleep patterns to identify sleep disorders and facilitate the formulation of targeted sleep interventions. However, the class imbalance issue is typically salient in sleep datasets, which severely affects classification performances. To address this issue and to extract optimal multimodal features of EEG, EOG, and EMG that can improve the accuracy of sleep stage classification, a Borderline Synthetic Minority Oversampling Technique (B-SMOTE)-Based Supervised Convolutional Contrastive Learning (BST-SCCL) is proposed, which can avoid the risk of data mismatch between various sleep knowledge domains (varying health conditions and annotation rules) and strengthening learning characteristics of the N1 stage from the pair-wise segments comparison strategy. The lightweight residual network architecture with a novel truncated cross-entropy loss function is designed to accommodate multimodal time series and boost the training speed and performance stability. The proposed model has been validated on four well-known public sleep datasets (Sleep-EDF-20, Sleep-EDF-78, ISRUC-1, and ISRUC-3) and its superior performance (overall accuracy of 91.31-92.34%, MF1 of 88.21-90.08%, and Cohen's Kappa coefficient k of 0.87-0.89) has further demonstrated its effectiveness. It shows the great potential of contrastive learning for cross-domain knowledge interaction in precision medicine.

GAC-SleepNet: A dual-structured sleep staging method based on graph structure and Euclidean structure

Sleep staging is a precondition for the diagnosis and treatment of sleep disorders. However, how to fully exploit the relationship between spatial features of the brain and sleep stages is an important task. Many current classical algorithms only extract the characteristic information of the brain in the Euclidean space without considering other spatial structures. In this study, a sleep staging network named GAC-SleepNet is designed. GAC-SleepNet uses the characteristic information in the dual structure of the graph structure and the Euclidean structure for the classification of sleep stages. In the graph structure, this study uses a graph convolutional neural network to learn the deep features of each sleep stage and converts the features in the topological structure into feature vectors by a multilayer perceptron. In the Euclidean structure, this study uses convolutional neural networks to learn the temporal features of sleep information and combine attention mechanism to portray the connection between different sleep periods and EEG signals, while enhancing the description of global features to avoid local optima. In this study, the performance of the proposed network is evaluated on two public datasets. The experimental results show that the dual spatial structure captures more adequate and comprehensive information about sleep features and shows advancement in terms of different evaluation metrics.

L-SeqSleepNet: Whole-cycle Long Sequence Modeling for Automatic Sleep Staging

Human sleep is cyclical with a period of approximately 90 minutes, implying long temporal dependency in the sleep data. Yet, exploring this long-term dependency when developing sleep staging models has remained untouched. In this work, we show that while encoding the logic of a whole sleep cycle is crucial to improve sleep staging performance, the sequential modelling approach in existing state-of-the-art deep learning models are inefficient for that purpose. We thus introduce a method for efficient long sequence modelling and propose a new deep learning model, L-SeqSleepNet, which takes into account whole-cycle sleep information for sleep staging. Evaluating L-SeqSleepNet on four distinct databases of various sizes, we demonstrate state-of-the-art performance obtained by the model over three different EEG setups, including scalp EEG in conventional Polysomnography (PSG), in-ear EEG, and around-the-ear EEG (cEEGrid), even with a single EEG channel input. Our analyses also show that L-SeqSleepNet is able to alleviate the predominance of N2 sleep (the major class in terms of classification) to bring down errors in other sleep stages. Moreover the network becomes much more robust, meaning that for all subjects where the baseline method had exceptionally poor performance, their performance are improved significantly. Finally, the computation time only grows at a sub-linear rate when the sequence length increases.

Challenges of Applying Automated Polysomnography Scoring at Scale

Automatic polysomnography analysis can be leveraged to shorten scoring times, reduce associated costs, and ultimately improve the overall diagnosis of sleep disorders. Multiple and diverse strategies have been attempted for implementation of this technology at scale in the routine workflow of sleep centers. The field, however, is complex and presents unsolved challenges in a number of areas. Recent developments in computer science and artificial intelligence are nevertheless closing the gap. Technological advances are also opening new pathways for expanding our current understanding of the domain and its analysis.

SeriesSleepNet: an EEG time series model with partial data augmentation for automatic sleep stage scoring

Introduction: We propose an automatic sleep stage scoring model, referred to as SeriesSleepNet, based on convolutional neural network (CNN) and bidirectional long short-term memory (bi-LSTM) with partial data augmentation. We used single-channel raw electroencephalography signals for automatic sleep stage scoring. Methods: Our framework was focused on time series information, so we applied partial data augmentation to learn the connected time information in small series. In specific, the CNN module learns the time information of one epoch (intra-epoch) whereas the bi-LSTM trains the sequential information between the adjacent epochs (inter-epoch). Note that the input of the bi-LSTM is the augmented CNN output. Moreover, the proposed loss function was used to fine-tune the model by providing additional weights. To validate the proposed framework, we conducted two experiments using the Sleep-EDF and SHHS datasets. Results and Discussion: The results achieved an overall accuracy of 0.87 and 0.84 and overall F1-score of 0.80 and 0.78 and kappa value of 0.81 and 0.78 for five-class classification, respectively. We showed that the SeriesSleepNet was superior to the baselines based on each component in the proposed framework. Our architecture also outperformed the state-of-the-art methods with overall F1-score, accuracy, and kappa value. Our framework could provide information on sleep disorders or quality of sleep to automatically classify sleep stages with high performance.

A Spatial-Temporal Transformer Architecture Using Multi-Channel Signals for Sleep Stage Classification

Sleep stage classification is a fundamental task in diagnosing and monitoring sleep diseases. There are 2 challenges that remain open: (1) Since most methods only rely on input from a single channel, the spatial-temporal relationship of sleep signals has not been fully explored. (2) Lack of sleep data makes models hard to train from scratch. Here, we propose a vision Transformer-based architecture to process multi-channel polysomnogram signals. The method is an end-to-end framework that consists of a spatial encoder, a temporal encoder, and an MLP head classifier. The spatial encoder using a pre-trained Vision Transformer captures spatial information from multiple PSG channels. The temporal encoder utilizing the self-attention mechanism understands transitions between nearby epochs. In addition, we introduce a tailored image generation method to extract features within multi-channel and reshape them for transfer learning. We validate our method on 3 datasets and outperform the state-of-the-art algorithms. Our method fully explores the spatial-temporal relationship among different brain regions and addresses the problem of data insufficiency in clinical environments. Benefiting from reformulating the problem as image classification, the method could be applied to other 1D-signal problems in the future.

An End-to-End Multi-Channel Convolutional Bi-LSTM Network for Automatic Sleep Stage Detection

Sleep stage detection from polysomnography (PSG) recordings is a widely used method of monitoring sleep quality. Despite significant progress in the development of machine-learning (ML)-based and deep-learning (DL)-based automatic sleep stage detection schemes focusing on single-channel PSG data, such as single-channel electroencephalogram (EEG), electrooculogram (EOG), and electromyogram (EMG), developing a standard model is still an active subject of research. Often, the use of a single source of information suffers from data inefficiency and data-skewed problems. Instead, a multi-channel input-based classifier can mitigate the aforementioned challenges and achieve better performance. However, it requires extensive computational resources to train the model, and, hence, a tradeoff between performance and computational resources cannot be ignored. In this article, we aim to introduce a multi-channel, more specifically a four-channel, convolutional bidirectional long short-term memory (Bi-LSTM) network that can effectively exploit spatiotemporal features of data collected from multiple channels of the PSG recording (e.g., EEG Fpz-Cz, EEG Pz-Oz, EOG, and EMG) for automatic sleep stage detection. First, a dual-channel convolutional Bi-LSTM network module has been designed and pre-trained utilizing data from every two distinct channels of the PSG recording. Subsequently, we have leveraged the concept of transfer learning circuitously and have fused two dual-channel convolutional Bi-LSTM network modules to detect sleep stages. In the dual-channel convolutional Bi-LSTM module, a two-layer convolutional neural network has been utilized to extract spatial features from two channels of the PSG recordings. These extracted spatial features are subsequently coupled and given as input at every level of the Bi-LSTM network to extract and learn rich temporal correlated features. Both Sleep EDF-20 and Sleep EDF-78 (expanded version of Sleep EDF-20) datasets are used in this study to evaluate the result. The model that includes an EEG Fpz-Cz + EOG module and an EEG Fpz-Cz + EMG module can classify sleep stage with the highest value of accuracy (ACC), Kappa (Kp), and F1 score (e.g., 91.44%, 0.89, and 88.69%, respectively) on the Sleep EDF-20 dataset. On the other hand, the model consisting of an EEG Fpz-Cz + EMG module and an EEG Pz-Oz + EOG module shows the best performance (e.g., the value of ACC, Kp, and F1 score are 90.21%, 0.86, and 87.02%, respectively) compared to other combinations for the Sleep EDF-78 dataset. In addition, a comparative study with respect to other existing literature has been provided and discussed in order to exhibit the efficacy of our proposed model.

Automatic sleep staging of single-channel EEG based on domain adversarial neural networks and domain self-attention

The diagnosis and management of sleep problems depend heavily on sleep staging. For autonomous sleep staging, many data-driven deep learning models have been presented by trying to construct a large-labeled auxiliary sleep dataset and test it by electroencephalograms on different subjects. These approaches suffer a significant setback cause it assumes the training and test data come from the same or similar distribution. However, this is almost impossible in scenario cross-dataset due to inherent domain shift between domains. Unsupervised domain adaption was recently created to address the domain shift issue. However, only a few customized UDA solutions for sleep staging due to two limitations in previous UDA methods. First, the domain classifier does not consider boundaries between classes. Second, they depend on a shared model to align the domain that could miss the information of domains when extracting features. Given those restrictions, we present a novel UDA approach that combines category decision boundaries and domain discriminator to align the distributions of source and target domains. Also, to keep the domain-specific features, we create an unshared attention method. In addition, we investigated effective data augmentation in cross-dataset sleep scenarios. The experimental results on three datasets validate the efficacy of our approach and show that the proposed method is superior to state-of-the-art UDA methods on accuracy and MF1-Score.

Generalizable Deep Learning-Based Sleep Staging Approach for Ambulatory Textile Electrode Headband Recordings

Reliable, automated, and user-friendly solutions for the identification of sleep stages in home environment are needed in various clinical and scientific research settings. Previously we have shown that signals recorded with an easily applicable textile electrode headband (FocusBand Technologies, T 2 Green Pty Ltd) contain characteristics similar to the standard electrooculography (EOG, E1-M2). We hypothesize that the electroencephalographic (EEG) signals recorded using the textile electrode headband are similar enough with standard EOG in order to develop an automatic neural network-based sleep staging method that generalizes from diagnostic polysomnographic (PSG) data to ambulatory sleep recordings of textile electrode-based forehead EEG. Standard EOG signals together with manually annotated sleep stages from clinical PSG dataset (n = 876) were used to train, validate, and test a fully convolutional neural network (CNN). Furthermore, ambulatory sleep recordings including a standard set of gel-based electrodes and the textile electrode headband were conducted for 10 healthy volunteers at their homes to test the generalizability of the model. In the test set (n = 88) of the clinical dataset, the model's accuracy for 5-stage sleep stage classification was 80% (κ = 0.73) using only the single-channel EOG. The model generalized well for the headband-data, reaching 82% (κ = 0.75) overall sleep staging accuracy. In comparison, accuracy of the model was 87% (κ = 0.82) in home recordings using the standard EOG. In conclusion, the CNN model shows potential on automatic sleep staging of healthy individuals using a reusable electrode headband in a home environment.

Sleep Stage Classification in Children Using Self-Attention and Gaussian Noise Data Augmentation

The analysis of sleep stages for children plays an important role in early diagnosis and treatment. This paper introduces our sleep stage classification method addressing the following two challenges: the first is the data imbalance problem, i.e., the highly skewed class distribution with underrepresented minority classes. For this, a Gaussian Noise Data Augmentation (GNDA) algorithm was applied to polysomnography recordings to seek the balance of data sizes for different sleep stages. The second challenge is the difficulty in identifying a minority class of sleep stages, given their short sleep duration and similarities to other stages in terms of EEG characteristics. To overcome this, we developed a DeConvolution- and Self-Attention-based Model (DCSAM) which can inverse the feature map of a hidden layer to the input space to extract local features and extract the correlations between all possible pairs of features to distinguish sleep stages. The results on our dataset show that DCSAM based on GNDA obtains an accuracy of 90.26% and a macro F1-score of 86.51% which are higher than those of our previous method. We also tested DCSAM on a well-known public dataset-Sleep-EDFX-to prove whether it is applicable to sleep data from adults. It achieves a comparable performance to state-of-the-art methods, especially accuracies of 91.77%, 92.54%, 94.73%, and 95.30% for six-stage, five-stage, four-stage, and three-stage classification, respectively. These results imply that our DCSAM based on GNDA has a great potential to offer performance improvements in various medical domains by considering the data imbalance problems and correlations among features in time series data.

U-Sleep's resilience to AASM guidelines

AASM guidelines are the result of decades of efforts aiming at standardizing sleep scoring procedure, with the final goal of sharing a worldwide common methodology. The guidelines cover several aspects from the technical/digital specifications, e.g., recommended EEG derivations, to detailed sleep scoring rules accordingly to age. Automated sleep scoring systems have always largely exploited the standards as fundamental guidelines. In this context, deep learning has demonstrated better performance compared to classical machine learning. Our present work shows that a deep learning-based sleep scoring algorithm may not need to fully exploit the clinical knowledge or to strictly adhere to the AASM guidelines. Specifically, we demonstrate that U-Sleep, a state-of-the-art sleep scoring algorithm, can be strong enough to solve the scoring task even using clinically non-recommended or non-conventional derivations, and with no need to exploit information about the chronological age of the subjects. We finally strengthen a well-known finding that using data from multiple data centers always results in a better performing model compared with training on a single cohort. Indeed, we show that this latter statement is still valid even by increasing the size and the heterogeneity of the single data cohort. In all our experiments we used 28528 polysomnography studies from 13 different clinical studies.

Quantifying the impact of data characteristics on the transferability of sleep stage scoring models

Deep learning models for scoring sleep stages based on single-channel EEG have been proposed as a promising method for remote sleep monitoring. However, applying these models to new datasets, particularly from wearable devices, raises two questions. First, when annotations on a target dataset are unavailable, which different data characteristics affect the sleep stage scoring performance the most and by how much? Second, when annotations are available, which dataset should be used as the source of transfer learning to optimize performance? In this paper, we propose a novel method for computationally quantifying the impact of different data characteristics on the transferability of deep learning models. Quantification is accomplished by training and evaluating two models with significant architectural differences, TinySleepNet and U-Time, under various transfer configurations in which the source and target datasets have different recording channels, recording environments, and subject conditions. For the first question, the environment had the highest impact on sleep stage scoring performance, with performance degrading by over 14% when sleep annotations were unavailable. For the second question, the most useful transfer sources for TinySleepNet and the U-Time models were MASS-SS1 and ISRUC-SG1, containing a high percentage of N1 (the rarest sleep stage) relative to the others. The frontal and central EEGs were preferred for TinySleepNet. The proposed approach enables full utilization of existing sleep datasets for training and planning model transfer to maximize the sleep stage scoring performance on a target problem when sleep annotations are limited or unavailable, supporting the realization of remote sleep monitoring.

Cross-Domain Transfer of EEG to EEG or ECG Learning for CNN Classification Models

Electroencephalography (EEG) is often used to evaluate several types of neurological brain disorders because of its noninvasive and high temporal resolution. In contrast to electrocardiography (ECG), EEG can be uncomfortable and inconvenient for patients. Moreover, deep-learning techniques require a large dataset and a long time for training from scratch. Therefore, in this study, EEG-EEG or EEG-ECG transfer learning strategies were applied to explore their effectiveness for the training of simple cross-domain convolutional neural networks (CNNs) used in seizure prediction and sleep staging systems, respectively. The seizure model detected interictal and preictal periods, whereas the sleep staging model classified signals into five stages. The patient-specific seizure prediction model with six frozen layers achieved 100% accuracy for seven out of nine patients and required only 40 s of training time for personalization. Moreover, the cross-signal transfer learning EEG-ECG model for sleep staging achieved an accuracy approximately 2.5% higher than that of the ECG model; additionally, the training time was reduced by >50%. In summary, transfer learning from an EEG model to produce personalized models for a more convenient signal can both reduce the training time and increase the accuracy; moreover, challenges such as data insufficiency, variability, and inefficiency can be effectively overcome.

Pre-training in Medical Data: A Survey

Medical data refers to health-related information associated with regular patient care or as part of a clinical trial program. There are many categories of such data, such as clinical imaging data, bio-signal data, electronic health records (EHR), and multi-modality medical data. With the development of deep neural networks in the last decade, the emerging pre-training paradigm has become dominant in that it has significantly improved machine learning methods’ performance in a data-limited scenario. In recent years, studies of pre-training in the medical domain have achieved significant progress. To summarize these technology advancements, this work provides a comprehensive survey of recent advances for pre-training on several major types of medical data. In this survey, we summarize a large number of related publications and the existing benchmarking in the medical domain. Especially, the survey briefly describes how some pre-training methods are applied to or developed for medical data. From a data-driven perspective, we examine the extensive use of pre-training in many medical scenarios. Moreover, based on the summary of recent pre-training studies, we identify several challenges in this field to provide insights for future studies.

Self-Supervised Learning for Label- Efficient Sleep Stage Classification: A Comprehensive Evaluation

The past few years have witnessed a remarkable advance in deep learning for EEG-based sleep stage classification (SSC). However, the success of these models is attributed to possessing a massive amount of labeled data for training, limiting their applicability in real-world scenarios. In such scenarios, sleep labs can generate a massive amount of data, but labeling can be expensive and time-consuming. Recently, the self-supervised learning (SSL) paradigm has emerged as one of the most successful techniques to overcome labels' scarcity. In this paper, we evaluate the efficacy of SSL to boost the performance of existing SSC models in the few-labels regime. We conduct a thorough study on three SSC datasets, and we find that fine-tuning the pretrained SSC models with only 5% of labeled data can achieve competitive performance to the supervised training with full labels. Moreover, self-supervised pretraining helps SSC models to be more robust to data imbalance and domain shift problems.

Detection of Mental Fatigue in the General Population: Feasibility Study of Keystroke Dynamics as a Real-world Biomarker

BACKGROUND

Mental fatigue is a common and potentially debilitating state that can affect individuals' health and quality of life. In some cases, its manifestation can precede or mask early signs of other serious mental or physiological conditions. Detecting and assessing mental fatigue can be challenging nowadays as it relies on self-evaluation and rating questionnaires, which are highly influenced by subjective bias. Introducing more objective, quantitative, and sensitive methods to characterize mental fatigue could be critical to improve its management and the understanding of its connection to other clinical conditions.

OBJECTIVE

This paper aimed to study the feasibility of using keystroke biometrics for mental fatigue detection during natural typing. As typing involves multiple motor and cognitive processes that are affected by mental fatigue, our hypothesis was that the information captured in keystroke dynamics can offer an interesting mean to characterize users' mental fatigue in a real-world setting.

METHODS

We apply domain transformation techniques to adapt and transform TypeNet, a state-of-the-art deep neural network, originally intended for user authentication, to generate a network optimized for the fatigue detection task. All experiments were conducted using 3 keystroke databases that comprise different contexts and data collection protocols.

RESULTS

Our preliminary results showed area under the curve performances ranging between 72.2% and 80% for fatigue versus rested sample classification, which is aligned with previously published models on daily alertness and circadian cycles. This demonstrates the potential of our proposed system to characterize mental fatigue fluctuations via natural typing patterns. Finally, we studied the performance of an active detection approach that leverages the continuous nature of keystroke biometric patterns for the assessment of users' fatigue in real time.

CONCLUSIONS

Our results suggest that the psychomotor patterns that characterize mental fatigue manifest during natural typing, which can be quantified via automated analysis of users' daily interaction with their device. These findings represent a step towards the development of a more objective, accessible, and transparent solution to monitor mental fatigue in a real-world environment.

Automatic sleep stages classification using multi-level fusion

Sleep efficiency is a factor that can determine a person's healthy life. Sleep efficiency can be calculated by analyzing the results of the sleep stage classification. There have been many studies to classify sleep stages automatically using multiple signals to improve the accuracy of the sleep stage classification. The fusion method is used to process multi-signal data. Fusion methods include data-level fusion, feature-level fusion, and decision-level fusion methods. We propose a multi-level fusion method to increase the accuracy of the sleep stage classification when using multi-signal data consisting of electroencephalography and electromyography signals. First, we used feature-level fusion to fuse the extracted features using a convolutional neural network for multi-signal data. Then, after obtaining each classified result using the fused feature data, the sleep stage was derived using a decision-level fusion method that fused classified results. We used public datasets, Sleep-EDF, to measure performance; we confirmed that the proposed multi-level fusion method yielded a higher accuracy of 87.2%, respectively, compared to single-level fusion method and more existing methods. The proposed multi-level fusion method showed the most improved performance in classifying N1 stage, where existing methods had the lowest performance.

Comparison of deep transfer learning algorithms and transferability measures for wearable sleep staging

Background

Obtaining medical data using wearable sensors is a potential replacement for in-hospital monitoring, but the lack of data for such sensors poses a challenge for development. One solution is using in-hospital recordings to boost performance via transfer learning. While there are many possible transfer learning algorithms, few have been tested in the domain of EEG-based sleep staging. Furthermore, there are few ways for determining which transfer learning method will work best besides exhaustive testing. Measures of transferability do exist, but are typically used for selection of pre-trained models rather than algorithms and few have been tested on medical signals. We tested several supervised transfer learning algorithms on a sleep staging task using a single channel of EEG (AF7-Fpz) captured from an in-home commercial system.

Results

Two neural networks—one bespoke and another state-of-art open-source architecture—were pre-trained on one of six source datasets comprising 11,561 subjects undergoing clinical polysomnograms (PSGs), then re-trained on a target dataset of 75 full-night recordings from 24 subjects. Several transferability measures were then tested to determine which is most effective for assessing performance on unseen target data. Performance on the target dataset was improved using transfer learning, with re-training the head layers being the most effective in the majority of cases (up to 63.9% of cases). Transferability measures generally provided significant correlations with accuracy (up to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$r = -0.53$$\end{document}r=-0.53).

Conclusion

Re-training the head layers provided the largest performance boost. Transferability measures are useful indicators of transfer learning effectiveness.

XSleepNet: Multi-View Sequential Model for Automatic Sleep Staging

Automating sleep staging is vital to scale up sleep assessment and diagnosis to serve millions experiencing sleep deprivation and disorders and enable longitudinal sleep monitoring in home environments. Learning from raw polysomnography signals and their derived time-frequency image representations has been prevalent. However, learning from multi-view inputs (e.g., both the raw signals and the time-frequency images) for sleep staging is difficult and not well understood. This work proposes a sequence-to-sequence sleep staging model, XSleepNet,¹ that is capable of learning a joint representation from both raw signals and time-frequency images. Since different views may generalize or overfit at different rates, the proposed network is trained such that the learning pace on each view is adapted based on their generalization/overfitting behavior. In simple terms, the learning on a particular view is speeded up when it is generalizing well and slowed down when it is overfitting. View-specific generalization/overfitting measures are computed on-the-fly during the training course and used to derive weights to blend the gradients from different views. As a result, the network is able to retain the representation power of different views in the joint features which represent the underlying distribution better than those learned by each individual view alone. Furthermore, the XSleepNet architecture is principally designed to gain robustness to the amount of training data and to increase the complementarity between the input views. Experimental results on five databases of different sizes show that XSleepNet consistently outperforms the single-view baselines and the multi-view baseline with a simple fusion strategy. Finally, XSleepNet also outperforms prior sleep staging methods and improves previous state-of-the-art results on the experimental databases.

Towards an automatic narcolepsy detection on ambiguous sleep staging and sleep transition dynamics joint model

Objective.Mixing/dissociation of sleep stages in narcolepsy adds to the difficulty in automatic sleep staging. Moreover, automatic analytical studies for narcolepsy and multiple sleep latency test (MSLT) have only done automatic sleep staging without leveraging the sleep stage profile for further patient identification. This study aims to establish an automatic narcolepsy detection method for MSLT.Approach.We construct a two-phase model on MSLT recordings, where ambiguous sleep staging and sleep transition dynamics make joint efforts to address this issue. In phase 1, we extract representative features from electroencephalogram (EEG) and electrooculogram (EOG) signals. Then, the features are input to an EasyEnsemble classifier for automatic sleep staging. In phase 2, we investigate sleep transition dynamics, including sleep stage transitions and sleep stages, and output likelihood of narcolepsy by virtue of principal component analysis (PCA) and a logistic regression classifier. To demonstrate the proposed framework in clinical application, we conduct experiments on 24 participants from our hospital, considering ten patients with narcolepsy and fourteen patients with MSLT negative.Main results.Applying the two-phase leave-one-subject-out testing scheme, the model reaches an accuracy, sensitivity, and specificity of 87.5%, 80.0%, and 92.9% for narcolepsy detection. Influenced by disease pathology, accuracy of automatic sleep staging in narcolepsy appears to decrease compared to that in the non-narcoleptic population.Significance.This method can automatically and efficiently distinguish patients with narcolepsy based on MSLT. It probes into the amalgamation of automatic sleep staging and sleep transition dynamics for narcolepsy detection, which would assist clinic and neuroelectrophysiology specialists in visual interpretation and diagnosis.

From unsupervised to semi-supervised adversarial domain adaptation in EEG-based sleep staging

OBJECTIVE

The recent breakthrough of wearable sleep monitoring devices results in large amounts of sleep data. However, as limited labels are available, interpreting these data requires automated sleep stage classification methods with a small need for labeled training data. Transfer learning and domain adaptation offer possible solutions by enabling models to learn on a source dataset and adapt to a target dataset.

APPROACH

In this paper, we investigate adversarial domain adaptation applied to real use cases with wearable sleep datasets acquired from diseased patient populations. Different practical aspects of the adversarial domain adaptation framework \hl{are examined}, including the added value of (pseudo-)labels from the target dataset and the influence of domain mismatch between the source and target data. The method is also implemented for personalization to specific patients.

MAIN RESULTS

The results show that adversarial domain adaptation is effective in the application of sleep staging on wearable data. When compared to a model applied on a target dataset without any adaptation, the domain adaptation method in its simplest form achieves relative gains of 7%-27% in accuracy. The performance on the target domain is further boosted by adding pseudo-labels and real target domain labels when available, and by choosing an appropriate source dataset. Furthermore, unsupervised adversarial domain adaptation can also personalize a model, improving the performance by 1%-2% compared to a non-personal model.

SIGNIFICANCE

In conclusion, adversarial domain adaptation provides a flexible framework for semi-supervised and unsupervised transfer learning. This is particularly useful in sleep staging and other wearable EEG applications.

Automatic sleep staging of EEG signals: recent development, challenges, and future directions

Modern deep learning holds a great potential to transform clinical practice on human sleep. Teaching a machine to carry out routine tasks would be a tremendous reduction in workload for clinicians. Sleep staging, a fundamental step in sleep practice, is a suitable task for this and will be the focus in this article. Recently, automatic sleep staging systems have been trained to mimic manual scoring, leading to similar performance to human sleep experts, at least on scoring of healthy subjects. Despite tremendous progress, we have not seen automatic sleep scoring adopted widely in clinical environments. This review aims to give a shared view of the authors on the most recent state-of-the-art development in automatic sleep staging, the challenges that still need to be addressed, and the future directions for automatic sleep scoring to achieve clinical value.

Transfer learning for non-image data in clinical research: A scoping review

BACKGROUND

Transfer learning is a form of machine learning where a pre-trained model trained on a specific task is reused as a starting point and tailored to another task in a different dataset. While transfer learning has garnered considerable attention in medical image analysis, its use for clinical non-image data is not well studied. Therefore, the objective of this scoping review was to explore the use of transfer learning for non-image data in the clinical literature.

METHODS AND FINDINGS

We systematically searched medical databases (PubMed, EMBASE, CINAHL) for peer-reviewed clinical studies that used transfer learning on human non-image data. We included 83 studies in the review. More than half of the studies (63%) were published within 12 months of the search. Transfer learning was most often applied to time series data (61%), followed by tabular data (18%), audio (12%) and text (8%). Thirty-three (40%) studies applied an image-based model to non-image data after transforming data into images (e.g. spectrograms). Twenty-nine (35%) studies did not have any authors with a health-related affiliation. Many studies used publicly available datasets (66%) and models (49%), but fewer shared their code (27%).

CONCLUSIONS

In this scoping review, we have described current trends in the use of transfer learning for non-image data in the clinical literature. We found that the use of transfer learning has grown rapidly within the last few years. We have identified studies and demonstrated the potential of transfer learning in clinical research in a wide range of medical specialties. More interdisciplinary collaborations and the wider adaption of reproducible research principles are needed to increase the impact of transfer learning in clinical research.

SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantification

BACKGROUND

Black-box skepticism is one of the main hindrances impeding deep-learning-based automatic sleep scoring from being used in clinical environments.

METHODS

Towards interpretability, this work proposes a sequence-to-sequence sleep staging model, namely SleepTransformer. It is based on the transformer backbone and offers interpretability of the models decisions at both the epoch and sequence level. We further propose a simple yet efficient method to quantify uncertainty in the models decisions. The method, which is based on entropy, can serve as a metric for deferring low-confidence epochs to a human expert for further inspection.

RESULTS

Making sense of the transformers self-attention scores for interpretability, at the epoch level, the attention scores are encoded as a heat map to highlight sleep-relevant features captured from the input EEG signal. At the sequence level, the attention scores are visualized as the influence of different neighboring epochs in an input sequence (i.e. the context) to recognition of a target epoch, mimicking the way manual scoring is done by human experts.

CONCLUSION

Additionally, we demonstrate that SleepTransformer performs on par with existing methods on two databases of different sizes.

SIGNIFICANCE

Equipped with interpretability and the ability of uncertainty quantification, SleepTransformer holds promise for being integrated into clinical settings.

Unsupervised Domain Adaptation by Statistics Alignment for Deep Sleep Staging Networks

Deep sleep staging networks have reached top performance on large-scale datasets. However, these models perform poorer when training and testing on small sleep cohorts due to data inefficiency. Transferring well-trained models from large-scale datasets (source domain) to small sleep cohorts (target domain) is a promising solution but still remains challenging due to the domain-shift issue. In this work, an unsupervised domain adaptation approach, domain statistics alignment (DSA), is developed to bridge the gap between the data distribution of source and target domains. DSA adapts the source models on the target domain by modulating the domain-specific statistics of deep features stored in the Batch Normalization (BN) layers. Furthermore, we have extended DSA by introducing cross-domain statistics in each BN layer to perform DSA adaptively (AdaDSA). The proposed methods merely need the well-trained source model without access to the source data, which may be proprietary and inaccessible. DSA and AdaDSA are universally applicable to various deep sleep staging networks that have BN layers. We have validated the proposed methods by extensive experiments on two state-of-the-art deep sleep staging networks, DeepSleepNet+ and U-time. The performance was evaluated by conducting various transfer tasks on six sleep databases, including two large-scale databases, MASS and SHHS, as the source domain, four small sleep databases as the target domain. Thereinto, clinical sleep records acquired in Huashan Hospital, Shanghai, were used. The results show that both DSA and AdaDSA could significantly improve the performance of source models on target domains, providing novel insights into the domain generalization problem in sleep staging tasks.