1
|
Wang J, Wang X, Ning X, Lin Y, Phan H, Jia Z. Subject-Adaptation Salient Wave Detection Network for Multimodal Sleep Stage Classification. IEEE J Biomed Health Inform 2025; 29:2172-2184. [PMID: 40030418 DOI: 10.1109/jbhi.2024.3512584] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/05/2025]
Abstract
Sleep stage classification is an important step in the diagnosis and treatment of sleep disorders. Despite the high classification performance of previous sleep stage classification work, some challenges remain unresolved: 1) How to effectively capture salient waves in sleep signals to improve sleep stage classification results. 2) How to capture salient waves affected by inter-subject variability. 3) How to adaptively regulate the importance of different modals for different sleep stages. To address these challenges, we propose SleepWaveNet, a multimodal salient wave detection network, which is motivated by the salient object detection task in computer vision. It has a U-Transformer structure to detect salient waves in sleep signals. Meanwhile, the subject-adaptation wave extraction architecture based on transfer learning can adapt to the information of target individuals and extract salient waves with inter-subject variability. In addition, the multimodal attention module can adaptively enhance the importance of specific modal data for sleep stage classification tasks. Experiments on three datasets show that SleepWaveNet has better overall performance than existing baselines. Moreover, visualization experiments show that the model has the ability to capture salient waves with inter-subject variability.
Collapse
|
2
|
Zhu Y, Wu Y, Wang Z, Zhou L, Chen C, Xu Z, Chen W. AFSleepNet: Attention-Based Multi-View Feature Fusion Framework for Pediatric Sleep Staging. IEEE Trans Neural Syst Rehabil Eng 2024; 32:4022-4032. [PMID: 39495693 DOI: 10.1109/tnsre.2024.3490757] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2024]
Abstract
The widespread prevalence of sleep problems in children highlights the importance of timely and accurate sleep staging in the diagnosis and treatment of pediatric sleep disorders. However, most existing sleep staging methods rely on one-dimensional raw polysomnograms or two-dimensional spectrograms, which omit critical details due to single-view processing. This shortcoming is particularly apparent in pediatric sleep staging, where the lack of a specialized network fails to meet the needs of precision medicine. Therefore, we introduce AFSleepNet, a novel attention-based multi-view feature fusion network tailored for pediatric sleep analysis. The model utilizes multimodal data (EEG, EOG, EMG), combining one-dimensional convolutional neural networks to extract time-invariant features and bidirectional-long-short-term memory to learn the transition rules among sleep stages, as well as employing short-time Fourier transform to generate two-dimensional spectral maps. This network employs a fusion method with self-attention mechanism and innovative pre-training strategy. This strategy can maintain the feature extraction capabilities of AFSleepNet from different views, enhancing the robustness of the multi-view model while effectively preventing model overfitting, thereby achieving efficient and accurate automatic sleep stage analysis. A "leave-one-subject-out" cross-validation on CHAT and clinical datasets demonstrated the excellent performance of AFSleepNet, with mean accuracies of 87.5% and 88.1%, respectively. Superiority over existing methods improves the accuracy and reliability of pediatric sleep staging.
Collapse
|
3
|
Phan H, Lorenzen KP, Heremans E, Chen OY, Tran MC, Koch P, Mertins A, Baumert M, Mikkelsen KB, De Vos M. L-SeqSleepNet: Whole-cycle Long Sequence Modeling for Automatic Sleep Staging. IEEE J Biomed Health Inform 2023; 27:4748-4757. [PMID: 37552591 DOI: 10.1109/jbhi.2023.3303197] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/10/2023]
Abstract
Human sleep is cyclical with a period of approximately 90 minutes, implying long temporal dependency in the sleep data. Yet, exploring this long-term dependency when developing sleep staging models has remained untouched. In this work, we show that while encoding the logic of a whole sleep cycle is crucial to improve sleep staging performance, the sequential modelling approach in existing state-of-the-art deep learning models are inefficient for that purpose. We thus introduce a method for efficient long sequence modelling and propose a new deep learning model, L-SeqSleepNet, which takes into account whole-cycle sleep information for sleep staging. Evaluating L-SeqSleepNet on four distinct databases of various sizes, we demonstrate state-of-the-art performance obtained by the model over three different EEG setups, including scalp EEG in conventional Polysomnography (PSG), in-ear EEG, and around-the-ear EEG (cEEGrid), even with a single EEG channel input. Our analyses also show that L-SeqSleepNet is able to alleviate the predominance of N2 sleep (the major class in terms of classification) to bring down errors in other sleep stages. Moreover the network becomes much more robust, meaning that for all subjects where the baseline method had exceptionally poor performance, their performance are improved significantly. Finally, the computation time only grows at a sub-linear rate when the sequence length increases.
Collapse
|
4
|
Liu J, Fan J, Zhong Z, Qiu H, Xiao J, Zhou Y, Zhu Z, Dai G, Wang N, Liu Q, Xie Y, Liu H, Chang L, Zhou J. An Ultra-Low Power Reconfigurable Biomedical AI Processor With Adaptive Learning for Versatile Wearable Intelligent Health Monitoring. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS 2023; 17:952-967. [PMID: 37192039 DOI: 10.1109/tbcas.2023.3276782] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/18/2023]
Abstract
Wearable intelligent health monitoring devices with on-device biomedical AI processor can be used to detect the abnormity in users' biomedical signals (e.g., ECG arrythmia classification, EEG-based seizure detection). This requires ultra-low power and reconfigurable biomedical AI processor to support battery-supplied wearable devices and versatile intelligent health monitoring applications while achieving high classification accuracy. However, existing designs have issues in meeting one or more of the above requirements. In this work, a reconfigurable biomedical AI processor (named BioAIP) is proposed, mainly featuring: 1) a reconfigurable biomedical AI processing architecture to support versatile biomedical AI processing. 2) an event-driven biomedical AI processing architecture with approximate data compression to reduce the power consumption. 3) an AI-based adaptive-learning architecture to address patient-to-patient variation and improve the classification accuracy. The design has been implemented and fabricated using a 65nm CMOS process technology. It has been demonstrated with three typical biomedical AI applications, including ECG arrythmia classification, EEG-based seizure detection and EMG-based hand gesture recognition. Compared with the state-of-the-art designs optimized for single biomedical AI tasks, the BioAIP achieves the lowest energy per classification among the designs with similar accuracy, while supporting various biomedical AI tasks.
Collapse
|
5
|
Xu F, Zhao J, Liu M, Yu X, Wang C, Lou Y, Shi W, Liu Y, Gao L, Yang Q, Zhang B, Lu S, Tang J, Leng J. Exploration of sleep function connection and classification strategies based on sub-period sleep stages. Front Neurosci 2023; 16:1088116. [PMID: 36760796 PMCID: PMC9906994 DOI: 10.3389/fnins.2022.1088116] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2022] [Accepted: 12/30/2022] [Indexed: 01/26/2023] Open
Abstract
Background As a medium for developing brain-computer interface systems, EEG signals are complex and difficult to identify due to their complexity, weakness, and differences between subjects. At present, most of the current research on sleep EEG signals are single-channel and dual-channel, ignoring the research on the relationship between different brain regions. Brain functional connectivity is considered to be closely related to brain activity and can be used to study the interaction relationship between brain areas. Methods Phase-locked value (PLV) is used to construct a functional connection network. The connection network is used to analyze the connection mechanism and brain interaction in different sleep stages. Firstly, the entire EEG signal is divided into multiple sub-periods. Secondly, Phase-locked value is used for feature extraction on the sub-periods. Thirdly, the PLV of multiple sub-periods is used for feature fusion. Fourthly, the classification performance optimization strategy is used to discuss the impact of different frequency bands on sleep stage classification performance and to find the optimal frequency band. Finally, the brain function network is constructed by using the average value of the fusion features to analyze the interaction of brain regions in different frequency bands during sleep stages. Results The experimental results have shown that when the number of sub-periods is 30, the α (8-13 Hz) frequency band has the best classification effect, The classification result after 10-fold cross-validation reaches 92.59%. Conclusion The proposed algorithm has good sleep staging performance, which can effectively promote the development and application of an EEG sleep staging system.
Collapse
Affiliation(s)
- Fangzhou Xu
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China,*Correspondence: Fangzhou Xu,
| | - Jinzhao Zhao
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
| | - Ming Liu
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
| | - Xin Yu
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
| | - Chongfeng Wang
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
| | - Yitai Lou
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
| | - Weiyou Shi
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
| | - Yanbing Liu
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
| | - Licai Gao
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
| | - Qingbo Yang
- School of Mathematics and Statistics, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China
| | - Baokun Zhang
- Department of Neurology, Shandong Institute of Neuroimmunology, Shandong Key Laboratory of Rheumatic Disease and Translational Medicine, The First Affliated Hospital of Shandong First Medical University, Shandong Provincial Qianfoshan Hospital, Jinan, China
| | - Shanshan Lu
- Department of Neurology, Shandong Institute of Neuroimmunology, Shandong Key Laboratory of Rheumatic Disease and Translational Medicine, The First Affliated Hospital of Shandong First Medical University, Shandong Provincial Qianfoshan Hospital, Jinan, China,Department of Neurology, Cheeloo College of Medicine, Shandong Qianfoshan Hospital, Shandong University, Jinan, Shandong, China,Shanshan Lu,
| | - Jiyou Tang
- Department of Neurology, Shandong Institute of Neuroimmunology, Shandong Key Laboratory of Rheumatic Disease and Translational Medicine, The First Affliated Hospital of Shandong First Medical University, Shandong Provincial Qianfoshan Hospital, Jinan, China,Department of Neurology, Cheeloo College of Medicine, Shandong Qianfoshan Hospital, Shandong University, Jinan, Shandong, China,Jiyou Tang,
| | - Jiancai Leng
- International School for Optoelectronic Engineering, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China,Jiancai Leng,
| |
Collapse
|
6
|
Dutt M, Redhu S, Goodwin M, Omlin CW. SleepXAI: An explainable deep learning approach for multi-class sleep stage identification. APPL INTELL 2022. [DOI: 10.1007/s10489-022-04357-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
Abstract
AbstractExtensive research has been conducted on the automatic classification of sleep stages utilizing deep neural networks and other neurophysiological markers. However, for sleep specialists to employ models as an assistive solution, it is necessary to comprehend how the models arrive at a particular outcome, necessitating the explainability of these models. This work proposes an explainable unified CNN-CRF approach (SleepXAI) for multi-class sleep stage classification designed explicitly for univariate time-series signals using modified gradient-weighted class activation mapping (Grad-CAM). The proposed approach significantly increases the overall accuracy of sleep stage classification while demonstrating the explainability of the multi-class labeling of univariate EEG signals, highlighting the parts of the signals emphasized most in predicting sleep stages. We extensively evaluated our approach to the sleep-EDF dataset, and it demonstrates the highest overall accuracy of 86.8% in identifying five sleep stage classes. More importantly, we achieved the highest accuracy when classifying the crucial sleep stage N1 with the lowest number of instances, outperforming the state-of-the-art machine learning approaches by 16.3%. These results motivate us to adopt the proposed approach in clinical practice as an aid to sleep experts.
Collapse
|
7
|
Phan H, Chen OY, Tran MC, Koch P, Mertins A, De Vos M. XSleepNet: Multi-View Sequential Model for Automatic Sleep Staging. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE 2022; 44:5903-5915. [PMID: 33788679 DOI: 10.1109/tpami.2021.3070057] [Citation(s) in RCA: 32] [Impact Index Per Article: 10.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
Automating sleep staging is vital to scale up sleep assessment and diagnosis to serve millions experiencing sleep deprivation and disorders and enable longitudinal sleep monitoring in home environments. Learning from raw polysomnography signals and their derived time-frequency image representations has been prevalent. However, learning from multi-view inputs (e.g., both the raw signals and the time-frequency images) for sleep staging is difficult and not well understood. This work proposes a sequence-to-sequence sleep staging model, XSleepNet,1 that is capable of learning a joint representation from both raw signals and time-frequency images. Since different views may generalize or overfit at different rates, the proposed network is trained such that the learning pace on each view is adapted based on their generalization/overfitting behavior. In simple terms, the learning on a particular view is speeded up when it is generalizing well and slowed down when it is overfitting. View-specific generalization/overfitting measures are computed on-the-fly during the training course and used to derive weights to blend the gradients from different views. As a result, the network is able to retain the representation power of different views in the joint features which represent the underlying distribution better than those learned by each individual view alone. Furthermore, the XSleepNet architecture is principally designed to gain robustness to the amount of training data and to increase the complementarity between the input views. Experimental results on five databases of different sizes show that XSleepNet consistently outperforms the single-view baselines and the multi-view baseline with a simple fusion strategy. Finally, XSleepNet also outperforms prior sleep staging methods and improves previous state-of-the-art results on the experimental databases.
Collapse
|
8
|
Multi-scale ResNet and BiGRU automatic sleep staging based on attention mechanism. PLoS One 2022; 17:e0269500. [PMID: 35709101 PMCID: PMC9202858 DOI: 10.1371/journal.pone.0269500] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2022] [Accepted: 05/20/2022] [Indexed: 11/29/2022] Open
Abstract
Sleep staging is the basis of sleep evaluation and a key step in the diagnosis of sleep-related diseases. Despite being useful, the existing sleep staging methods have several disadvantages, such as relying on artificial feature extraction, failing to recognize temporal sequence patterns in the long-term associated data, and reaching the accuracy upper limit of sleep staging. Hence, this paper proposes an automatic Electroencephalogram (EEG) sleep signal staging model, which based on Multi-scale Attention Residual Nets (MAResnet) and Bidirectional Gated Recurrent Unit (BiGRU). The proposed model is based on the residual neural network in deep learning. Compared with the traditional residual learning module, the proposed model additionally uses the improved channel and spatial feature attention units and convolution kernels of different sizes in parallel at the same position. Thus, multiscale feature extraction of the EEG sleep signals and residual learning of the neural networks is performed to avoid network degradation. Finally, BiGRU is used to determine the dependence between the sleep stages and to realize the automatic learning of sleep data staging features and sleep cycle extraction. According to the experiment, the classification accuracy and kappa coefficient of the proposed method on sleep-EDF data set are 84.24% and 0.78, which are respectively 0.24% and 0.21 higher than the traditional residual net. At the same time, this paper also verified the proposed method on UCD and SHHS data sets, and the figure of classification accuracy is 79.34% and 81.6%, respectively. Compared to related existing studies, the recognition accuracy is significantly improved, which validates the effectiveness and generalization performance of the proposed method.
Collapse
|
9
|
Phan H, Mikkelsen K. Automatic sleep staging of EEG signals: recent development, challenges, and future directions. Physiol Meas 2022; 43. [PMID: 35320788 DOI: 10.1088/1361-6579/ac6049] [Citation(s) in RCA: 47] [Impact Index Per Article: 15.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2021] [Accepted: 03/23/2022] [Indexed: 11/11/2022]
Abstract
Modern deep learning holds a great potential to transform clinical practice on human sleep. Teaching a machine to carry out routine tasks would be a tremendous reduction in workload for clinicians. Sleep staging, a fundamental step in sleep practice, is a suitable task for this and will be the focus in this article. Recently, automatic sleep staging systems have been trained to mimic manual scoring, leading to similar performance to human sleep experts, at least on scoring of healthy subjects. Despite tremendous progress, we have not seen automatic sleep scoring adopted widely in clinical environments. This review aims to give a shared view of the authors on the most recent state-of-the-art development in automatic sleep staging, the challenges that still need to be addressed, and the future directions for automatic sleep scoring to achieve clinical value.
Collapse
Affiliation(s)
- Huy Phan
- School of Electronic Engineering and Computer Science, Queen Mary University of London, Mile End Rd, London, E1 4NS, UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND
| | - Kaare Mikkelsen
- Department of Electrical and Computer Engineering, Aarhus Universitet, Finlandsgade 22, Aarhus, 8000, DENMARK
| |
Collapse
|
10
|
Phan H, Mikkelsen K, Chen OY, Koch P, Mertins A, De Vos M. SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantification. IEEE Trans Biomed Eng 2022; 69:2456-2467. [PMID: 35100107 DOI: 10.1109/tbme.2022.3147187] [Citation(s) in RCA: 66] [Impact Index Per Article: 22.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
BACKGROUND Black-box skepticism is one of the main hindrances impeding deep-learning-based automatic sleep scoring from being used in clinical environments. METHODS Towards interpretability, this work proposes a sequence-to-sequence sleep staging model, namely SleepTransformer. It is based on the transformer backbone and offers interpretability of the models decisions at both the epoch and sequence level. We further propose a simple yet efficient method to quantify uncertainty in the models decisions. The method, which is based on entropy, can serve as a metric for deferring low-confidence epochs to a human expert for further inspection. RESULTS Making sense of the transformers self-attention scores for interpretability, at the epoch level, the attention scores are encoded as a heat map to highlight sleep-relevant features captured from the input EEG signal. At the sequence level, the attention scores are visualized as the influence of different neighboring epochs in an input sequence (i.e. the context) to recognition of a target epoch, mimicking the way manual scoring is done by human experts. CONCLUSION Additionally, we demonstrate that SleepTransformer performs on par with existing methods on two databases of different sizes. SIGNIFICANCE Equipped with interpretability and the ability of uncertainty quantification, SleepTransformer holds promise for being integrated into clinical settings.
Collapse
|
11
|
Sun S, Li C, Lv N, Zhang X, Yu Z, Wang H. Attention based convolutional network for automatic sleep stage classification. ACTA ACUST UNITED AC 2021; 66:335-343. [PMID: 33544475 DOI: 10.1515/bmt-2020-0051] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/20/2020] [Accepted: 01/06/2021] [Indexed: 11/15/2022]
Abstract
Sleep staging is an important basis for diagnosing sleep-related problems. In this paper, an attention based convolutional network for automatic sleep staging is proposed. The network takes time-frequency image as input and predict sleep stage for each 30-s epoch as output. For each CNN feature maps, our model generate attention maps along two separate dimensions, time and filter, and then multiplied to form the final attention map. Residual-like fusion structure is used to append the attention map to the input feature map for adaptive feature refinement. In addition, to get the global feature representation with less information loss, the generalized mean pooling is introduced. To prove the efficacy of the proposed method, we have compared with two baseline method on sleep-EDF data set with different setting of the framework and input channel type, the experimental results show that the paper model has achieved significant improvements in terms of overall accuracy, Cohen's kappa, MF1, sensitivity and specificity. The performance of the proposed network is compared with that of the state-of-the-art algorithms with an overall accuracy of 83.4%, a macro F1-score of 77.3%, κ = 0.77, sensitivity = 77.1% and specificity = 95.4%, respectively. The experimental results demonstrate the superiority of the proposed network.
Collapse
Affiliation(s)
- Shasha Sun
- Shandong Provincial ENT Hospital, Cheeloo College of Medicine, Shandong University, Shandong, China
| | | | - Ning Lv
- Shandong Provincial ENT Hospital, Cheeloo College of Medicine, Shandong University, Shandong, China
| | - Xiaoman Zhang
- Shandong Provincial ENT Hospital, Cheeloo College of Medicine, Shandong University, Shandong, China
| | - Zhaoyan Yu
- Shandong Provincial ENT Hospital, Cheeloo College of Medicine, Shandong University, Shandong, China
| | - Haibo Wang
- Shandong Provincial ENT Hospital, Cheeloo College of Medicine, Shandong University, Shandong, China
| |
Collapse
|
12
|
A single-channel EEG based automatic sleep stage classification method leveraging deep one-dimensional convolutional neural network and hidden Markov model. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2021.102581] [Citation(s) in RCA: 19] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
|
13
|
Phan H, Chen OY, Koch P, Lu Z, McLoughlin I, Mertins A, De Vos M. Towards More Accurate Automatic Sleep Staging via Deep Transfer Learning. IEEE Trans Biomed Eng 2021; 68:1787-1798. [PMID: 32866092 DOI: 10.1109/tbme.2020.3020381] [Citation(s) in RCA: 48] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
BACKGROUND Despite recent significant progress in the development of automatic sleep staging methods, building a good model still remains a big challenge for sleep studies with a small cohort due to the data-variability and data-inefficiency issues. This work presents a deep transfer learning approach to overcome these issues and enable transferring knowledge from a large dataset to a small cohort for automatic sleep staging. METHODS We start from a generic end-to-end deep learning framework for sequence-to-sequence sleep staging and derive two networks as the means for transfer learning. The networks are first trained in the source domain (i.e. the large database). The pretrained networks are then finetuned in the target domain (i.e. the small cohort) to complete knowledge transfer. We employ the Montreal Archive of Sleep Studies (MASS) database consisting of 200 subjects as the source domain and study deep transfer learning on three different target domains: the Sleep Cassette subset and the Sleep Telemetry subset of the Sleep-EDF Expanded database, and the Surrey-cEEGrid database. The target domains are purposely adopted to cover different degrees of data mismatch to the source domains. RESULTS Our experimental results show significant performance improvement on automatic sleep staging on the target domains achieved with the proposed deep transfer learning approach. CONCLUSIONS These results suggest the efficacy of the proposed approach in addressing the above-mentioned data-variability and data-inefficiency issues. SIGNIFICANCE As a consequence, it would enable one to improve the quality of automatic sleep staging models when the amount of data is relatively small.11The source code and the pretrained models are published at https://github.com/pquochuy/sleep_transfer_learning.
Collapse
|
14
|
Neng W, Lu J, Xu L. CCRRSleepNet: A Hybrid Relational Inductive Biases Network for Automatic Sleep Stage Classification on Raw Single-Channel EEG. Brain Sci 2021; 11:456. [PMID: 33918506 PMCID: PMC8065855 DOI: 10.3390/brainsci11040456] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/31/2020] [Revised: 03/20/2021] [Accepted: 03/30/2021] [Indexed: 01/31/2023] Open
Abstract
In the inference process of existing deep learning models, it is usually necessary to process the input data level-wise, and impose a corresponding relational inductive bias on each level. This kind of relational inductive bias determines the theoretical performance upper limit of the deep learning method. In the field of sleep stage classification, only a single relational inductive bias is adopted at the same level in the mainstream methods based on deep learning. This will make the feature extraction method of deep learning incomplete and limit the performance of the method. In view of the above problems, a novel deep learning model based on hybrid relational inductive biases is proposed in this paper. It is called CCRRSleepNet. The model divides the single channel Electroencephalogram (EEG) data into three levels: frame, epoch, and sequence. It applies hybrid relational inductive biases from many aspects based on three levels. Meanwhile, multiscale atrous convolution block (MSACB) is adopted in CCRRSleepNet to learn the features of different attributes. However, in practice, the actual performance of the deep learning model depends on the nonrelational inductive biases, so a variety of matching nonrelational inductive biases are adopted in this paper to optimize CCRRSleepNet. The CCRRSleepNet is tested on the Fpz-Cz and Pz-Oz channel data of the Sleep-EDF dataset. The experimental results show that the method proposed in this paper is superior to many existing methods.
Collapse
Affiliation(s)
- Wenpeng Neng
- College of Computer Science and Technology, Heilongjiang University, Harbin 150080, China; (W.N.); (L.X.)
| | - Jun Lu
- College of Computer Science and Technology, Heilongjiang University, Harbin 150080, China; (W.N.); (L.X.)
- Key Laboratory of Database and Parallel Computing of Heilongjiang Province, Heilongjiang University, Harbin 150080, China
| | - Lei Xu
- College of Computer Science and Technology, Heilongjiang University, Harbin 150080, China; (W.N.); (L.X.)
| |
Collapse
|
15
|
Huang H, Zhang J, Zhu L, Tang J, Lin G, Kong W, Lei X, Zhu L. EEG-Based Sleep Staging Analysis with Functional Connectivity. SENSORS (BASEL, SWITZERLAND) 2021; 21:1988. [PMID: 33799850 PMCID: PMC7999974 DOI: 10.3390/s21061988] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/29/2020] [Revised: 02/26/2021] [Accepted: 03/08/2021] [Indexed: 12/20/2022]
Abstract
Sleep staging is important in sleep research since it is the basis for sleep evaluation and disease diagnosis. Related works have acquired many desirable outcomes. However, most of current studies focus on time-domain or frequency-domain measures as classification features using single or very few channels, which only obtain the local features but ignore the global information exchanging between different brain regions. Meanwhile, brain functional connectivity is considered to be closely related to brain activity and can be used to study the interaction relationship between brain areas. To explore the electroencephalography (EEG)-based brain mechanisms of sleep stages through functional connectivity, especially from different frequency bands, we applied phase-locked value (PLV) to build the functional connectivity network and analyze the brain interaction during sleep stages for different frequency bands. Then, we performed the feature-level, decision-level and hybrid fusion methods to discuss the performance of different frequency bands for sleep stages. The results show that (1) PLV increases in the lower frequency band (delta and alpha bands) and vice versa during different stages of non-rapid eye movement (NREM); (2) alpha band shows a better discriminative ability for sleeping stages; (3) the classification accuracy of feature-level fusion (six frequency bands) reaches 96.91% and 96.14% for intra-subject and inter-subjects respectively, which outperforms decision-level and hybrid fusion methods.
Collapse
Affiliation(s)
- Hui Huang
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou 310018, China; (H.H.); (J.Z.); (J.T.); (G.L.)
- Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou Dianzi University, Hangzhou 310018, China;
| | - Jianhai Zhang
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou 310018, China; (H.H.); (J.Z.); (J.T.); (G.L.)
- Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou Dianzi University, Hangzhou 310018, China;
| | - Li Zhu
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou 310018, China; (H.H.); (J.Z.); (J.T.); (G.L.)
- Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou Dianzi University, Hangzhou 310018, China;
| | - Jiajia Tang
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou 310018, China; (H.H.); (J.Z.); (J.T.); (G.L.)
- Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou Dianzi University, Hangzhou 310018, China;
| | - Guang Lin
- School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou 310018, China; (H.H.); (J.Z.); (J.T.); (G.L.)
- Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou Dianzi University, Hangzhou 310018, China;
| | - Wanzeng Kong
- Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou Dianzi University, Hangzhou 310018, China;
| | - Xu Lei
- Sleep and NeuroImaging Center, Faculty of Psychology, Southwest University, Chongqing 400715, China;
- Key Laboratory of Cognition and Personality, Ministry of Education, Chongqing 400715, China
| | - Lei Zhu
- School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China;
| |
Collapse
|
16
|
End-to-end sleep staging using convolutional neural network in raw single-channel EEG. Biomed Signal Process Control 2021. [DOI: 10.1016/j.bspc.2020.102203] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/27/2022]
|
17
|
Automated Detection of Sleep Stages Using Deep Learning Techniques: A Systematic Review of the Last Decade (2010–2020). APPLIED SCIENCES-BASEL 2020. [DOI: 10.3390/app10248963] [Citation(s) in RCA: 43] [Impact Index Per Article: 8.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
Abstract
Sleep is vital for one’s general well-being, but it is often neglected, which has led to an increase in sleep disorders worldwide. Indicators of sleep disorders, such as sleep interruptions, extreme daytime drowsiness, or snoring, can be detected with sleep analysis. However, sleep analysis relies on visuals conducted by experts, and is susceptible to inter- and intra-observer variabilities. One way to overcome these limitations is to support experts with a programmed diagnostic tool (PDT) based on artificial intelligence for timely detection of sleep disturbances. Artificial intelligence technology, such as deep learning (DL), ensures that data are fully utilized with low to no information loss during training. This paper provides a comprehensive review of 36 studies, published between March 2013 and August 2020, which employed DL models to analyze overnight polysomnogram (PSG) recordings for the classification of sleep stages. Our analysis shows that more than half of the studies employed convolutional neural networks (CNNs) on electroencephalography (EEG) recordings for sleep stage classification and achieved high performance. Our study also underscores that CNN models, particularly one-dimensional CNN models, are advantageous in yielding higher accuracies for classification. More importantly, we noticed that EEG alone is not sufficient to achieve robust classification results. Future automated detection systems should consider other PSG recordings, such as electroencephalogram (EEG), electrooculogram (EOG), and electromyogram (EMG) signals, along with input from human experts, to achieve the required sleep stage classification robustness. Hence, for DL methods to be fully realized as a practical PDT for sleep stage scoring in clinical applications, inclusion of other PSG recordings, besides EEG recordings, is necessary. In this respect, our report includes methods published in the last decade, underscoring the use of DL models with other PSG recordings, for scoring of sleep stages.
Collapse
|
18
|
Guillot A, Sauvet F, During EH, Thorey V. Dreem Open Datasets: Multi-Scored Sleep Datasets to Compare Human and Automated Sleep Staging. IEEE Trans Neural Syst Rehabil Eng 2020; 28:1955-1965. [PMID: 32746326 DOI: 10.1109/tnsre.2020.3011181] [Citation(s) in RCA: 36] [Impact Index Per Article: 7.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
Sleep stage classification constitutes an important element of sleep disorder diagnosis. It relies on the visual inspection of polysomnography records by trained sleep technologists. Automated approaches have been designed to alleviate this resource-intensive task. However, such approaches are usually compared to a single human scorer annotation despite an inter-rater agreement of about 85% only. The present study introduces two publicly-available datasets, DOD-H including 25 healthy volunteers and DOD-O including 55 patients suffering from obstructive sleep apnea (OSA). Both datasets have been scored by 5 sleep technologists from different sleep centers. We developed a framework to compare automated approaches to a consensus of multiple human scorers. Using this framework, we benchmarked and compared the main literature approaches to a new deep learning method, SimpleSleepNet, which reach state-of-the-art performances while being more lightweight. We demonstrated that many methods can reach human-level performance on both datasets. SimpleSleepNet achieved an F1 of 89.9% vs 86.8% on average for human scorers on DOD-H, and an F1 of 88.3% vs 84.8% on DOD-O. Our study highlights that state-of-the-art automated sleep staging outperforms human scorers performance for healthy volunteers and patients suffering from OSA. Considerations could be made to use automated approaches in the clinical setting.
Collapse
|
19
|
Zhu T, Luo W, Yu F. Convolution-and Attention-Based Neural Network for Automated Sleep Stage Classification. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2020; 17:ijerph17114152. [PMID: 32532084 PMCID: PMC7313068 DOI: 10.3390/ijerph17114152] [Citation(s) in RCA: 40] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/22/2020] [Revised: 06/08/2020] [Accepted: 06/09/2020] [Indexed: 11/23/2022]
Abstract
Analyzing polysomnography (PSG) is an effective method for evaluating sleep health; however, the sleep stage scoring required for PSG analysis is a time-consuming effort for an experienced medical expert. When scoring sleep epochs, experts pay attention to find specific signal characteristics (e.g., K-complexes and spindles), and sometimes need to integrate information from preceding and subsequent epochs in order to make a decision. To imitate this process and to build a more interpretable deep learning model, we propose a neural network based on a convolutional network (CNN) and attention mechanism to perform automatic sleep staging. The CNN learns local signal characteristics, and the attention mechanism excels in learning inter- and intra-epoch features. In experiments on the public sleep-edf and sleep-edfx databases with different training and testing set partitioning methods, our model achieved overall accuracies of 93.7% and 82.8%, and macro-average F1-scores of 84.5 and 77.8, respectively, outperforming recently reported machine learning-based methods.
Collapse
|
20
|
Phan H, Chen OY, Koch P, Mertins A, De Vos M. Fusion of End-to-End Deep Learning Models for Sequence-to-Sequence Sleep Staging. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2019:1829-1833. [PMID: 31946253 DOI: 10.1109/embc.2019.8857348] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Sleep staging, a process of identifying the sleep stages associated with polysomnography (PSG) epochs, plays an important role in sleep monitoring and diagnosing sleep disorders. We present in this work a model fusion approach to automate this task. The fusion model is composed of two base sleep-stage classifiers, SeqSleepNet and DeepSleepNet, both of which are state-of-the-art end-to-end deep learning models complying to the sequence-to-sequence sleep staging scheme. In addition, in the light of ensemble methods, we reason and demonstrate that these two networks form a good ensemble of models due to their high diversity. Experiments show that the fusion approach is able to preserve the strength of the base networks in the fusion model, leading to consistent performance gains over the two base networks. The fusion model obtain the best modelling results we have observed so far on the Montreal Archive of Sleep Studies (MASS) dataset with 200 subjects, achieving an overall accuracy of 88.0%, a macro F1-score of 84.3%, and a Cohen's kappa of 0.828.
Collapse
|
21
|
Rim B, Sung NJ, Min S, Hong M. Deep Learning in Physiological Signal Data: A Survey. SENSORS (BASEL, SWITZERLAND) 2020; 20:E969. [PMID: 32054042 PMCID: PMC7071412 DOI: 10.3390/s20040969] [Citation(s) in RCA: 77] [Impact Index Per Article: 15.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/07/2020] [Revised: 01/31/2020] [Accepted: 02/09/2020] [Indexed: 12/11/2022]
Abstract
Deep Learning (DL), a successful promising approach for discriminative and generative tasks, has recently proved its high potential in 2D medical imaging analysis; however, physiological data in the form of 1D signals have yet to be beneficially exploited from this novel approach to fulfil the desired medical tasks. Therefore, in this paper we survey the latest scientific research on deep learning in physiological signal data such as electromyogram (EMG), electrocardiogram (ECG), electroencephalogram (EEG), and electrooculogram (EOG). We found 147 papers published between January 2018 and October 2019 inclusive from various journals and publishers. The objective of this paper is to conduct a detailed study to comprehend, categorize, and compare the key parameters of the deep-learning approaches that have been used in physiological signal analysis for various medical applications. The key parameters of deep-learning approach that we review are the input data type, deep-learning task, deep-learning model, training architecture, and dataset sources. Those are the main key parameters that affect system performance. We taxonomize the research works using deep-learning method in physiological signal analysis based on: (1) physiological signal data perspective, such as data modality and medical application; and (2) deep-learning concept perspective such as training architecture and dataset sources.
Collapse
Affiliation(s)
- Beanbonyka Rim
- Department of Computer Science, Soonchunhyang University, Asan 31538, Korea
| | - Nak-Jun Sung
- Department of Computer Science, Soonchunhyang University, Asan 31538, Korea
| | - Sedong Min
- Department of Medical IT Engineering, Soonchunhyang University, Asan 31538, Korea
| | - Min Hong
- Department of Computer Software Engineering, Soonchunhyang University, Asan 31538, Korea
| |
Collapse
|
22
|
Phan H, Andreotti F, Cooray N, Chén OY, De Vos M. Joint Classification and Prediction CNN Framework for Automatic Sleep Stage Classification. IEEE Trans Biomed Eng 2019; 66:1285-1296. [PMID: 30346277 PMCID: PMC6487915 DOI: 10.1109/tbme.2018.2872652] [Citation(s) in RCA: 146] [Impact Index Per Article: 24.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2018] [Accepted: 09/22/2018] [Indexed: 11/07/2022]
Abstract
Correctly identifying sleep stages is important in diagnosing and treating sleep disorders. This paper proposes a joint classification-and-prediction framework based on convolutional neural networks (CNNs) for automatic sleep staging, and, subsequently, introduces a simple yet efficient CNN architecture to power the framework. Given a single input epoch, the novel framework jointly determines its label (classification) and its neighboring epochs' labels (prediction) in the contextual output. While the proposed framework is orthogonal to the widely adopted classification schemes, which take one or multiple epochs as contextual inputs and produce a single classification decision on the target epoch, we demonstrate its advantages in several ways. First, it leverages the dependency among consecutive sleep epochs while surpassing the problems experienced with the common classification schemes. Second, even with a single model, the framework has the capacity to produce multiple decisions, which are essential in obtaining a good performance as in ensemble-of-models methods, with very little induced computational overhead. Probabilistic aggregation techniques are then proposed to leverage the availability of multiple decisions. To illustrate the efficacy of the proposed framework, we conducted experiments on two public datasets: Sleep-EDF Expanded (Sleep-EDF), which consists of 20 subjects, and Montreal Archive of Sleep Studies (MASS) dataset, which consists of 200 subjects. The proposed framework yields an overall classification accuracy of 82.3% and 83.6%, respectively. We also show that the proposed framework not only is superior to the baselines based on the common classification schemes but also outperforms existing deep-learning approaches. To our knowledge, this is the first work going beyond the standard single-output classification to consider multitask neural networks for automatic sleep staging. This framework provides avenues for further studies of different neural-network architectures for automatic sleep staging.
Collapse
Affiliation(s)
- Huy Phan
- Institute of Biomedical EngineeringUniversity of OxfordOxfordOX3 7DQU.K.
| | | | - Navin Cooray
- Institute of Biomedical EngineeringUniversity of Oxford
| | | | | |
Collapse
|
23
|
Phan H. SeqSleepNet: End-to-End Hierarchical Recurrent Neural Network for Sequence-to-Sequence Automatic Sleep Staging. IEEE Trans Neural Syst Rehabil Eng 2019; 27:400-410. [PMID: 30716040 PMCID: PMC6481557 DOI: 10.1109/tnsre.2019.2896659] [Citation(s) in RCA: 205] [Impact Index Per Article: 34.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/26/2022]
Abstract
Automatic sleep staging has been often treated as a simple classification problem that aims at determining the label of individual target polysomnography epochs one at a time. In this paper, we tackle the task as a sequence-to-sequence classification problem that receives a sequence of multiple epochs as input and classifies all of their labels at once. For this purpose, we propose a hierarchical recurrent neural network named SeqSleepNet (source code is available at http://github.com/pquochuy/SeqSleepNet). At the epoch processing level, the network consists of a filterbank layer tailored to learn frequency-domain filters for preprocessing and an attention-based recurrent layer designed for short-term sequential modeling. At the sequence processing level, a recurrent layer placed on top of the learned epoch-wise features for long-term modeling of sequential epochs. The classification is then carried out on the output vectors at every time step of the top recurrent layer to produce the sequence of output labels. Despite being hierarchical, we present a strategy to train the network in an end-to-end fashion. We show that the proposed network outperforms the state-of-the-art approaches, achieving an overall accuracy, macro F1-score, and Cohen's kappa of 87.1%, 83.3%, and 0.815 on a publicly available dataset with 200 subjects.
Collapse
Affiliation(s)
- Huy Phan
- School of Computing, University of Kent, Chatham Maritime, Kent ME4 4AG, United Kingdom and the Institute of Biomedical Engineering, University of Oxford, Oxford OX3 7DQ, United Kingdom
| |
Collapse
|