1
|
Quantifying and Maximizing the Information Flux in Recurrent Neural Networks. Neural Comput 2024; 36:351-384. [PMID: 38363658 DOI: 10.1162/neco_a_01651] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2023] [Accepted: 12/04/2023] [Indexed: 02/18/2024]
Abstract
Free-running recurrent neural networks (RNNs), especially probabilistic models, generate an ongoing information flux that can be quantified with the mutual information I[x→(t),x→(t+1)] between subsequent system states x→. Although previous studies have shown that I depends on the statistics of the network's connection weights, it is unclear how to maximize I systematically and how to quantify the flux in large systems where computing the mutual information becomes intractable. Here, we address these questions using Boltzmann machines as model systems. We find that in networks with moderately strong connections, the mutual information I is approximately a monotonic transformation of the root-mean-square averaged Pearson correlations between neuron pairs, a quantity that can be efficiently computed even in large systems. Furthermore, evolutionary maximization of I[x→(t),x→(t+1)] reveals a general design principle for the weight matrices enabling the systematic construction of systems with a high spontaneous information flux. Finally, we simultaneously maximize information flux and the mean period length of cyclic attractors in the state-space of these dynamical networks. Our results are potentially useful for the construction of RNNs that serve as short-time memories or pattern generators.
Collapse
|
2
|
Predictive coding and stochastic resonance as fundamental principles of auditory phantom perception. Brain 2023; 146:4809-4825. [PMID: 37503725 PMCID: PMC10690027 DOI: 10.1093/brain/awad255] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2022] [Revised: 06/27/2023] [Accepted: 07/15/2023] [Indexed: 07/29/2023] Open
Abstract
Mechanistic insight is achieved only when experiments are employed to test formal or computational models. Furthermore, in analogy to lesion studies, phantom perception may serve as a vehicle to understand the fundamental processing principles underlying healthy auditory perception. With a special focus on tinnitus-as the prime example of auditory phantom perception-we review recent work at the intersection of artificial intelligence, psychology and neuroscience. In particular, we discuss why everyone with tinnitus suffers from (at least hidden) hearing loss, but not everyone with hearing loss suffers from tinnitus. We argue that intrinsic neural noise is generated and amplified along the auditory pathway as a compensatory mechanism to restore normal hearing based on adaptive stochastic resonance. The neural noise increase can then be misinterpreted as auditory input and perceived as tinnitus. This mechanism can be formalized in the Bayesian brain framework, where the percept (posterior) assimilates a prior prediction (brain's expectations) and likelihood (bottom-up neural signal). A higher mean and lower variance (i.e. enhanced precision) of the likelihood shifts the posterior, evincing a misinterpretation of sensory evidence, which may be further confounded by plastic changes in the brain that underwrite prior predictions. Hence, two fundamental processing principles provide the most explanatory power for the emergence of auditory phantom perceptions: predictive coding as a top-down and adaptive stochastic resonance as a complementary bottom-up mechanism. We conclude that both principles also play a crucial role in healthy auditory perception. Finally, in the context of neuroscience-inspired artificial intelligence, both processing principles may serve to improve contemporary machine learning techniques.
Collapse
|
3
|
Age-integrated artificial intelligence framework for sleep stage classification and obstructive sleep apnea screening. Front Neurosci 2023; 17:1059186. [PMID: 37389364 PMCID: PMC10300414 DOI: 10.3389/fnins.2023.1059186] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2022] [Accepted: 05/03/2023] [Indexed: 07/01/2023] Open
Abstract
Introduction Sleep is an essential function to sustain a healthy life, and sleep dysfunction can cause various physical and mental issues. In particular, obstructive sleep apnea (OSA) is one of the most common sleep disorders and, if not treated in a timely manner, OSA can lead to critical problems such as hypertension or heart disease. Methods The first crucial step in evaluating individuals' quality of sleep and diagnosing sleep disorders is to classify sleep stages using polysomnographic (PSG) data including electroencephalography (EEG). To date, such sleep stage scoring has been mainly performed manually via visual inspection by experts, which is not only a time-consuming and laborious process but also may yield subjective results. Therefore, we have developed a computational framework that enables automatic sleep stage classification utilizing the power spectral density (PSD) features of sleep EEG based on three different learning algorithms: support vector machine, k-nearest neighbors, and multilayer perceptron (MLP). In particular, we propose an integrated artificial intelligence (AI) framework to further inform the risk of OSA based on the characteristics in automatically scored sleep stages. Given the previous finding that the characteristics of sleep EEG differ by age group, we employed a strategy of training age-specific models (younger and older groups) and a general model and comparing their performance. Results The performance of the younger age-specific group model was similar to that of the general model (and even higher than the general model at certain stages), but the performance of the older age-specific group model was rather low, suggesting that bias in individual variables, such as age bias, should be considered during model training. Our integrated model yielded an accuracy of 73% in sleep stage classification and 73% in OSA screening when MLP algorithm was applied, which indicates that patients with OSA could be screened with the corresponding accuracy level only with sleep EEG without respiration-related measures. Discussion The current outcomes demonstrate the feasibility of AI-based computational studies that when combined with advances in wearable devices and relevant technologies could contribute to personalized medicine by not only assessing an individuals' sleep status conveniently at home but also by alerting them to the risk of sleep disorders and enabling early intervention.
Collapse
|
4
|
Extracting continuous sleep depth from EEG data without machine learning. Neurobiol Sleep Circadian Rhythms 2023; 14:100097. [PMID: 37275555 PMCID: PMC10238579 DOI: 10.1016/j.nbscr.2023.100097] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2023] [Revised: 04/28/2023] [Accepted: 05/02/2023] [Indexed: 06/07/2023] Open
Abstract
The human sleep-cycle has been divided into discrete sleep stages that can be recognized in electroencephalographic (EEG) and other bio-signals by trained specialists or machine learning systems. It is however unclear whether these human-defined stages can be re-discovered with unsupervised methods of data analysis, using only a minimal amount of generic pre-processing. Based on EEG data, recorded overnight from sleeping human subjects, we investigate the degree of clustering of the sleep stages using the General Discrimination Value as a quantitative measure of class separability. Virtually no clustering is found in the raw data, even after transforming the EEG signals of each 30-s epoch from the time domain into the more informative frequency domain. However, a Principal Component Analysis (PCA) of these epoch-wise frequency spectra reveals that the sleep stages separate significantly better in the low-dimensional sub-space of certain PCA components. In particular the component C1(t) can serve as a robust, continuous 'master variable' that encodes the depth of sleep and therefore correlates strongly with the 'hypnogram', a common plot of the discrete sleep stages over time. Moreover, C1(t) shows persistent trends during extended time periods where the sleep stage is constant, suggesting that sleep may be better understood as a continuum. These intriguing properties of C1(t) are not only relevant for understanding brain dynamics during sleep, but might also be exploited in low-cost single-channel sleep tracking devices for private and clinical use.
Collapse
|
5
|
Neural network based formation of cognitive maps of semantic spaces and the putative emergence of abstract concepts. Sci Rep 2023; 13:3644. [PMID: 36871003 PMCID: PMC9985610 DOI: 10.1038/s41598-023-30307-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2022] [Accepted: 02/21/2023] [Indexed: 03/06/2023] Open
Abstract
How do we make sense of the input from our sensory organs, and put the perceived information into context of our past experiences? The hippocampal-entorhinal complex plays a major role in the organization of memory and thought. The formation of and navigation in cognitive maps of arbitrary mental spaces via place and grid cells can serve as a representation of memories and experiences and their relations to each other. The multi-scale successor representation is proposed to be the mathematical principle underlying place and grid cell computations. Here, we present a neural network, which learns a cognitive map of a semantic space based on 32 different animal species encoded as feature vectors. The neural network successfully learns the similarities between different animal species, and constructs a cognitive map of 'animal space' based on the principle of successor representations with an accuracy of around 30% which is near to the theoretical maximum regarding the fact that all animal species have more than one possible successor, i.e. nearest neighbor in feature space. Furthermore, a hierarchical structure, i.e. different scales of cognitive maps, can be modeled based on multi-scale successor representations. We find that, in fine-grained cognitive maps, the animal vectors are evenly distributed in feature space. In contrast, in coarse-grained maps, animal vectors are highly clustered according to their biological class, i.e. amphibians, mammals and insects. This could be a putative mechanism enabling the emergence of new, abstract semantic concepts. Finally, even completely new or incomplete input can be represented by interpolation of the representations from the cognitive map with remarkable high accuracy of up to 95%. We conclude that the successor representation can serve as a weighted pointer to past memories and experiences, and may therefore be a crucial building block to include prior knowledge, and to derive context knowledge from novel input. Thus, our model provides a new tool to complement contemporary deep learning approaches on the road towards artificial general intelligence.
Collapse
|
6
|
Neural correlates of linguistic collocations during continuous speech perception. Front Psychol 2022; 13:1076339. [PMID: 36619132 PMCID: PMC9822706 DOI: 10.3389/fpsyg.2022.1076339] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2022] [Accepted: 12/02/2022] [Indexed: 12/25/2022] Open
Abstract
Language is fundamentally predictable, both on a higher schematic level as well as low-level lexical items. Regarding predictability on a lexical level, collocations are frequent co-occurrences of words that are often characterized by high strength of association. So far, psycho- and neurolinguistic studies have mostly employed highly artificial experimental paradigms in the investigation of collocations by focusing on the processing of single words or isolated sentences. In contrast, here we analyze EEG brain responses recorded during stimulation with continuous speech, i.e., audio books. We find that the N400 response to collocations is significantly different from that of non-collocations, whereas the effect varies with respect to cortical region (anterior/posterior) and laterality (left/right). Our results are in line with studies using continuous speech, and they mostly contradict those using artificial paradigms and stimuli. To the best of our knowledge, this is the first neurolinguistic study on collocations using continuous speech stimulation.
Collapse
|
7
|
Classification at the accuracy limit: facing the problem of data ambiguity. Sci Rep 2022; 12:22121. [PMID: 36543849 PMCID: PMC9772417 DOI: 10.1038/s41598-022-26498-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2022] [Accepted: 12/15/2022] [Indexed: 12/24/2022] Open
Abstract
Data classification, the process of analyzing data and organizing it into categories or clusters, is a fundamental computing task of natural and artificial information processing systems. Both supervised classification and unsupervised clustering work best when the input vectors are distributed over the data space in a highly non-uniform way. These tasks become however challenging in weakly structured data sets, where a significant fraction of data points is located in between the regions of high point density. We derive the theoretical limit for classification accuracy that arises from this overlap of data categories. By using a surrogate data generation model with adjustable statistical properties, we show that sufficiently powerful classifiers based on completely different principles, such as perceptrons and Bayesian models, all perform at this universal accuracy limit under ideal training conditions. Remarkably, the accuracy limit is not affected by certain non-linear transformations of the data, even if these transformations are non-reversible and drastically reduce the information content of the input data. We further compare the data embeddings that emerge by supervised and unsupervised training, using the MNIST data set and human EEG recordings during sleep. We find for MNIST that categories are significantly separated not only after supervised training with back-propagation, but also after unsupervised dimensionality reduction. A qualitatively similar cluster enhancement by unsupervised compression is observed for the EEG sleep data, but with a very small overall degree of cluster separation. We conclude that the handwritten letters in MNIST can be considered as 'natural kinds', whereas EEG sleep recordings are a relatively weakly structured data set, so that unsupervised clustering will not necessarily re-cover the human-defined sleep stages.
Collapse
|
8
|
Convulsant doses of abused synthetic cannabinoid receptor agonists AB-PINACA, 5F-AB-PINACA, 5F-ADB-PINACA and JWH-018 do not elicit electroencephalographic (EEG) seizures in male mice. Psychopharmacology (Berl) 2022; 239:3237-3248. [PMID: 35933518 DOI: 10.1007/s00213-022-06205-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/05/2022] [Accepted: 07/26/2022] [Indexed: 10/15/2022]
Abstract
RATIONALE Synthetic cannabinoid receptor agonists (SCRAs) are found in illicit smoking products, such as "K2" or "Spice." Convulsions are commonly reported adverse effects of SCRAs but are poorly understood. OBJECTIVES We determined convulsant effects of SCRAs AB-PINACA, and 5F-ADB-PINACA in adult male NIH Swiss mice, and then determined if convulsant effects of AB-PINACA, 5F-AB-PINACA, 5F-ADB-PINACA, and JWH-018 elicited seizure-like effects using EEG. METHODS Mice were administered SCRAs or pentylenetetrazole (PTZ) and placed in observation chambers where convulsant effects were scored. The capacity of the CB1R antagonist rimonabant, the benzodiazepine diazepam, or the non-specific CYP450 inhibitor 1-aminobenzotriazole (1-ABT) to attenuate convulsant effects was determined. Other mice were prepared with EEG headmounts to ascertain whether observed convulsions occurred concurrently with seizure-like effects by assessing root-mean-square (RMS) power, high amplitude EEG spike analysis, and videography. RESULTS Mice receiving AB-PINACA or 5F-ADB-PINACA exhibited dose-dependent convulsant effects that were blocked by 10 mg/kg rimonabant pretreatment but not by pretreatment with 10 mg/kg diazepam; these convulsant effects were not altered in the presence of 100 mg/kg 1-ABT. Repeated administration of 10 mg/kg AB-PINACA and 3 mg/kg 5F-ADB-PINACA produced partial tolerance to convulsant effects but did not lead to cross-tolerance to PTZ-induced convulsions. In EEG studies, convulsant doses of AB-PINACA, 5F-AB-PINACA, 5F-ADB-PINACA, and JWH-018 did not produce seizures concomitantly with convulsions. CONCLUSIONS These data extend previous findings of convulsant effects of SCRAs and suggest that convulsant effects of AB-PINACA, 5F-AB-PINACA, 5F-ADB-PINACA, and JWH-018 are CB1R-mediated but are not associated with electroencephalographic seizures. These results further suggest that benzodiazepines may not effectively treat convulsions elicited by SCRA use in humans.
Collapse
|
9
|
Neural network based successor representations to form cognitive maps of space and language. Sci Rep 2022; 12:11233. [PMID: 35787659 PMCID: PMC9253065 DOI: 10.1038/s41598-022-14916-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2022] [Accepted: 06/15/2022] [Indexed: 11/21/2022] Open
Abstract
How does the mind organize thoughts? The hippocampal-entorhinal complex is thought to support domain-general representation and processing of structural knowledge of arbitrary state, feature and concept spaces. In particular, it enables the formation of cognitive maps, and navigation on these maps, thereby broadly contributing to cognition. It has been proposed that the concept of multi-scale successor representations provides an explanation of the underlying computations performed by place and grid cells. Here, we present a neural network based approach to learn such representations, and its application to different scenarios: a spatial exploration task based on supervised learning, a spatial navigation task based on reinforcement learning, and a non-spatial task where linguistic constructions have to be inferred by observing sample sentences. In all scenarios, the neural network correctly learns and approximates the underlying structure by building successor representations. Furthermore, the resulting neural firing patterns are strikingly similar to experimentally observed place and grid cell firing patterns. We conclude that cognitive maps and neural network-based successor representations of structured knowledge provide a promising way to overcome some of the short comings of deep learning towards artificial general intelligence.
Collapse
|
10
|
Sleep staging classification based on a new parallel fusion method of multiple sources signals. Physiol Meas 2022; 43. [PMID: 35381584 DOI: 10.1088/1361-6579/ac647b] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2021] [Accepted: 04/05/2022] [Indexed: 11/12/2022]
Abstract
APPROACH First, the heart rate variability (HRV) is extracted from EOG with the Weight Calculation Algorithm (WCA) and an "HYF" RR interval detection algorithm. Second, three feature sets were extracted from HRV segments and EOG segments: time-domain features, frequency domain features and nonlinear-domain features. The frequency domain features and nonlinear-domain features were extracted by using Discrete Wavelet Transform (DWT), Autoregressive (AR), and Power Spectral entropy (PSE), and Refined Composite Multiscale Dispersion Entropy (RCMDE). Third, a new "Parallel Fusion Method" (PFM) for sleep stage classification is proposed. Three kinds of feature sets from EOG and HRV segments are fused by using PFM. Fourth, Extreme Gradient Boosting (XGBoost) and Support Vector Machine (SVM) classification models is employed for sleep staging. MAIN RESULTS Our experimental results show significant performance improvement on automatic sleep staging on the target domains achieved with the new sleep staging approach. The performance of the proposed method is testedby evaluating the average accuracy, Kappa coefficient. The average accuracy of sleep classification results by using XGBoost classification model with PFM is 82.7% and the kappa coefficient is 0.711, also by using SVM classification model with the PFM is 83.7%, and the kappa coefficient is 0.724. Experimental results show that the performance of the proposed method is competitive with the most current methods and results, and the recognition rate of S1 stage is significantly improved. Significance: As a consequence, it would enable one to improve the quality of automatic sleep staging models when the EOG and HRV signals are fused, which can be beneficial for monitor sleep quality and keep abreast of health conditions. Besides, our study provides good research ideas and methods for scholars, doctors and individuals.
Collapse
|
11
|
Sleep as a random walk: a super-statistical analysis of EEG data across sleep stages. Commun Biol 2021; 4:1385. [PMID: 34893700 PMCID: PMC8664947 DOI: 10.1038/s42003-021-02912-6] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2021] [Accepted: 11/23/2021] [Indexed: 11/15/2022] Open
Abstract
In clinical practice, human sleep is classified into stages, each associated with different levels of muscular activity and marked by characteristic patterns in the EEG signals. It is however unclear whether this subdivision into discrete stages with sharply defined boundaries is truly reflecting the dynamics of human sleep. To address this question, we consider one-channel EEG signals as heterogeneous random walks: stochastic processes controlled by hyper-parameters that are themselves time-dependent. We first demonstrate the heterogeneity of the random process by showing that each sleep stage has a characteristic distribution and temporal correlation function of the raw EEG signals. Next, we perform a super-statistical analysis by computing hyper-parameters, such as the standard deviation, kurtosis, and skewness of the raw signal distributions, within subsequent 30-second epochs. It turns out that also the hyper-parameters have characteristic, sleep-stage-dependent distributions, which can be exploited for a simple Bayesian sleep stage detection. Moreover, we find that the hyper-parameters are not piece-wise constant, as the traditional hypnograms would suggest, but show rising or falling trends within and across sleep stages, pointing to an underlying continuous rather than sub-divided process that controls human sleep. Based on the hyper-parameters, we finally perform a pairwise similarity analysis between the different sleep stages, using a quantitative measure for the separability of data clusters in multi-dimensional spaces. To improve our understand of how EEG activity reflects the dynamics of human sleep, Metzner et al. use human EEG data and superstatistical analysis to demonstrate that each sleep stage has a characteristic distribution and temporal correlation function of raw EEG signals. They also show that the hyper-parameters controlling the EEG signals have characteristic, sleep-stage-dependent distributions, which can be exploited for a simple Bayesian sleep stage detection.
Collapse
|
12
|
|
13
|
An Online Data Visualization Feedback Protocol for Motor Imagery-Based BCI Training. Front Hum Neurosci 2021; 15:625983. [PMID: 34163337 PMCID: PMC8215169 DOI: 10.3389/fnhum.2021.625983] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2020] [Accepted: 04/29/2021] [Indexed: 11/13/2022] Open
Abstract
Brain-computer interface (BCI) has developed rapidly over the past two decades, mainly due to advancements in machine learning. Subjects must learn to modulate their brain activities to ensure a successful BCI. Feedback training is a practical approach to this learning process; however, the commonly used classifier-dependent approaches have inherent limitations such as the need for calibration and a lack of continuous feedback over long periods of time. This paper proposes an online data visualization feedback protocol that intuitively reflects the EEG distribution in Riemannian geometry in real time. Rather than learning a hyperplane, the Riemannian geometry formulation allows iterative learning of prototypical covariance matrices that are translated into visualized feedback through diffusion map process. Ten subjects were recruited for MI-BCI (motor imagery-BCI) training experiments. The subjects learned to modulate their sensorimotor rhythm to centralize the points within one category and to separate points belonging to different categories. The results show favorable overall training effects in terms of the class distinctiveness and EEG feature discriminancy over a 3-day training with 30% learners. A steadily increased class distinctiveness in the last three sessions suggests that the advanced training protocol is effective. The optimal frequency band was consistent during the 3-day training, and the difference between subjects with good or low MI-BCI performance could be clearly observed. We believe that the proposed feedback protocol has promising application prospect.
Collapse
|
14
|
Analysis and visualization of sleep stages based on deep neural networks. Neurobiol Sleep Circadian Rhythms 2021; 10:100064. [PMID: 33763623 PMCID: PMC7973384 DOI: 10.1016/j.nbscr.2021.100064] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2020] [Revised: 02/27/2021] [Accepted: 03/01/2021] [Indexed: 11/28/2022] Open
Abstract
Automatic sleep stage scoring based on deep neural networks has come into focus of sleep researchers and physicians, as a reliable method able to objectively classify sleep stages would save human resources and simplify clinical routines. Due to novel open-source software libraries for machine learning, in combination with enormous recent progress in hardware development, a paradigm shift in the field of sleep research towards automatic diagnostics might be imminent. We argue that modern machine learning techniques are not just a tool to perform automatic sleep stage classification, but are also a creative approach to find hidden properties of sleep physiology. We have already developed and established algorithms to visualize and cluster EEG data, facilitating first assessments on sleep health in terms of sleep-apnea and consequently reduced daytime vigilance. In the following study, we further analyze cortical activity during sleep by determining the probabilities of momentary sleep stages, represented as hypnodensity graphs and then computing vectorial cross-correlations of different EEG channels. We can show that this measure serves to estimate the period length of sleep cycles and thus can help to find disturbances due to pathological conditions.
Collapse
|
15
|
Quantifying the separability of data classes in neural networks. Neural Netw 2021; 139:278-293. [PMID: 33862387 DOI: 10.1016/j.neunet.2021.03.035] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2020] [Revised: 03/23/2021] [Accepted: 03/24/2021] [Indexed: 11/18/2022]
Abstract
We introduce the Generalized Discrimination Value (GDV) that measures, in a non-invasive manner, how well different data classes separate in each given layer of an artificial neural network. It turns out that, at the end of the training period, the GDV in each given layer L attains a highly reproducible value, irrespective of the initialization of the network's connection weights. In the case of multi-layer perceptrons trained with error backpropagation, we find that classification of highly complex data sets requires a temporal reduction of class separability, marked by a characteristic 'energy barrier' in the initial part of the GDV(L) curve. Even more surprisingly, for a given data set, the GDV(L) is running through a fixed 'master curve', independently from the total number of network layers. Finally, due to its invariance with respect to dimensionality, the GDV may serve as a useful tool to compare the internal representational dynamics of artificial neural networks with different architectures for neural architecture search or network compression; or even with brain activity in order to decide between different candidate models of brain function.
Collapse
|
16
|
EEG-Based Sleep Staging Analysis with Functional Connectivity. SENSORS (BASEL, SWITZERLAND) 2021; 21:1988. [PMID: 33799850 PMCID: PMC7999974 DOI: 10.3390/s21061988] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/29/2020] [Revised: 02/26/2021] [Accepted: 03/08/2021] [Indexed: 12/20/2022]
Abstract
Sleep staging is important in sleep research since it is the basis for sleep evaluation and disease diagnosis. Related works have acquired many desirable outcomes. However, most of current studies focus on time-domain or frequency-domain measures as classification features using single or very few channels, which only obtain the local features but ignore the global information exchanging between different brain regions. Meanwhile, brain functional connectivity is considered to be closely related to brain activity and can be used to study the interaction relationship between brain areas. To explore the electroencephalography (EEG)-based brain mechanisms of sleep stages through functional connectivity, especially from different frequency bands, we applied phase-locked value (PLV) to build the functional connectivity network and analyze the brain interaction during sleep stages for different frequency bands. Then, we performed the feature-level, decision-level and hybrid fusion methods to discuss the performance of different frequency bands for sleep stages. The results show that (1) PLV increases in the lower frequency band (delta and alpha bands) and vice versa during different stages of non-rapid eye movement (NREM); (2) alpha band shows a better discriminative ability for sleeping stages; (3) the classification accuracy of feature-level fusion (six frequency bands) reaches 96.91% and 96.14% for intra-subject and inter-subjects respectively, which outperforms decision-level and hybrid fusion methods.
Collapse
|
17
|
Abstract
The question of whether artificial beings or machines could become self-aware or conscious has been a philosophical question for centuries. The main problem is that self-awareness cannot be observed from an outside perspective and the distinction of being really self-aware or merely a clever imitation cannot be answered without access to knowledge about the mechanism's inner workings. We investigate common machine learning approaches with respect to their potential ability to become self-aware. We realize that many important algorithmic steps toward machines with a core consciousness have already been taken.
Collapse
|
18
|
Electro-Encephalography and Electro-Oculography in Aeronautics: A Review Over the Last Decade (2010-2020). FRONTIERS IN NEUROERGONOMICS 2020; 1:606719. [PMID: 38234309 PMCID: PMC10790927 DOI: 10.3389/fnrgo.2020.606719] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/15/2020] [Accepted: 11/17/2020] [Indexed: 01/19/2024]
Abstract
Electro-encephalography (EEG) and electro-oculography (EOG) are methods of electrophysiological monitoring that have potentially fruitful applications in neuroscience, clinical exploration, the aeronautical industry, and other sectors. These methods are often the most straightforward way of evaluating brain oscillations and eye movements, as they use standard laboratory or mobile techniques. This review describes the potential of EEG and EOG systems and the application of these methods in aeronautics. For example, EEG and EOG signals can be used to design brain-computer interfaces (BCI) and to interpret brain activity, such as monitoring the mental state of a pilot in determining their workload. The main objectives of this review are to, (i) offer an in-depth review of literature on the basics of EEG and EOG and their application in aeronautics; (ii) to explore the methodology and trends of research in combined EEG-EOG studies over the last decade; and (iii) to provide methodological guidelines for beginners and experts when applying these methods in environments outside the laboratory, with a particular focus on human factors and aeronautics. The study used databases from scientific, clinical, and neural engineering fields. The review first introduces the characteristics and the application of both EEG and EOG in aeronautics, undertaking a large review of relevant literature, from early to more recent studies. We then built a novel taxonomy model that includes 150 combined EEG-EOG papers published in peer-reviewed scientific journals and conferences from January 2010 to March 2020. Several data elements were reviewed for each study (e.g., pre-processing, extracted features and performance metrics), which were then examined to uncover trends in aeronautics and summarize interesting methods from this important body of literature. Finally, the review considers the advantages and limitations of these methods as well as future challenges.
Collapse
|
19
|
Sleep stage classification for child patients using DeConvolutional Neural Network. Artif Intell Med 2020; 110:101981. [PMID: 33250147 DOI: 10.1016/j.artmed.2020.101981] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2020] [Revised: 10/08/2020] [Accepted: 10/27/2020] [Indexed: 02/08/2023]
Abstract
Studies from the literature show that the prevalence of sleep disorder in children is far higher than that in adults. Although much research effort has been made on sleep stage classification for adults, children have significantly different characteristics of sleep stages. Therefore, there is an urgent need for sleep stage classification targeting children in particular. Our method focuses on two issues: The first is timestamp-based segmentation (TSS) to deal with the fine-grained annotation of sleep stage labels for each timestamp. Compared to this, popular sliding window approaches unnecessarily aggregate such labels into coarse-grained ones. We utilize DeConvolutional Neural Network (DCNN) that inversely maps features of a hidden layer back to the input space to predict the sleep stage label at each timestamp. Thus, our DCNN can yield better classification performances by considering labels at numerous timestamps. The second issue is the necessity of multiple channels. Different clinical signs, symptoms or other auxiliary examinations could be represented by different Polysomnography (PSG) recordings, so all of them should be analyzed comprehensively. We therefor exploit multivariate time-series of PSG recordings, including 6 electroencephalograms (EEGs) channels, 2 electrooculograms (EOGs) channels (left and right), 1 electromyogram (chin EMG) channel and two leg electromyogram channels. Our DCNN-based method is tested on our SDCP dataset collected from child patients aged from 5 to 10 years old. The results show that our method yields the overall classification accuracy of 84.27% and macro F1-score of 72.51% which are higher than those of existing sliding window-based methods. One of the biggest advantages of our DCNN-based method is that it processes raw PSG recordings and internally extracts features useful for accurate sleep stage classification. We examine whether this is applicable for sleep data of adult patients by testing our method on a well-known public dataset Sleep-EDFX. Our method achieves the average overall accuracy of 90.89% which is comparable to those of state-of-the-art methods without using any hand-crafted features. This result indicates the great potential of our method because it can be generally used for timestamp-level classification on multivariate time-series in various medical fields. Additionally, we provide source codes so that researchers can reproduce the results in this paper and extend our method.
Collapse
|
20
|
Sleep State Classification Using Power Spectral Density and Residual Neural Network with Multichannel EEG Signals. APPLIED SCIENCES-BASEL 2020. [DOI: 10.3390/app10217639] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/01/2023]
Abstract
This paper proposes a classification framework for automatic sleep stage detection in both male and female human subjects by analyzing the electroencephalogram (EEG) data of polysomnography (PSG) recorded for three regions of the human brain, i.e., the pre-frontal, central, and occipital lobes. Without considering any artifact removal approach, the residual neural network (ResNet) architecture is used to automatically learn the distinctive features of different sleep stages from the power spectral density (PSD) of the raw EEG data. The residual block of the ResNet learns the intrinsic features of different sleep stages from the EEG data while avoiding the vanishing gradient problem. The proposed approach is validated using the sleep dataset of the Dreams database, which comprises of EEG signals for 20 healthy human subjects, 16 female and 4 male. Our experimental results demonstrate the effectiveness of the ResNet based approach in identifying different sleep stages in both female and male subjects compared to state-of-the-art methods with classification accuracies of 87.8% and 83.7%, respectively.
Collapse
|
21
|
An Autoencoder-based Approach to Predict Subjective Pain Perception from High-density Evoked EEG Potentials. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2020:1507-1511. [PMID: 33018277 DOI: 10.1109/embc44109.2020.9176644] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Pain is a subjective experience and clinicians need to treat patients with accurate pain levels. EEG has emerged as a useful tool for objective pain assessment, but due to the low signal-to-noise ratio of pain-related EEG signals, the prediction accuracy of EEG-based pain prediction models is still unsatisfactory. In this paper, we proposed an autoencoder model based on convolutional neural networks for feature extraction of pain-related EEG signals. More precisely, we used EEGNet to build an autoencoder model to extract a small set of features from high-density pain-evoked EEG potentials and then establish a machine learning models to predict pain levels (high pain vs. low pain) from extracted features. Experimental results show that the new autoencoder-based approach can effectively identify pain-related features and can achieve better classification results than conventional methods.
Collapse
|
22
|
Microstructure of cortical activity during sleep reflects respiratory events and state of daytime vigilance. SOMNOLOGIE 2019. [DOI: 10.1007/s11818-019-0201-0] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|