1
|
Selcuk Nogay H, Adeli H. Diagnostic of autism spectrum disorder based on structural brain MRI images using, grid search optimization, and convolutional neural networks. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104234] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/14/2022]
|
2
|
Fu R, Xu D, Li W, Shi P. Single-trial motor imagery electroencephalogram intention recognition by optimal discriminant hyperplane and interpretable discriminative rectangle mixture model. Cogn Neurodyn 2022; 16:1073-1085. [PMID: 36237407 PMCID: PMC9508315 DOI: 10.1007/s11571-021-09768-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2021] [Revised: 11/24/2021] [Accepted: 12/05/2021] [Indexed: 11/03/2022] Open
Abstract
Spatial filtering is widely used in brain-computer interface (BCI) systems to augmented signal characteristics of electroencephalogram (EEG) signals. In this study, a spatial domain filtering based EEG feature extraction method, optimal discriminant hyperplane-common spatial subspace decomposition (ODH-CSSD) is proposed. Specifically, the multi-dimensional EEG features were extracted from the original EEG signals by common space subspace decomposition (CSSD) algorithm, and the optimal feature criterion was established to find the multi-dimensional optimal projection space. A classic method of data dimension optimizing is using the eigenvectors of a lumped covariance matrix corresponding to the maximum eigenvalues. Then, the cost function is defined as the extreme value of the discriminant criterion, and the orthogonal N discriminant vectors corresponding to the N extreme value of the criterion are solved and constructed into the N-dimensional optimal feature space. Finally, the multi-dimensional EEG features are projected into the N-dimensional optimal projection space to obtain the optimal N-dimensional EEG features. Moreover, this study involves the extraction of two-dimensional and three-dimensional optimal EEG features from motor imagery EEG datasets, and the optimal EEG features are identified using the interpretable discriminative rectangular mixture model (DRMM). Experimental results show that the accuracy of DRMM to identify two-dimensional optimal features is more than 0.91, and the highest accuracy even reaches 0.975. Meanwhile, DRMM has the most stable recognition accuracy for two-dimensional optimal features, and its average clustering accuracy reaches 0.942, the gap between the accuracy of the DRMM with the accuracy of the FCM and K-means can reach 0.26. And the optimal three-dimensional features, for most subjects, the clustering accuracy of DRMM is higher than that of FCM and K-means. In general, the decision rectangle obtained by DRMM can clearly explain the difference of each cluster, notably, the optimization of multidimensional EEG features by optimal projection is superior to Fisher's ratio, and this method provides an alternative for the application of BCI. Supplementary Information The online version contains supplementary material available at 10.1007/s11571-021-09768-w.
Collapse
Affiliation(s)
- Rongrong Fu
- Measurement Technology and Instrumentation Key Lab of Hebei Province, Yanshan University, Qinhuangdao, 066004 China
| | - Dong Xu
- Measurement Technology and Instrumentation Key Lab of Hebei Province, Yanshan University, Qinhuangdao, 066004 China
| | - Weishuai Li
- Measurement Technology and Instrumentation Key Lab of Hebei Province, Yanshan University, Qinhuangdao, 066004 China
| | - Peiming Shi
- Measurement Technology and Instrumentation Key Lab of Hebei Province, Yanshan University, Qinhuangdao, 066004 China
| |
Collapse
|
3
|
Wu X, Zheng WL, Li Z, Lu BL. Investigating EEG-based functional connectivity patterns for multimodal emotion recognition. J Neural Eng 2022; 19. [DOI: 10.1088/1741-2552/ac49a7] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2021] [Accepted: 01/10/2022] [Indexed: 02/04/2023]
Abstract
Abstract
Objective. Previous studies on emotion recognition from electroencephalography (EEG) mainly rely on single-channel-based feature extraction methods, which ignore the functional connectivity between brain regions. Hence, in this paper, we propose a novel emotion-relevant critical subnetwork selection algorithm and investigate three EEG functional connectivity network features: strength, clustering coefficient, and eigenvector centrality. Approach. After constructing the brain networks by the correlations between pairs of EEG signals, we calculated critical subnetworks through the average of brain network matrices with the same emotion label to eliminate the weak associations. Then, three network features were conveyed to a multimodal emotion recognition model using deep canonical correlation analysis along with eye movement features. The discrimination ability of the EEG connectivity features in emotion recognition is evaluated on three public datasets: SEED, SEED-V, and DEAP. Main results. The experimental results reveal that the strength feature outperforms the state-of-the-art features based on single-channel analysis. The classification accuracies of multimodal emotion recognition are
95.08
±
6.42
%
on the SEED dataset,
84.51
±
5.11
%
on the SEED-V dataset, and
85.34
±
2.90
%
and
86.61
±
3.76
%
for arousal and valence on the DEAP dataset, respectively, which all achieved the best performance. In addition, the brain networks constructed with 18 channels achieve comparable performance with that of the 62-channel network and enable easier setups in real scenarios. Significance. The EEG functional connectivity networks combined with emotion-relevant critical subnetworks selection algorithm we proposed is a successful exploration to excavate the information between channels.
Collapse
|
4
|
Ieracitano C, Morabito FC, Hussain A, Mammone N. A Hybrid-Domain Deep Learning-Based BCI For Discriminating Hand Motion Planning From EEG Sources. Int J Neural Syst 2021; 31:2150038. [PMID: 34376121 DOI: 10.1142/s0129065721500386] [Citation(s) in RCA: 25] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
In this paper, a hybrid-domain deep learning (DL)-based neural system is proposed to decode hand movement preparation phases from electroencephalographic (EEG) recordings. The system exploits information extracted from the temporal-domain and time-frequency-domain, as part of a hybrid strategy, to discriminate the temporal windows (i.e. EEG epochs) preceding hand sub-movements (open/close) and the resting state. To this end, for each EEG epoch, the associated cortical source signals in the motor cortex and the corresponding time-frequency (TF) maps are estimated via beamforming and Continuous Wavelet Transform (CWT), respectively. Two Convolutional Neural Networks (CNNs) are designed: specifically, the first CNN is trained over a dataset of temporal (T) data (i.e. EEG sources), and is referred to as T-CNN; the second CNN is trained over a dataset of TF data (i.e. TF-maps of EEG sources), and is referred to as TF-CNN. Two sets of features denoted as T-features and TF-features, extracted from T-CNN and TF-CNN, respectively, are concatenated in a single features vector (denoted as TTF-features vector) which is used as input to a standard multi-layer perceptron for classification purposes. Experimental results show a significant performance improvement of our proposed hybrid-domain DL approach as compared to temporal-only and time-frequency-only-based benchmark approaches, achieving an average accuracy of [Formula: see text]%.
Collapse
Affiliation(s)
- Cosimo Ieracitano
- DICEAM, University Mediterranea of Reggio Calabria, Via Graziella Feo di Vito, Reggio Calabria, 89124, Italy
| | - Francesco Carlo Morabito
- DICEAM, University Mediterranea of Reggio Calabria, Via Graziella Feo di Vito, Reggio Calabria, 89124, Italy
| | - Amir Hussain
- School of Computing, Edinburgh Napier University, Edinburgh EH10 5DT, Scotland, UK
| | - Nadia Mammone
- DICEAM, University Mediterranea of Reggio Calabria, Via Graziella Feo di Vito, Reggio Calabria, 89124, Italy
| |
Collapse
|
5
|
Bricher D, Müller A. Supervised Detection of Connector Lock Events with Optical Microphone Data. Int J Neural Syst 2021; 31:2150017. [PMID: 33752578 DOI: 10.1142/s0129065721500179] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
In manufacturing industry, one of the main targets is to increase automation and ultimately to avoid failures under all circumstances. The plugging and locking of connectors is a class of tasks which is yet hard to be automatized with sufficiently high process stability. Due to the variation of plugging positions and external disturbances, e.g. occlusion due to cables, the quality assessment of plugging processes has emerged as a challenging task for image-based systems. For this reason, the proposed approach analyzes the inherent acoustic connector locking properties in combination with different neural network architectures in order to correctly identify connector locking signals and further to distinguish them from other machining events occurring in assembly plants. For this specific task, highly sensitive optical microphones have been applied for data acquisition. The proposed experiments are carried out under laboratory conditions as well as for the more complex situation in a real manufacturing environment. In this context, the usage of multimodal neural network architectures achieved highest levels in classification performance with accuracy levels close to 90%.
Collapse
Affiliation(s)
- David Bricher
- Institute of Robotics, Johannes Kepler University, Altenberger Straße 69, 4040 Linz, Austria
| | - Andreas Müller
- Institute of Robotics, Johannes Kepler University, Altenberger Straße 69, 4040 Linz, Austria
| |
Collapse
|
6
|
Hajek P, Barushka A, Munk M. Neural Networks with Emotion Associations, Topic Modeling and Supervised Term Weighting for Sentiment Analysis. Int J Neural Syst 2021; 31:2150013. [PMID: 33573532 DOI: 10.1142/s0129065721500131] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Automated sentiment analysis is becoming increasingly recognized due to the growing importance of social media and e-commerce platform review websites. Deep neural networks outperform traditional lexicon-based and machine learning methods by effectively exploiting contextual word embeddings to generate dense document representation. However, this representation model is not fully adequate to capture topical semantics and the sentiment polarity of words. To overcome these problems, a novel sentiment analysis model is proposed that utilizes richer document representations of word-emotion associations and topic models, which is the main computational novelty of this study. The sentiment analysis model integrates word embeddings with lexicon-based sentiment and emotion indicators, including negations and emoticons, and to further improve its performance, a topic modeling component is utilized together with a bag-of-words model based on a supervised term weighting scheme. The effectiveness of the proposed model is evaluated using large datasets of Amazon product reviews and hotel reviews. Experimental results prove that the proposed document representation is valid for the sentiment analysis of product and hotel reviews, irrespective of their class imbalance. The results also show that the proposed model improves on existing machine learning methods.
Collapse
Affiliation(s)
- Petr Hajek
- Science and Research Centre, Faculty of Economics and Administration, University of Pardubice, Studentska 84, 532 10 Pardubice, Czech Republic
| | - Aliaksandr Barushka
- Science and Research Centre, Faculty of Economics and Administration, University of Pardubice, Studentska 84, 532 10 Pardubice, Czech Republic
| | - Michal Munk
- Science and Research Centre, Faculty of Economics and Administration, University of Pardubice, Studentska 84, 532 10 Pardubice, Czech Republic.,Department of Computer Science, Constantine the Philosopher University in Nitra, 949 74 Nitra, Slovakia
| |
Collapse
|
7
|
Nogay HS, Adeli H. Detection of Epileptic Seizure Using Pretrained Deep Convolutional Neural Network and Transfer Learning. Eur Neurol 2021; 83:602-614. [PMID: 33423031 DOI: 10.1159/000512985] [Citation(s) in RCA: 90] [Impact Index Per Article: 22.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2020] [Accepted: 11/11/2020] [Indexed: 11/19/2022]
Abstract
INTRODUCTION The diagnosis of epilepsy takes a certain process, depending entirely on the attending physician. However, the human factor may cause erroneous diagnosis in the analysis of the EEG signal. In the past 2 decades, many advanced signal processing and machine learning methods have been developed for the detection of epileptic seizures. However, many of these methods require large data sets and complex operations. METHODS In this study, an end-to-end machine learning model is presented for detection of epileptic seizure using the pretrained deep two-dimensional convolutional neural network (CNN) and the concept of transfer learning. The EEG signal is converted directly into visual data with a spectrogram and used directly as input data. RESULTS The authors analyzed the results of the training of the proposed pretrained AlexNet CNN model. Both binary and ternary classifications were performed without any extra procedure such as feature extraction. By performing data set creation from short-term spectrogram graphic images, the authors were able to achieve 100% accuracy for binary classification for epileptic seizure detection and 100% for ternary classification. DISCUSSION/CONCLUSION The proposed automatic identification and classification model can help in the early diagnosis of epilepsy, thus providing the opportunity for effective early treatment.
Collapse
Affiliation(s)
- Hidir Selcuk Nogay
- Department of Electrical and Energy, Kayseri University, Kayseri, Turkey
| | - Hojjat Adeli
- Departments of Biomedical Informatics and Neuroscience, The Ohio State University, Columbus, Ohio, USA,
| |
Collapse
|
8
|
Gong S, Xing K, Cichocki A, Li J. Deep Learning in EEG: Advance of the Last Ten-Year Critical Period. IEEE Trans Cogn Dev Syst 2021. [DOI: 10.1109/tcds.2021.3079712] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
9
|
Avola D, Cascio M, Cinque L, Fagioli A, Foresti GL. LieToMe: An Ensemble Approach for Deception Detection from Facial Cues. Int J Neural Syst 2020; 31:2050068. [PMID: 33200620 DOI: 10.1142/s0129065720500689] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Deception detection is a relevant ability in high stakes situations such as police interrogatories or court trials, where the outcome is highly influenced by the interviewed person behavior. With the use of specific devices, e.g. polygraph or magnetic resonance, the subject is aware of being monitored and can change his behavior, thus compromising the interrogation result. For this reason, video analysis-based methods for automatic deception detection are receiving ever increasing interest. In this paper, a deception detection approach based on RGB videos, leveraging both facial features and stacked generalization ensemble, is proposed. First, a face, which is well-known to present several meaningful cues for deception detection, is identified, aligned, and masked to build video signatures. These signatures are constructed starting from five different descriptors, which allow the system to capture both static and dynamic facial characteristics. Then, video signatures are given as input to four base-level algorithms, which are subsequently fused applying the stacked generalization technique, resulting in a more robust meta-level classifier used to predict deception. By exploiting relevant cues via specific features, the proposed system achieves improved performances on a public dataset of famous court trials, with respect to other state-of-the-art methods based on facial features, highlighting the effectiveness of the proposed method.
Collapse
Affiliation(s)
- Danilo Avola
- Department of Computer Science, Sapienza University, Via Salaria 113, 00198 Rome, Italy
| | - Marco Cascio
- Department of Computer Science, Sapienza University, Via Salaria 113, 00198 Rome, Italy
| | - Luigi Cinque
- Department of Computer Science, Sapienza University, Via Salaria 113, 00198 Rome, Italy
| | - Alessio Fagioli
- Department of Computer Science, Sapienza University, Via Salaria 113, 00198 Rome, Italy
| | - Gian Luca Foresti
- Department of Computer Science, Mathematics and Physics, University of Udine, Via delle Scienze, 33100 Udine, Italy
| |
Collapse
|
10
|
Moradi F, Mohammadi H, Rezaei M, Sariaslani P, Razazian N, Khazaie H, Adeli H. A Novel Method for Sleep-Stage Classification Based on Sonification of Sleep Electroencephalogram Signals Using Wavelet Transform and Recurrent Neural Network. Eur Neurol 2020; 83:468-486. [PMID: 33120386 DOI: 10.1159/000511306] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2020] [Accepted: 09/02/2020] [Indexed: 11/19/2022]
Abstract
INTRODUCTION Visual sleep-stage scoring is a time-consuming technique that cannot extract the nonlinear characteristics of electroencephalogram (EEG). This article presents a novel method for sleep-stage differentiation based on sonification of sleep-EEG signals using wavelet transform and recurrent neural network (RNN). METHODS Two RNNs were designed and trained separately based on a database of classical guitar pieces and Kurdish tanbur Makams using a long short-term memory model. Moreover, discrete wavelet transform and wavelet packet decomposition were used to determine the association between the EEG signals and musical pitches. Continuous wavelet transform was applied to extract musical beat-based features from the EEG. Then, the pretrained RNN was used to generate music. To test the proposed model, 11 sleep EEGs were mapped onto the guitar and tanbur frequency intervals and presented to the pretrained RNN. Next, the generated music was randomly presented to 2 neurologists. RESULTS The proposed model classified the sleep stages with an accuracy of >81% for tanbur and more than 93% for guitar musical pieces. The inter-rater reliability measured by Cohen's kappa coefficient (κ) revealed good reliability for both tanbur (κ = 0.64, p < 0.001) and guitar musical pieces (κ = 0.85, p < 0.001). CONCLUSION The present EEG sonification method leads to valid sleep staging by clinicians. The method could be used on various EEG databases for classification, differentiation, diagnosis, and treatment purposes. Real-time EEG sonification can be used as a feedback tool for replanning of neurophysiological functions for the management of many neurological and psychiatric disorders in the future.
Collapse
Affiliation(s)
- Foad Moradi
- Sleep Disorders Research Center, Kermanshah University of Medical Sciences, Kermanshah, Iran.,Electrical and Computer Engineering Faculty, Semnan University, Semnan, Iran.,Department of Neurology, School of Medicine, Kermanshah University of Medical Sciences, Kermanshah, Iran
| | - Hiwa Mohammadi
- Sleep Disorders Research Center, Kermanshah University of Medical Sciences, Kermanshah, Iran, .,Department of Neurology, School of Medicine, Kermanshah University of Medical Sciences, Kermanshah, Iran, .,Clinical Research Development Center, Imam Reza Hospital, Kermanshah University of Medical Sciences, Kermanshah, Iran,
| | - Mohammad Rezaei
- Sleep Disorders Research Center, Kermanshah University of Medical Sciences, Kermanshah, Iran.,Department of Medical Physics and Biomedical Engineering, School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - Payam Sariaslani
- Department of Neurology, School of Medicine, Kermanshah University of Medical Sciences, Kermanshah, Iran.,Clinical Research Development Center, Imam Reza Hospital, Kermanshah University of Medical Sciences, Kermanshah, Iran
| | - Nazanin Razazian
- Department of Neurology, School of Medicine, Kermanshah University of Medical Sciences, Kermanshah, Iran.,Clinical Research Development Center, Imam Reza Hospital, Kermanshah University of Medical Sciences, Kermanshah, Iran
| | - Habibolah Khazaie
- Sleep Disorders Research Center, Kermanshah University of Medical Sciences, Kermanshah, Iran
| | - Hojjat Adeli
- Department of Biomedical Informatics and Department of Neuroscience, The Ohio State University, Columbus, Ohio, USA
| |
Collapse
|
11
|
Zhu Y, Liu J, Ristaniemi T, Cong F. Distinct Patterns of Functional Connectivity During the Comprehension of Natural, Narrative Speech. Int J Neural Syst 2020; 30:2050007. [PMID: 32116090 DOI: 10.1142/s0129065720500070] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Recent continuous task studies, such as narrative speech comprehension, show that fluctuations in brain functional connectivity (FC) are altered and enhanced compared to the resting state. Here, we characterized the fluctuations in FC during comprehension of speech and time-reversed speech conditions. The correlations of Hilbert envelope of source-level EEG data were used to quantify FC between spatially separate brain regions. A symmetric multivariate leakage correction was applied to address the signal leakage issue before calculating FC. The dynamic FC was estimated based on a sliding time window. Then, principal component analysis (PCA) was performed on individually concatenated and temporally concatenated FC matrices to identify FC patterns. We observed that the mode of FC induced by speech comprehension can be characterized with a single principal component. The condition-specific FC demonstrated decreased correlations between frontal and parietal brain regions and increased correlations between frontal and temporal brain regions. The fluctuations of the condition-specific FC characterized by a shorter time demonstrated that dynamic FC also exhibited condition specificity over time. The FC is dynamically reorganized and FC dynamic pattern varies along a single mode of variation during speech comprehension. The proposed analysis framework seems valuable for studying the reorganization of brain networks during continuous task experiments.
Collapse
Affiliation(s)
- Yongjie Zhu
- School of Biomedical Engineering, Faculty of Electronic and Electrical Engineering, Dalian University of Technology, Dalian, Liaoning 116024, P. R. China.,Faculty of Information Technology, P. O. Box 35, University of Jyväskylä, FI-40014 University of Jyväskylä, Finland
| | - Jia Liu
- Faculty of Information Technology, P. O. Box 35, University of Jyväskylä, FI-40014 University of Jyväskylä, Finland
| | - Tapani Ristaniemi
- Faculty of Information Technology, P. O. Box 35, University of Jyväskylä, FI-40014 University of Jyväskylä, Finland
| | - Fengyu Cong
- School of Biomedical Engineering, Faculty of Electronic and Electrical Engineering, Dalian University of Technology, Dalian, Liaoning 116024, P. R. China.,Faculty of Information Technology, P. O. Box 35, University of Jyväskylä, FI-40014 University of Jyväskylä, Finland
| |
Collapse
|
12
|
Mishra P, Piciarelli C, Foresti GL. A Neural Network for Image Anomaly Detection with Deep Pyramidal Representations and Dynamic Routing. Int J Neural Syst 2020; 30:2050060. [DOI: 10.1142/s0129065720500604] [Citation(s) in RCA: 22] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Image anomaly detection is an application-driven problem where the aim is to identify novel samples, which differ significantly from the normal ones. We here propose Pyramidal Image Anomaly DEtector (PIADE), a deep reconstruction-based pyramidal approach, in which image features are extracted at different scale levels to better catch the peculiarities that could help to discriminate between normal and anomalous data. The features are dynamically routed to a reconstruction layer and anomalies can be identified by comparing the input image with its reconstruction. Unlike similar approaches, the comparison is done by using structural similarity and perceptual loss rather than trivial pixel-by-pixel comparison. The proposed method performed at par or better than the state-of-the-art methods when tested on publicly available datasets such as CIFAR10, COIL-100 and MVTec.
Collapse
Affiliation(s)
- Pankaj Mishra
- Dipartimento di Scienze Matematiche, Informatiche e Fisiche, Università Degli Studi di Udine, Via Delle Scienze 206, 33100 Udine, Italy
| | - Claudio Piciarelli
- Dipartimento di Scienze Matematiche, Informatiche e Fisiche, Università Degli Studi di Udine, Via Delle Scienze 206, 33100 Udine, Italy
| | - Gian Luca Foresti
- Dipartimento di Scienze Matematiche, Informatiche e Fisiche, Università Degli Studi di Udine, Via Delle Scienze 206, 33100 Udine, Italy
| |
Collapse
|
13
|
Hashimoto H, Kameda S, Maezawa H, Oshino S, Tani N, Khoo HM, Yanagisawa T, Yoshimine T, Kishima H, Hirata M. A Swallowing Decoder Based on Deep Transfer Learning: AlexNet Classification of the Intracranial Electrocorticogram. Int J Neural Syst 2020; 31:2050056. [PMID: 32938263 DOI: 10.1142/s0129065720500562] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/30/2023]
Abstract
To realize a brain-machine interface to assist swallowing, neural signal decoding is indispensable. Eight participants with temporal-lobe intracranial electrode implants for epilepsy were asked to swallow during electrocorticogram (ECoG) recording. Raw ECoG signals or certain frequency bands of the ECoG power were converted into images whose vertical axis was electrode number and whose horizontal axis was time in milliseconds, which were used as training data. These data were classified with four labels (Rest, Mouth open, Water injection, and Swallowing). Deep transfer learning was carried out using AlexNet, and power in the high-[Formula: see text] band (75-150[Formula: see text]Hz) was the training set. Accuracy reached 74.01%, sensitivity reached 82.51%, and specificity reached 95.38%. However, using the raw ECoG signals, the accuracy obtained was 76.95%, comparable to that of the high-[Formula: see text] power. We demonstrated that a version of AlexNet pre-trained with visually meaningful images can be used for transfer learning of visually meaningless images made up of ECoG signals. Moreover, we could achieve high decoding accuracy using the raw ECoG signals, allowing us to dispense with the conventional extraction of high-[Formula: see text] power. Thus, the images derived from the raw ECoG signals were equivalent to those derived from the high-[Formula: see text] band for transfer deep learning.
Collapse
Affiliation(s)
- Hiroaki Hashimoto
- Department of Neurological Diagnosis and Restoration, Graduate School of Medicine, Osaka University, Yamadaoka 2-2, Suita, Osaka 565-0871, Japan.,Department of Neurosurgery, Otemae Hospital, Chuo-Ku Otemae 1-5-34, Osaka, Osaka 540-0008, Japan.,Endowed Research Department of Clinical Neuroengineering, Global Center for Medical Engineering and Informatics, Osaka University, Yamadaoka 2-2, Suita, Osaka 565-0871, Japan
| | - Seiji Kameda
- Department of Neurological Diagnosis and Restoration, Graduate School of Medicine, Osaka University, Yamadaoka 2-2, Suita, Osaka 565-0871, Japan
| | - Hitoshi Maezawa
- Department of Neurological Diagnosis and Restoration, Graduate School of Medicine, Osaka University, Yamadaoka 2-2, Suita, Osaka 565-0871, Japan
| | - Satoru Oshino
- Department of Neurosurgery, Graduate School of Medicine, Osaka University, Yamadaoka 2-2, Suita, Osaka 565-0871, Japan
| | - Naoki Tani
- Department of Neurosurgery, Graduate School of Medicine, Osaka University, Yamadaoka 2-2, Suita, Osaka 565-0871, Japan
| | - Hui Ming Khoo
- Department of Neurosurgery, Graduate School of Medicine, Osaka University, Yamadaoka 2-2, Suita, Osaka 565-0871, Japan
| | - Takufumi Yanagisawa
- Department of Neurosurgery, Graduate School of Medicine, Osaka University, Yamadaoka 2-2, Suita, Osaka 565-0871, Japan
| | - Toshiki Yoshimine
- Endowed Research Department of Clinical Neuroengineering, Global Center for Medical Engineering and Informatics, Osaka University, Yamadaoka 2-2, Suita, Osaka 565-0871, Japan
| | - Haruhiko Kishima
- Department of Neurosurgery, Graduate School of Medicine, Osaka University, Yamadaoka 2-2, Suita, Osaka 565-0871, Japan
| | - Masayuki Hirata
- Department of Neurological Diagnosis and Restoration, Graduate School of Medicine, Osaka University, Yamadaoka 2-2, Suita, Osaka 565-0871, Japan.,Endowed Research Department of Clinical Neuroengineering, Global Center for Medical Engineering and Informatics, Osaka University, Yamadaoka 2-2, Suita, Osaka 565-0871, Japan.,Department of Neurosurgery, Graduate School of Medicine, Osaka University, Yamadaoka 2-2, Suita, Osaka 565-0871, Japan
| |
Collapse
|
14
|
Nogay HS, Adeli H. Machine learning (ML) for the diagnosis of autism spectrum disorder (ASD) using brain imaging. Rev Neurosci 2020; 31:/j/revneuro.ahead-of-print/revneuro-2020-0043/revneuro-2020-0043.xml. [PMID: 32866134 DOI: 10.1515/revneuro-2020-0043] [Citation(s) in RCA: 70] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2020] [Accepted: 07/25/2020] [Indexed: 02/24/2024]
Abstract
Autism spectrum disorder (ASD) is a neurodevelopmental incurable disorder with a long diagnostic period encountered in the early years of life. If diagnosed early, the negative effects of this disease can be reduced by starting special education early. Machine learning (ML), an increasingly ubiquitous technology, can be applied for the early diagnosis of ASD. The aim of this study is to examine and provide a comprehensive state-of-the-art review of ML research for the diagnosis of ASD based on (a) structural magnetic resonance image (MRI), (b) functional MRI and (c) hybrid imaging techniques over the past decade. The accuracy of the studies with a large number of participants is in general lower than those with fewer participants leading to the conclusion that further large-scale studies are needed. An examination of the age of the participants shows that the accuracy of the automated diagnosis of ASD is higher at a younger age range. ML technology is expected to contribute significantly to the early and rapid diagnosis of ASD in the coming years and become available to clinicians in the near future. This review is aimed to facilitate that.
Collapse
Affiliation(s)
- Hidir Selcuk Nogay
- Department of Electrical and Energy, Kayseri University, Kayseri, Turkey
- The Ohio State University, Mathematical Bioscience Institute, Columbus, OH, USA
| | - Hojjat Adeli
- Departments of Biomedical Informatics and Neuroscience, The Ohio State University, Columbus, US
| |
Collapse
|
15
|
Upper Limb Movement Classification Via Electromyographic Signals and an Enhanced Probabilistic Network. J Med Syst 2020; 44:176. [DOI: 10.1007/s10916-020-01639-x] [Citation(s) in RCA: 22] [Impact Index Per Article: 4.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/17/2019] [Accepted: 08/05/2020] [Indexed: 11/26/2022]
|
16
|
Castillo-Barnes D, Martinez-Murcia FJ, Ortiz A, Salas-Gonzalez D, RamÍrez J, Górriz JM. Morphological Characterization of Functional Brain Imaging by Isosurface Analysis in Parkinson’s Disease. Int J Neural Syst 2020; 30:2050044. [DOI: 10.1142/s0129065720500446] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Finding new biomarkers to model Parkinson’s Disease (PD) is a challenge not only to help discerning between Healthy Control (HC) subjects and patients with potential PD but also as a way to measure quantitatively the loss of dopaminergic neurons mainly concentrated at substantia nigra. Within this context, this work presented here tries to provide a set of imaging features based on morphological characteristics extracted from I[Formula: see text]-Ioflupane SPECT scans to discern between HC and PD participants in a balanced set of [Formula: see text] scans from Parkinson’s Progression Markers Initiative (PPMI) database. These features, obtained from isosurfaces of each scan at different intensity levels, have been classified through the use of classical Machine Learning classifiers such as Support-Vector-Machines (SVM) or Naïve Bayesian and compared with the results obtained using a Multi-Layer Perceptron (MLP). The proposed system, based on a Mann–Whitney–Wilcoxon U-Test for feature selection and the SVM approach, yielded a [Formula: see text] balanced accuracy when the performance was evaluated using a [Formula: see text]-fold cross-validation. This proves the reliability of these biomarkers, especially those related to sphericity, center of mass, number of vertices, 2D-projected perimeter or the 2D-projected eccentricity, among others, but including both internal and external isosurfaces.
Collapse
Affiliation(s)
- Diego Castillo-Barnes
- Department of Signal Theory, Telematics and Communications, University of Granada, Periodista Daniel Saucedo Aranda, Granada 18071, Spain
| | | | - Andres Ortiz
- Department of Communications Engineering, University of Malaga, Bulevar Louis Pasteur 35, Malaga 29071, Spain
| | - Diego Salas-Gonzalez
- Department of Signal Theory, Telematics and Communications, University of Granada, Periodista Daniel Saucedo Aranda, Granada 18071, Spain
| | - Javier RamÍrez
- Department of Signal Theory, Telematics and Communications, University of Granada, Periodista Daniel Saucedo Aranda, Granada 18071, Spain
| | - Juan M. Górriz
- Department of Signal Theory, Telematics and Communications, University of Granada, Periodista Daniel Saucedo Aranda, Granada 18071, Spain
| |
Collapse
|
17
|
Wang L, Li X, Zhu Y, Lin B, Bo Q, Li F, Wang C. Discriminative Analysis of Symptom Severity and Ultra-High Risk of Schizophrenia Using Intrinsic Functional Connectivity. Int J Neural Syst 2020; 30:2050047. [PMID: 32689843 DOI: 10.1142/s0129065720500471] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
Abstract
Past studies have consistently shown functional dysconnectivity of large-scale brain networks in schizophrenia. In this study, we aimed to further assess whether multivariate pattern analysis (MVPA) could yield a sensitive predictor of patient symptoms, as well as identify ultra-high risk (UHR) stage of schizophrenia from intrinsic functional connectivity of whole-brain networks. We first combined rank-based feature selection and support vector machine methods to distinguish between 43 schizophrenia patients and 52 healthy controls. The constructed classifier was then applied to examine functional connectivity profiles of 18 UHR individuals. The classifier indicated reliable relationship between MVPA measures and symptom severity, with higher classification accuracy in more severely affected schizophrenia patients. The UHR subjects had classification scores falling between those of healthy controls and patients, suggesting an intermediate level of functional brain abnormalities. Moreover, UHR individuals with schizophrenia-like connectivity profiles at baseline presented higher rate of conversion to full-blown illness in the follow-up visits. Spatial maps of discriminative brain regions implicated increases of functional connectivity in the default mode network, whereas decreases of functional connectivity in the cerebellum, thalamus and visual areas in schizophrenia. The findings may have potential utility in the early diagnosis and intervention of schizophrenia.
Collapse
Affiliation(s)
- Lubin Wang
- Institute of Military Cognition and Brain Sciences, Academy of Military Medical Sciences, Beijing 100850, P. R. China
| | - Xianbin Li
- The National Clinical Research Center for Mental Disorders & Beijing Key Laboratory of Mental Disorders, Beijing Anding Hospital, Capital Medical University, Beijing 100088, P. R. China
| | - Yuyang Zhu
- Institute of Military Cognition and Brain Sciences, Academy of Military Medical Sciences, Beijing 100850, P. R. China
| | - Bei Lin
- Institute of Military Cognition and Brain Sciences, Academy of Military Medical Sciences, Beijing 100850, P. R. China
| | - Qijing Bo
- The National Clinical Research Center for Mental Disorders & Beijing Key Laboratory of Mental Disorders, Beijing Anding Hospital, Capital Medical University, Beijing 100088, P. R. China
| | - Feng Li
- The National Clinical Research Center for Mental Disorders & Beijing Key Laboratory of Mental Disorders, Beijing Anding Hospital, Capital Medical University, Beijing 100088, P. R. China
| | - Chuanyue Wang
- The National Clinical Research Center for Mental Disorders & Beijing Key Laboratory of Mental Disorders, Beijing Anding Hospital, Capital Medical University, Beijing 100088, P. R. China
| |
Collapse
|
18
|
Apicella A, Isgrò F, Prevete R, Tamburrini G. Middle-Level Features for the Explanation of Classification Systems by Sparse Dictionary Methods. Int J Neural Syst 2020; 30:2050040. [PMID: 32727317 DOI: 10.1142/s0129065720500409] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/22/2023]
Abstract
Machine learning (ML) systems are affected by a pervasive lack of transparency. The eXplainable Artificial Intelligence (XAI) research area addresses this problem and the related issue of explaining the behavior of ML systems in terms that are understandable to human beings. In many explanation of XAI approaches, the output of ML systems are explained in terms of low-level features of their inputs. However, these approaches leave a substantive explanatory burden with human users, insofar as the latter are required to map low-level properties into more salient and readily understandable parts of the input. To alleviate this cognitive burden, an alternative model-agnostic framework is proposed here. This framework is instantiated to address explanation problems in the context of ML image classification systems, without relying on pixel relevance maps and other low-level features of the input. More specifically, one obtains sets of middle-level properties of classification inputs that are perceptually salient by applying sparse dictionary learning techniques. These middle-level properties are used as building blocks for explanations of image classifications. The achieved explanations are parsimonious, for their reliance on a limited set of middle-level image properties. And they can be contrastive, because the set of middle-level image properties can be used to explain why the system advanced the proposed classification over other antagonist classifications. In view of its model-agnostic character, the proposed framework is adaptable to a variety of other ML systems and explanation problems.
Collapse
Affiliation(s)
- A Apicella
- Dipartimento di Ingegneria Elettrica e delle Tecnologie dell'Informazione, Università degli Studi di Napoli Federico II, 80125 Napoli, Italy
| | - F Isgrò
- Dipartimento di Ingegneria Elettrica e delle Tecnologie dell'Informazione, Università degli Studi di Napoli Federico II, 80125 Napoli, Italy
| | - R Prevete
- Dipartimento di Ingegneria Elettrica e delle Tecnologie dell'Informazione, Università degli Studi di Napoli Federico II, 80125 Napoli, Italy
| | - G Tamburrini
- Dipartimento di Ingegneria Elettrica e delle Tecnologie dell'Informazione, Università degli Studi di Napoli Federico II, 80125 Napoli, Italy
| |
Collapse
|
19
|
Leming M, Górriz JM, Suckling J. Ensemble Deep Learning on Large, Mixed-Site fMRI Datasets in Autism and Other Tasks. Int J Neural Syst 2020; 30:2050012. [DOI: 10.1142/s0129065720500124] [Citation(s) in RCA: 33] [Impact Index Per Article: 6.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
Abstract
Deep learning models for MRI classification face two recurring problems: they are typically limited by low sample size, and are abstracted by their own complexity (the “black box problem”). In this paper, we train a convolutional neural network (CNN) with the largest multi-source, functional MRI (fMRI) connectomic dataset ever compiled, consisting of 43,858 datapoints. We apply this model to a cross-sectional comparison of autism spectrum disorder (ASD) versus typically developing (TD) controls that has proved difficult to characterize with inferential statistics. To contextualize these findings, we additionally perform classifications of gender and task versus rest. Employing class-balancing to build a training set, we trained [Formula: see text] modified CNNs in an ensemble model to classify fMRI connectivity matrices with overall AUROCs of 0.6774, 0.7680, and 0.9222 for ASD versus TD, gender, and task versus rest, respectively. Additionally, we aim to address the black box problem in this context using two visualization methods. First, class activation maps show which functional connections of the brain our models focus on when performing classification. Second, by analyzing maximal activations of the hidden layers, we were also able to explore how the model organizes a large and mixed-center dataset, finding that it dedicates specific areas of its hidden layers to processing different covariates of data (depending on the independent variable analyzed), and other areas to mix data from different sources. Our study finds that deep learning models that distinguish ASD from TD controls focus broadly on temporal and cerebellar connections, with a particularly high focus on the right caudate nucleus and paracentral sulcus.
Collapse
Affiliation(s)
- Matthew Leming
- Department of Psychiatry, University of Cambridge, Herchel Smith Building for Brain & Mind Sciences, Forvie Site, Robinson Way, Cambridge, CB20SZ, UK
| | - Juan Manuel Górriz
- Department of Signal Theory, Networking and Communications, University of Granada, Avenida del Hospicio, E-18071 Granada, Spain
| | - John Suckling
- Department of Psychiatry, University of Cambridge, Herchel Smith Building for Brain & Mind Sciences, Forvie Site, Robinson Way, Cambridge, CB20SZ, UK
| |
Collapse
|
20
|
Liu X, Sun CX, Gao J, Xu SY. Controllability of Networks of Multiple Coupled Neural Populations: An Analytical Method for Neuromodulation's Feasibility. Int J Neural Syst 2020; 30:2050001. [PMID: 31969078 DOI: 10.1142/s012906572050001x] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Neuromodulation plays a vital role in the prevention and treatment of neurological and psychiatric disorders. Neuromodulation's feasibility is a long-standing issue because it provides the necessity for neuromodulation to realize the desired purpose. A controllability analysis of neural dynamics is necessary to ensure neuromodulation's feasibility. Here, we present such a theoretical method by using the concept of controllability from the control theory that neuromodulation's feasibility can be studied smoothly. Firstly, networks of multiple coupled neural populations with different topologies are established to mathematically model complicated neural dynamics. Secondly, an analytical method composed of a linearization method, the Kalman controllable rank condition and a controllability index is applied to analyze the controllability of the established network models. Finally, the relationship between network dynamics or topological characteristic parameters and controllability is studied by using the analytical method. The proposed method provides a new idea for the study of neuromodulation's feasibility, and the results are expected to guide us to better modulate neurodynamics by optimizing network dynamics and network topology.
Collapse
Affiliation(s)
- Xian Liu
- Institute of Electrical Engineering, Yanshan University, Qinhuangdao 066004, P. R. China
| | - Cheng-Xia Sun
- Institute of Electrical Engineering, Yanshan University, Qinhuangdao 066004, P. R. China
| | - Jing Gao
- Institute of Electrical Engineering, Yanshan University, Qinhuangdao 066004, P. R. China
| | - Shi-Yun Xu
- China Electric Power Research Institute, Beijing 100192, P. R. China.,NAAM Group, Faculty of Science, King Abdulaziz University, Jeddah 999088, Saudi Arabia
| |
Collapse
|