Unveiling complex brain dynamics during movie viewing via deep recurrent autoencoder model

Naturalistic stimuli have become an effective tool to uncover the dynamic functional brain networks triggered by cognitive and emotional real-life experiences through multimodal and dynamic stimuli. However, current research predominantly focused on exploring dynamic functional connectivity generated via chosen templates under resting-state paradigm, with relatively limited investigation into the dynamic functional interactions among large-scale brain networks. Moreover, these studies might overlook the longer time-scale adaptability and information transmission that occur over extended periods during naturalistic stimuli. In this study, we introduced an unsupervised deep recurrent autoencoder (DRAE) model combined with a sliding window approach, effectively capturing the brain's long-term temporal dependencies, as measured in functional magnetic resonance imaging (fMRI), when subjects viewing a long-duration and emotional film. The experimental results revealed that naturalistic stimuli can induce dynamic large-scale brain networks, of which functional interactions covary with the development of the film's narrative. Furthermore, the dynamic interactions among brain networks were temporally synchronized with specific features of the movie, especially with the emotional arousal and valence. Our study provided novel insight to the underlying neural mechanisms of dynamic functional interactions among brain regions in an ecologically valid sensory experience.

Long-Short-Term-Memory-Based Deep Stacked Sequence-to-Sequence Autoencoder for Health Prediction of Industrial Workers in Closed Environments Based on Wearable Devices

To reduce the risks and challenges faced by frontline workers in confined workspaces, accurate real-time health monitoring of their vital signs is essential for improving safety and productivity and preventing accidents. Machine-learning-based data-driven methods have shown promise in extracting valuable information from complex monitoring data. However, practical industrial settings still struggle with the data collection difficulties and low prediction accuracy of machine learning models due to the complex work environment. To tackle these challenges, a novel approach called a long short-term memory (LSTM)-based deep stacked sequence-to-sequence autoencoder is proposed for predicting the health status of workers in confined spaces. The first step involves implementing a wireless data acquisition system using edge-cloud platforms. Smart wearable devices are used to collect data from multiple sources, like temperature, heart rate, and pressure. These comprehensive data provide insights into the workers' health status within the closed space of a manufacturing factory. Next, a hybrid model combining deep learning and support vector machine (SVM) is constructed for anomaly detection. The LSTM-based deep stacked sequence-to-sequence autoencoder is specifically designed to learn deep discriminative features from the time-series data by reconstructing the input data and thus generating fused deep features. These features are then fed into a one-class SVM, enabling accurate recognition of workers' health status. The effectiveness and superiority of the proposed approach are demonstrated through comparisons with other existing approaches.

Autoencoder and restricted Boltzmann machine for transfer learning in functional magnetic resonance imaging task classification

Deep neural networks (DNNs) have been adopted widely as classifiers for functional magnetic resonance imaging (fMRI) data, advancing beyond traditional machine learning models. Consequently, transfer learning of the pre-trained DNN becomes crucial to enhance DNN classification performance, specifically by alleviating an overfitting issue that occurs when a substantial number of DNN parameters are fitted to a relatively small number of fMRI samples. In this study, we first systematically compared the two most popularly used, unsupervised pretraining models for resting-state fMRI (rfMRI) volume data to pre-train the DNNs, namely autoencoder (AE) and restricted Boltzmann machine (RBM). The group in-brain mask used when training AE and RBM displayed a sizable overlap ratio with Yeo's seven functional brain networks (FNs). The parcellated FNs obtained from the RBM were fine-grained compared to those from the AE. The pre-trained AE and RBM served as the weight parameters of the first of the two hidden DNN layers, and the DNN fulfilled the task classifier role for fMRI (tfMRI) data in the Human Connectome Project (HCP). We tested two transfer learning schemes: (1) fixing and (2) fine-tuning the DNN's pre-trained AE or RBM weights. The DNN with transfer learning was compared to a baseline DNN, trained using random initial weights. Overall, DNN classification performance from the transfer learning proved superior when the pre-trained RBM weights were fixed and when the pre-trained AE weights were fine-tuned (average error rates: 14.8% for fixed RBM, 15.1% fine-tuned AE, and 15.5% for the baseline model) compared to the alternative scenarios of DNN transfer learning schemes. Moreover, the optimal transfer learning scheme between the fixed RBM and fine-tuned AE varied according to seven task conditions in the HCP. Nonetheless, the computational load reduced substantially for the fixed-weight-based transfer learning compared to the fine-tuning-based transfer learning (e.g., the number of weight parameters for the fixed-weight-based DNN model reduced to 1.9% compared with a baseline/fine-tuned DNN model). Our findings suggest that weight initialization at the DNN's first layer using RBM-based pre-trained weights provides the most promising approach when the whole-brain fMRI volume supports associated task classification. We believe that our proposed scheme could be applied to a variety of task conditions to improve their classification performance and to utilize computational resources efficiently using our AE/RBM-based pre-trained weights compared to random initial weights for DNN training.

An explainable deep learning framework for characterizing and interpreting human brain states

Deep learning approaches have been widely adopted in the medical image analysis field. However, a most of existing deep learning approaches focus on achieving promising performances such as classification, detection, and segmentation, and much less effort is devoted to the explanation of the designed models. Similarly, in the brain imaging field, many deep learning approaches have been designed and applied to characterize and predict human brain states. However, these models lack interpretation. In response, we propose a novel domain knowledge informed self-attention graph pooling-based (SAGPool) graph convolutional neural network to study human brain states. Specifically, the dense individualized and common connectivity-based cortical landmarks system (DICCCOL, structural brain connectivity profiles) and holistic atlases of functional networks and interactions system (HAFNI, functional brain connectivity profiles) are integrated with the SAGPool model to better characterize and interpret the brain states. Extensive experiments are designed and carried out on the large-scale human connectome project (HCP) Q1 and S1200 dataset. Promising brain state classification performances are observed (e.g., an average of 93.7% for seven-task classification and 100% for binary classification). In addition, the importance of the brain regions, which contributes most to the accurate classification, is successfully quantified and visualized. A thorough neuroscientific interpretation suggests that these extracted brain regions and their importance calculated from self-attention graph pooling layer offer substantial explainability.

Evolutional Neural Architecture Search for Optimization of Spatiotemporal Brain Network Decomposition

Using deep neural networks (DNNs) to explore spatial patterns and temporal dynamics of human brain activities has been an important yet challenging problem because the artificial neural networks are hard to be designed manually. There have been several promising deep learning methods, e.g., deep belief network (DBN), convolutional neural network (CNN), and deep sparse recurrent auto-encoder (DSRAE), that can decompose neuroscientific and meaningful spatiotemporal patterns from 4D functional Magnetic Resonance Imaging (fMRI) data. However, those previous studies still depend on hand-crafted neural network architectures and hyperparameters, which are not optimal in various senses. In this paper, we employ the evolutionary algorithms (EA) to optimize the deep neural architecture of DSRAE by minimizing the expected loss of initialized models, named eNAS-DSRAE (evolutionary Neural Architecture Search on Deep Sparse Recurrent Auto-Encoder). Also, validation experiments are designed and performed on the publicly available human connectome project (HCP) 900 datasets, and the results achieved by the optimized eNAS-DSRAE suggested that our framework can successfully identify the spatiotemporal features and perform better than the hand-crafted neural network models. To our best knowledge, the proposed eNAS-DSRAE is not only among the earliest NAS models that can extract connectome-scale meaningful spatiotemporal brain networks from 4D fMRI data, but also is an effective framework to optimize the RNN-based models.

Simultaneous spatial-temporal decomposition for connectome-scale brain networks by deep sparse recurrent auto-encoder

Exploring the spatial patterns and temporal dynamics of human brain activity has been of great interest, in the quest to better understand connectome-scale brain networks. Though modeling spatial and temporal patterns of functional brain networks have been researched for a long time, the development of a unified and simultaneous spatial-temporal model has yet to be realized. For instance, although some deep learning methods have been proposed recently in order to model functional brain networks, most of them can only represent either spatial or temporal perspective of functional Magnetic Resonance Imaging (fMRI) data and rarely model both domains simultaneously. Due to the recent success in applying sequential auto-encoders for brain decoding, in this paper, we propose a deep sparse recurrent auto-encoder (DSRAE) to be applied unsupervised to learn spatial patterns and temporal fluctuations of brain networks at the same time. The proposed DSRAE was evaluated and validated based on three tasks of the publicly available Human Connectome Project (HCP) fMRI dataset, resulting with promising evidence. To the best of our knowledge, the proposed DSRAE is among the early efforts in developing unified models that can extract connectome-scale spatial-temporal networks from 4D fMRI data simultaneously.

Functional Neuroimaging in the New Era of Big Data

The field of functional neuroimaging has substantially advanced as a big data science in the past decade, thanks to international collaborative projects and community efforts. Here we conducted a literature review on functional neuroimaging, with focus on three general challenges in big data tasks: data collection and sharing, data infrastructure construction, and data analysis methods. The review covers a wide range of literature types including perspectives, database descriptions, methodology developments, and technical details. We show how each of the challenges was proposed and addressed, and how these solutions formed the three core foundations for the functional neuroimaging as a big data science and helped to build the current data-rich and data-driven community. Furthermore, based on our review of recent literature on the upcoming challenges and opportunities toward future scientific discoveries, we envisioned that the functional neuroimaging community needs to advance from the current foundations to better data integration infrastructure, methodology development toward improved learning capability, and multi-discipline translational research framework for this new era of big data.