Neural Decoding of Multi-Modal Imagery Behavior Focusing on Temporal Complexity

Mental imagery behaviors of various modalities include visual, auditory, and motor behaviors. Their alterations are pathologically involved in various psychiatric disorders. Results of earlier studies suggest that imagery behaviors are correlated with the modulated activities of the respective modality-specific regions and the additional activities of supramodal imagery-related regions. Additionally, despite the availability of complexity analysis in the neuroimaging field, it has not been used for neural decoding approaches. Therefore, we sought to characterize neural oscillation related to multimodal imagery through complexity-based neural decoding. For this study, we modified existing complexity measures to characterize the time evolution of temporal complexity. We took magnetoencephalography (MEG) data of eight healthy subjects as they performed multimodal imagery and non-imagery tasks. The MEG data were decomposed into amplitude and phase of sub-band frequencies by Hilbert-Huang transform. Subsequently, we calculated the complexity values of each reconstructed time series, along with raw data and band power for comparison, and applied these results as inputs to decode visual perception (VP), visual imagery (VI), motor execution (ME), and motor imagery (MI) functions. Consequently, intra-subject decoding with the complexity yielded a characteristic sensitivity map for each task with high decoding accuracy. The map is inverted in the occipital regions between VP and VI and in the central regions between ME and MI. Additionally, replacement of the labels into two classes as imagery and non-imagery also yielded better classification performance and characteristic sensitivity with the complexity. It is particularly interesting that some subjects showed characteristic sensitivities not only in modality-specific regions, but also in supramodal regions. These analyses indicate that two-class and four-class classifications each provided better performance when using complexity than when using raw data or band power as input. When inter-subject decoding was used with the same model, characteristic sensitivity maps were also obtained, although their decoding performance was lower. Results of this study underscore the availability of complexity measures in neural decoding approaches and suggest the possibility of a modality-independent imagery-related mechanism. The use of time evolution of temporal complexity in neural decoding might extend our knowledge of the neural bases of hierarchical functions in the human brain.

Decomposed Temporal Complexity Analysis of Neural Oscillations and Machine Learning Applied to Alzheimer's Disease Diagnosis

Despite growing evidence of aberrant neuronal complexity in Alzheimer's disease (AD), it remains unclear how this variation arises. Neural oscillations reportedly comprise different functions depending on their own properties. Therefore, in this study, we investigated details of the complexity of neural oscillations by decomposing the oscillations into frequency, amplitude, and phase for AD patients. We applied resting-state magnetoencephalography (MEG) to 17 AD patients and 21 healthy control subjects. We first decomposed the source time series of the MEG signal into five intrinsic mode functions using ensemble empirical mode decomposition. We then analyzed the temporal complexities of these time series using multiscale entropy. Results demonstrated that AD patients had lower complexity on short time scales and higher complexity on long time scales in the alpha band in temporal regions of the brain. We evaluated the alpha band complexity further by decomposing it into amplitude and phase using Hilbert spectral analysis. Consequently, we found lower amplitude complexity and higher phase complexity in AD patients. Correlation analyses between spectral complexity and decomposed complexities revealed scale-dependency. Specifically, amplitude complexity was positively correlated with spectral complexity on short time scales, whereas phase complexity was positively correlated with spectral complexity on long time scales. Regarding the relevance of cognitive function to the complexity measures, the phase complexity on the long time scale was found to be correlated significantly with the Mini-Mental State Examination score. Additionally, we examined the diagnostic utility of the complexity characteristics using machine learning (ML) methods. We prepared a feature pool using multiple sparse autoencoders (SAEs), chose some discriminating features, and applied them to a support vector machine (SVM). Compared to the simple SVM and the SVM after feature selection (FS + SVM), the SVM with multiple SAEs (SAE + FS + SVM) had improved diagnostic accuracy. Through this study, we 1) advanced the understanding of neuronal complexity in AD patients using decomposed temporal complexity analysis and 2) demonstrated the effectiveness of combining ML methods with information about signal complexity for the diagnosis of AD.