Group Component Analysis for Multiblock Data: Common and Individual Feature Extraction

Diagnosis of Major Depressive Disorder Based on Individualized Brain Functional and Structural Connectivity

BACKGROUND

Traditional neuroimaging studies have primarily emphasized analysis at the group level, often neglecting the specificity at the individual level. Recently, there has been a growing interest in individual differences in brain connectivity. Investigating individual-specific connectivity is important for understanding the mechanisms of major depressive disorder (MDD) and the variations among individuals.

PURPOSE

To integrate individualized functional connectivity and structural connectivity with machine learning techniques to distinguish people with MDD and healthy controls (HCs).

STUDY TYPE

Prospective.

SUBJECTS

A total of 182 patients with MDD and 157 HCs and a verification cohort including 54 patients and 46 HCs.

FIELD STRENGTH/SEQUENCE

3.0 T/T1-weighted imaging, resting-state functional MRI with echo-planar sequence, and diffusion tensor imaging with single-shot spin echo.

ASSESSMENT

Functional and structural brain networks from rs-fMRI and DTI data were constructed, respectively. Based on these networks, individualized functional connectivity (IFC) and individualized structural connectivity (ISC) were extracted using common orthogonal basis extraction (COBE). Subsequently, multimodal canonical correlation analysis combined with joint independent component analysis (mCCA + jICA) was conducted to fusion analysis to identify the joint and unique independent components (ICs) across multiple modes. These ICs were utilized to generate features, and a support vector machine (SVM) model was implemented for the classification of MDD.

STATISTICAL TESTS

The differences in individualized connectivity between patients and controls were compared using two-sample t test, with a significance threshold set at P < 0.05. The established model was tested and evaluated using the receiver operating characteristic (ROC) curve.

RESULTS

The classification performance of the constructed individualized connectivity feature model after multisequence fusion increased from 72.2% to 90.3%. Furthermore, the prediction model showed significant predictive power for assessing the severity of depression in patients with MDD (r = 0.544).

DATA CONCLUSION

The integration of IFC and ISC through multisequence fusion enhances our capacity to identify MDD, highlighting the advantages of the individualized approach and underscoring its significance in MDD research.

LEVEL OF EVIDENCE

1 TECHNICAL EFFICACY: Stage 2.

Deep graph learning of multimodal brain networks defines treatment-predictive signatures in major depression

Major depressive disorder (MDD) presents a substantial health burden with low treatment response rates. Predicting antidepressant efficacy is challenging due to MDD's complex and varied neuropathology. Identifying biomarkers for antidepressant treatment requires thorough analysis of clinical trial data. Multimodal neuroimaging, combined with advanced data-driven methods, can enhance our understanding of the neurobiological processes influencing treatment outcomes. To address this, we analyzed resting-state fMRI and EEG connectivity data from 130 patients treated with sertraline and 135 patients with placebo from the Establishing Moderators and Biosignatures of Antidepressant Response in Clinical Care (EMBARC) study. A deep learning framework was developed using graph neural networks to integrate data-augmented connectivity and cross-modality correlation, aiming to predict individual symptom changes by revealing multimodal brain network signatures. The results showed that our model demonstrated promising prediction accuracy, with an R2 value of 0.24 for sertraline and 0.20 for placebo. It also exhibited potential in transferring predictions using only EEG. Key brain regions identified for predicting sertraline response included the inferior temporal gyrus (fMRI) and posterior cingulate cortex (EEG), while for placebo response, the precuneus (fMRI) and supplementary motor area (EEG) were critical. Additionally, both modalities identified the superior temporal gyrus and posterior cingulate cortex as significant for sertraline response, while the anterior cingulate cortex and postcentral gyrus were common predictors in the placebo arm. Additionally, variations in the frontoparietal control, ventral attention, dorsal attention, and limbic networks were notably associated with MDD treatment. By integrating fMRI and EEG, our study established novel multimodal brain network signatures to predict individual responses to sertraline and placebo in MDD, providing interpretable neural circuit patterns that may guide future targeted interventions. Trial Registration: Establishing Moderators and Biosignatures of Antidepressant Response for Clinical Care for Depression (EMBARC) ClinicalTrials.gov Identifier: NCT#01407094.

Individualized resting-state functional connectivity abnormalities unveil two major depressive disorder subtypes with contrasting abnormal patterns of abnormality

Modern neuroimaging research has recognized that major depressive disorder (MDD) is a connectome disorder, characterized by altered functional connectivity across large-scale brain networks. However, the clinical heterogeneity, likely stemming from diverse neurobiological disturbances, complicates findings from standard group comparison methods. This variability has driven the search for MDD subtypes using objective neuroimaging markers. In this study, we sought to identify potential MDD subtypes from subject-level abnormalities in functional connectivity, leveraging a large multi-site dataset of resting-state MRI from 1276 MDD patients and 1104 matched healthy controls. Subject-level extreme functional connections, determined by comparing against normative ranges derived from healthy controls using tolerance intervals, were used to identify biological subtypes of MDD. We identified a set of extreme functional connections that were predominantly between the visual network and the frontoparietal network, the default mode network and the ventral attention network, with the key regions in the anterior cingulate cortex, bilateral orbitofrontal cortex, and supramarginal gyrus. In MDD patients, these extreme functional connections were linked to age of onset and reward-related processes. Using these features, we identified two subtypes with distinct patterns of functional connectivity abnormalities compared to healthy controls (p < 0.05, Bonferroni correction). When considering all patients together, no significant differences were found. These subtypes significantly enhanced case-control discriminability and showed strong internal discriminability between subtypes. Furthermore, the subtypes were reproducible across varying parameters, study sites, and in untreated patients. Our findings provide new insights into the taxonomy and have potential implications for both diagnosis and treatment of MDD.

Tensor Robust Kernel PCA for Multidimensional Data

Recently, the tensor nuclear norm (TNN)-based tensor robust principle component analysis (TRPCA) has achieved impressive performance in multidimensional data processing. The underlying assumption in TNN is the low-rankness of frontal slices of the tensor in the transformed domain (e.g., Fourier domain). However, the low-rankness assumption is usually violative for real-world multidimensional data (e.g., video and image) due to their intrinsically nonlinear structure. How to effectively and efficiently exploit the intrinsic structure of multidimensional data remains a challenge. In this article, we first suggest a kernelized TNN (KTNN) by leveraging the nonlinear kernel mapping in the transform domain, which faithfully captures the intrinsic structure (i.e., implicit low-rankness) of multidimensional data and is computed at a lower cost by introducing kernel trick. Armed with KTNN, we propose a tensor robust kernel PCA (TRKPCA) model for handling multidimensional data, which decomposes the observed tensor into an implicit low-rank component and a sparse component. To tackle the nonlinear and nonconvex model, we develop an efficient alternating direction method of multipliers (ADMM)-based algorithm. Extensive experiments on real-world applications collectively verify that TRKPCA achieves superiority over the state-of-the-art RPCA methods.

Group Integrative Dynamic Factor Models With Application to Multiple Subject Brain Connectivity

This work introduces a novel framework for dynamic factor model-based group-level analysis of multiple subjects time-series data, called GRoup Integrative DYnamic factor (GRIDY) models. The framework identifies and characterizes intersubject similarities and differences between two predetermined groups by considering a combination of group spatial information and individual temporal dynamics. Furthermore, it enables the identification of intrasubject similarities and differences over time by employing different model configurations for each subject. Methodologically, the framework combines a novel principal angle-based rank selection algorithm and a noniterative integrative analysis framework. Inspired by simultaneous component analysis, this approach also reconstructs identifiable latent factor series with flexible covariance structures. The performance of the GRIDY models is evaluated through simulations conducted under various scenarios. An application is also presented to compare resting-state functional MRI data collected from multiple subjects in autism spectrum disorder and control groups.

A Multiview Brain Network Transformer Fusing Individualized Information for Autism Spectrum Disorder Diagnosis

Functional connectivity (FC) networks, built from analyses of resting-state magnetic resonance imaging (rs-fMRI), serve as efficacious biomarkers for identifying Autism Spectrum Disorders (ASD) patients. Given the neurobiological heterogeneity across individuals and the unique presentation of ASD symptoms, the fusion of individualized information into diagnosis becomes essential. However, this aspect is overlooked in most methods. Furthermore, the existing methods typically focus on studying direct pairwise connections between brain ROIs, while disregarding interactions between indirectly connected neighbors. To overcome above challenges, we build common FC and individualized FC by tangent pearson embedding (TP) and common orthogonal basis extraction (COBE) respectively, and present a novel multiview brain transformer (MBT) aimed at effectively fusing common and indivinformation of subjects. MBT is mainly constructed by transformer layers with diffusion kernel (DK), fusion quality-inspired weighting module (FQW), similarity loss and orthonormal clustering fusion readout module (OCFRead). DK transformer can incorporate higher-order random walk methods to capture wider interactions among indirectly connected brain regions. FQW promotes adaptive fusion of features between views, and similarity loss and OCFRead are placed on the last layer to accomplish the ultimate integration of information. In our method, TP, DK and FQW modules all help to model wider connectivity in the brain that make up for the shortcomings of traditional methods. We conducted experiments on the public ABIDE dataset based on AAL and CC200 respectively. Our framework has shown promising results, outperforming state-of-the-art methods on both templates. This suggests its potential as a valuable approach for clinical ASD diagnosis.

Disentangling the Individual-Shared and Individual-Specific Subspace of Altered Brain Functional Connectivity in Autism Spectrum Disorder

BACKGROUND

Despite considerable effort toward understanding the neural basis of autism spectrum disorder (ASD) using case-control analyses of resting-state functional magnetic resonance imaging data, findings are often not reproducible, largely due to biological and clinical heterogeneity among individuals with ASD. Thus, exploring the individual-shared and individual-specific altered functional connectivity (AFC) in ASD is important to understand this complex, heterogeneous disorder.

METHODS

We considered 254 individuals with ASD and 295 typically developing individuals from the Autism Brain Imaging Data Exchange to explore the individual-shared and individual-specific subspaces of AFC. First, we computed AFC matrices of individuals with ASD compared with typically developing individuals. Then, common orthogonal basis extraction was used to project AFC of ASD onto 2 subspaces: an individual-shared subspace, which represents altered connectivity patterns shared across ASD, and an individual-specific subspace, which represents the remaining individual characteristics after eliminating the individual-shared altered connectivity patterns.

RESULTS

Analysis yielded 3 common components spanning the individual-shared subspace. Common components were associated with differences of functional connectivity at the group level. AFC in the individual-specific subspace improved the prediction of clinical symptoms. The default mode network-related and cingulo-opercular network-related magnitudes of AFC in the individual-specific subspace were significantly correlated with symptom severity in social communication deficits and restricted, repetitive behaviors in ASD.

CONCLUSIONS

Our study decomposed AFC of ASD into individual-shared and individual-specific subspaces, highlighting the importance of capturing and capitalizing on individual-specific brain connectivity features for dissecting heterogeneity. Our analysis framework provides a blueprint for parsing heterogeneity in other prevalent neurodevelopmental conditions.

Network-wise analysis of movie-specific information in dynamic functional connectivity using COBE

Movie watching during functional magnetic resonance imaging has emerged as a promising tool to measure the complex behavior of the brain in response to a stimulus similar to real-life situations. It has been observed that presenting a movie (sequence of events) as a stimulus will lead to a unique time course of dynamic functional connectivity related to movie stimuli that can be compared across the participants. We assume that the observed dynamic functional connectivity across subjects can be divided into following 2 components: (i) specific to a movie stimulus (depicting group-level behavior in functional connectivity) and (ii) individual-specific behavior (not necessarily common across the subjects). In this work, using the dynamic time warping distance measure, we have shown the extent of similarity between the temporal sequences of functional connectivity while the underlying movie stimuli were same and different. Further, the temporal sequence of functional connectivity patterns related to a movie is enhanced by suppressing the subject-specific components of dynamic functional connectivity using common and orthogonal basis extraction. Quantitative analysis using the F-ratio measure reveals significant differences in dynamic functional connectivity within the somatomotor network and default mode network, as well as between the occipital network and somatomotor networks.

Harmonizing T1-Weighted Images to Improve Consistency of Brain Morphology Among Different Scanner Manufacturers in Alzheimer's disease

BACKGROUND

Brain MRI scanner variability can introduce bias in measurements. Harmonizing scanner variability is crucial.

PURPOSE

To develop a harmonization method aimed at removing scanner variability, and to evaluate the consistency of results in multicenter studies.

STUDY TYPE

Retrospective.

POPULATION

Multicenter data from 170 healthy participants (males/females = 98/72; age = 73.8 ± 7.3) and 170 Alzheimer's disease patients (males/females = 98/72; age = 76.2 ± 8.5) were compared with reference data from another 340 participants.

FIELD STRENGTH/SEQUENCE

3-T, magnetization prepared rapid gradient echo and turbo field echo; 1.5-T, inversion recovery prepared fast spoiled gradient echo T1-weighted sequences.

ASSESSMENT

Gray matter (GM) brain images, obtained through segmentation of T1-weighted images, were utilized to evaluate the performance of the harmonization method using common orthogonal basis extraction (HCOBE) and four other methods (removal of artificial voxel effect by linear regression, RAVEL; Z_score; general linear model, GLM; ComBat). Linear discriminant analysis (LDA) was used to access the effectiveness of different methods in reducing scanner variability. The performance of harmonization methods in preserving GM volumes heterogeneity was evaluated by the similarity of the relationship between GM proportion and age in the reference and multicenter data. Furthermore, the consistency of the harmonized multicenter data with the reference data were evaluated based on classification results (train/test = 7/3) and brain atrophy.

STATISTICAL TESTS

Two-sample t-tests, area under the curve (AUC), and Dice coefficients were used to analyze the consistency of results from the reference and harmonized multicenter data. A P-value <0.01 was considered statistically significant.

RESULTS

HCOBE reduced the scanner variability from 0.09 before harmonization to 0.003 (ideal: 0, RAVEL/Z_score/GLM/ComBat = 0.087/0.003/0.006/0.013). GM volumes showed no significant difference (P = 0.52) between the reference and HCOBE-harmonized multicenter data. Consistency evaluation showed that AUC values of 0.95 for both reference and HCOBE-harmonized multicenter data (RAVEL/Z_score/GLM/ComBat = 0.86/0.86/0.84/0.89), and the Dice coefficient increased from 0.73 before harmonization to 0.82 (ideal: 1, RAVEL/Z_score/GLM/ComBat = 0.39/0.64/0.59/0.74).

DATA CONCLUSION

HCOBE may help to remove scanner variability and could improve the consistency of results in multicenter studies.

LEVEL OF EVIDENCE

2 TECHNICAL EFFICACY STAGE: 1.

Hierarchical nuclear norm penalization for multi-view data integration

The prevalence of data collected on the same set of samples from multiple sources (i.e., multi-view data) has prompted significant development of data integration methods based on low-rank matrix factorizations. These methods decompose signal matrices from each view into the sum of shared and individual structures, which are further used for dimension reduction, exploratory analyses, and quantifying associations across views. However, existing methods have limitations in modeling partially-shared structures due to either too restrictive models, or restrictive identifiability conditions. To address these challenges, we propose a new formulation for signal structures that include partially-shared signals based on grouping the views into so-called hierarchical levels with identifiable guarantees under suitable conditions. The proposed hierarchy leads us to introduce a new penalty, hierarchical nuclear norm (HNN), for signal estimation. In contrast to existing methods, HNN penalization avoids scores and loadings factorization of the signals and leads to a convex optimization problem, which we solve using a dual forward-backward algorithm. We propose a simple refitting procedure to adjust the penalization bias and develop an adapted version of bi-cross-validation for selecting tuning parameters. Extensive simulation studies and analysis of the genotype-tissue expression data demonstrate the advantages of our method over existing alternatives.

Statistical Learning Methods for Neuroimaging Data Analysis with Applications

The aim of this review is to provide a comprehensive survey of statistical challenges in neuroimaging data analysis, from neuroimaging techniques to large-scale neuroimaging studies and statistical learning methods. We briefly review eight popular neuroimaging techniques and their potential applications in neuroscience research and clinical translation. We delineate four themes of neuroimaging data and review major image processing analysis methods for processing neuroimaging data at the individual level. We briefly review four large-scale neuroimaging-related studies and a consortium on imaging genomics and discuss four themes of neuroimaging data analysis at the population level. We review nine major population-based statistical analysis methods and their associated statistical challenges and present recent progress in statistical methodology to address these challenges.

Decomposition of variation of mixed variables by a latent mixed Gaussian copula model

Many biomedical studies collect data of mixed types of variables from multiple groups of subjects. Some of these studies aim to find the group-specific and the common variation among all these variables. Even though similar problems have been studied by some previous works, their methods mainly rely on the Pearson correlation, which cannot handle mixed data. To address this issue, we propose a latent mixed Gaussian copula (LMGC) model that can quantify the correlations among binary, ordinal, continuous, and truncated variables in a unified framework. We also provide a tool to decompose the variation into the group-specific and the common variation over multiple groups via solving a regularized M-estimation problem. We conduct extensive simulation studies to show the advantage of our proposed method over the Pearson correlation-based methods. We also demonstrate that by jointly solving the M-estimation problem over multiple groups, our method is better than decomposing the variation group by group. We also apply our method to a Chlamydia trachomatis genital tract infection study to demonstrate how it can be used to discover informative biomarkers that differentiate patients.

Individualized fMRI connectivity defines signatures of antidepressant and placebo responses in major depression

Though sertraline is commonly prescribed in patients with major depressive disorder (MDD), its superiority over placebo is only marginal. This is in part due to the neurobiological heterogeneity of the individuals. Characterizing individual-unique functional architecture of the brain may help better dissect the heterogeneity, thereby defining treatment-predictive signatures to guide personalized medication. In this study, we investigate whether individualized brain functional connectivity (FC) can define more predictable signatures of antidepressant and placebo treatment in MDD. The data used in the present work were collected by the Establishing Moderators and Biosignatures of Antidepressant Response in Clinical Care (EMBARC) study. Patients (N = 296) were randomly assigned to antidepressant sertraline or placebo double-blind treatment for 8 weeks. The whole-brain FC networks were constructed from pre-treatment resting-state functional magnetic resonance imaging (rs-fMRI). Then, FC was individualized by removing the common components extracted from the raw baseline FC to train regression-based connectivity predictive models. With individualized FC features, the established prediction models successfully identified signatures that explained 22% variance for the sertraline group and 31% variance for the placebo group in predicting HAMD17 change. Compared with the raw FC-based models, the individualized FC-defined signatures significantly improved the prediction performance, as confirmed by cross-validation. For sertraline treatment, predictive FC metrics were predominantly located in the left middle temporal cortex and right insula. For placebo, predictive FC metrics were primarily located in the bilateral cingulate cortex and left superior temporal cortex. Our findings demonstrated that through the removal of common FC components, individualization of FC metrics enhanced the prediction performance compared to raw FC. Associated with previous MDD clinical studies, our identified predictive biomarkers provided new insights into the neuropathology of antidepressant and placebo treatment.

A Novel Low Rank Smooth Flat-Field Correction Algorithm for Hyperspectral Microscopy Imaging

A flat-field correction method is proposed for multiple measured hyperspectral microscopy imaging in this paper. As the most crucial preprocessing process in quantitative microscopic analysis, flat-field correction solves the uneven illumination caused by vignetting in microscopic images, and guarantees the precision of spatial and spectral information in hyperspectral microscopic imaging. In order to carry out flat-field correction and extract uneven illumination among groups of hyperspectral microscopic data containing hundreds of bands simultaneously, two properties of vignetting have been exploited: i) low-rank property is reflected by little information contained in vignetting; ii) local smoothness can be observed as a gradual change in brightness of vignetting, which is typically equivalent to the sparseness in spatial frequency domain. Combining the two properties above, a novel Low Rank Smooth Flat-field Correction (LRSFC) model modified from common orthogonal basis extraction is proposed, while an optimization is solved based on alternating direction multiplier method (ADMM), obtaining a unique flat-field term with low-rank and smooth properties. Qualitative and quantitative experimental assessments indicate that LRSFC does not add extra cell texture to the extracted flat-field term, whose performance appears prior to other state-of-the-art flat-field correction methods.

Modern views of machine learning for precision psychiatry

In light of the National Institute of Mental Health (NIMH)'s Research Domain Criteria (RDoC), the advent of functional neuroimaging, novel technologies and methods provide new opportunities to develop precise and personalized prognosis and diagnosis of mental disorders. Machine learning (ML) and artificial intelligence (AI) technologies are playing an increasingly critical role in the new era of precision psychiatry. Combining ML/AI with neuromodulation technologies can potentially provide explainable solutions in clinical practice and effective therapeutic treatment. Advanced wearable and mobile technologies also call for the new role of ML/AI for digital phenotyping in mobile mental health. In this review, we provide a comprehensive review of ML methodologies and applications by combining neuroimaging, neuromodulation, and advanced mobile technologies in psychiatry practice. We further review the role of ML in molecular phenotyping and cross-species biomarker identification in precision psychiatry. We also discuss explainable AI (XAI) and neuromodulation in a closed human-in-the-loop manner and highlight the ML potential in multi-media information extraction and multi-modal data fusion. Finally, we discuss conceptual and practical challenges in precision psychiatry and highlight ML opportunities in future research.

Interpretive JIVE: Connections with CCA and an application to brain connectivity

Joint and Individual Variation Explained (JIVE) is a model that decomposes multiple datasets obtained on the same subjects into shared structure, structure unique to each dataset, and noise. JIVE is an important tool for multimodal data integration in neuroimaging. The two most common algorithms are R.JIVE, an iterative approach, and AJIVE, which uses principal angle analysis. The joint structure in JIVE is defined by shared subspaces, but interpreting these subspaces can be challenging. In this paper, we reinterpret AJIVE as a canonical correlation analysis of principal component scores. This reformulation, which we call CJIVE, (1) provides an intuitive view of AJIVE; (2) uses a permutation test for the number of joint components; (3) can be used to predict subject scores for out-of-sample observations; and (4) is computationally fast. We conduct simulation studies that show CJIVE and AJIVE are accurate when the total signal ranks are correctly specified but, generally inaccurate when the total ranks are too large. CJIVE and AJIVE can still extract joint signal even when the joint signal variance is relatively small. JIVE methods are applied to integrate functional connectivity (resting-state fMRI) and structural connectivity (diffusion MRI) from the Human Connectome Project. Surprisingly, the edges with largest loadings in the joint component in functional connectivity do not coincide with the same edges in the structural connectivity, indicating more complex patterns than assumed in spatial priors. Using these loadings, we accurately predict joint subject scores in new participants. We also find joint scores are associated with fluid intelligence, highlighting the potential for JIVE to reveal important shared structure.

Group linear non-Gaussian component analysis with applications to neuroimaging

Independent component analysis (ICA) is an unsupervised learning method popular in functional magnetic resonance imaging (fMRI). Group ICA has been used to search for biomarkers in neurological disorders including autism spectrum disorder and dementia. However, current methods use a principal component analysis (PCA) step that may remove low-variance features. Linear non-Gaussian component analysis (LNGCA) enables simultaneous dimension reduction and feature estimation including low-variance features in single-subject fMRI. A group LNGCA model is proposed to extract group components shared by more than one subject. Unlike group ICA methods, this novel approach also estimates individual (subject-specific) components orthogonal to the group components. To determine the total number of components in each subject, a parametric resampling test is proposed that samples spatially correlated Gaussian noise to match the spatial dependence observed in data. In simulations, estimated group components achieve higher accuracy compared to group ICA. The method is applied to a resting-state fMRI study on autism spectrum disorder in 342 children (252 typically developing, 90 with autism), where the group signals include resting-state networks. The discovered group components appear to exhibit different levels of temporal engagement in autism versus typically developing children, as revealed using group LNGCA. This novel approach to matrix decomposition is a promising direction for feature detection in neuroimaging.

Multisubject Task-Related fMRI Data Processing via a Two-Stage Generalized Canonical Correlation Analysis

Functional magnetic resonance imaging (fMRI) is one of the most popular methods for studying the human brain. Task-related fMRI data processing aims to determine which brain areas are activated when a specific task is performed and is usually based on the Blood Oxygen Level Dependent (BOLD) signal. The background BOLD signal also reflects systematic fluctuations in regional brain activity which are attributed to the existence of resting-state brain networks. We propose a new fMRI data generating model which takes into consideration the existence of common task-related and resting-state components. We first estimate the common task-related temporal component, via two successive stages of generalized canonical correlation analysis and, then, we estimate the common task-related spatial component, leading to a task-related activation map. The experimental tests of our method with synthetic data reveal that we are able to obtain very accurate temporal and spatial estimates even at very low Signal to Noise Ratio (SNR), which is usually the case in fMRI data processing. The tests with real-world fMRI data show significant advantages over standard procedures based on General Linear Models (GLMs).

Accommodating Multiple Tasks' Disparities With Distributed Knowledge-Sharing Mechanism

Deep multitask learning (MTL) shares beneficial knowledge across participating tasks, alleviating the impacts of extreme learning conditions on their performances such as the data scarcity problem. In practice, participators stemming from different domain sources often have varied complexities and input sizes, for example, in the joint learning of computer vision tasks with RGB and grayscale images. For adapting to these differences, it is appropriate to design networks with proper representational capacities and construct neural layers with corresponding widths. Nevertheless, most of the state-of-the-art methods pay little attention to such situations, and actually fail to handle the disparities. To work with the dissimilitude of tasks' network designs, this article presents a distributed knowledge-sharing framework called tensor ring multitask learning (TRMTL), in which the relationship between knowledge sharing and original weight matrices is cut up. The framework of TRMTL is flexible, which is not only capable of sharing knowledge across heterogenous networks but also able to jointly learn tasks with varied input sizes, significantly improving performances of data-insufficient tasks. Comprehensive experiments on challenging datasets are conducted to empirically validate the effectiveness, efficiency, and flexibility of TRMTL in dealing with the disparities in MTL.

A Generalized Graph Regularized Non-Negative Tucker Decomposition Framework for Tensor Data Representation

Non-negative Tucker decomposition (NTD) is one of the most popular techniques for tensor data representation. To enhance the representation ability of NTD by multiple intrinsic cues, that is, manifold structure and supervisory information, in this article, we propose a generalized graph regularized NTD (GNTD) framework for tensor data representation. We first develop the unsupervised GNTD (UGNTD) method by constructing the nearest neighbor graph to maintain the intrinsic manifold structure of tensor data. Then, when limited must-link and cannot-link constraints are given, unlike most existing semisupervised learning methods that only use the pregiven supervisory information, we propagate the constraints through the entire dataset and then build a semisupervised graph weight matrix by which we can formulate the semisupervised GNTD (SGNTD). Moreover, we develop a fast and efficient alternating proximal gradient-based algorithm to solve the optimization problem and show its convergence and correctness. The experimental results on unsupervised and semisupervised clustering tasks using four image datasets demonstrate the effectiveness and high efficiency of the proposed methods.

EEG artifact rejection by extracting spatial and spatio-spectral common components

BACKGROUND

Removing artifacts is a prerequisite step for the analysis of electroencephalographic (EEG) signals. Artifacts appear in both time and time-frequency as well as spatial (multi-channel) domains.

NEW METHODS

Here, we introduce two novel methods for removing EEG artifacts. In the first method, the common components among EEG channels are extracted and eliminated as artifacts, called common component rejection (CCR). In the second method, wavelet decomposition is employed to decompose the EEG signals, then the CCR method is applied to remove artifacts in the time- frequency domain, referred to as automatic wavelet CCR (AWCCR). The proposed methods are evaluated using semi-simulated data as well as application in real EEG data for motor imaginary classification.

RESULTS

For semi-simulated data, the AWCCR showed higher performance in removing artifacts than CCR. Also, applying each of the proposed methods to the real EEG data to remove artifacts before motor imaginary classification increased the classification accuracy by about 10% compared to not removing artifacts.

COMPARISON WITH EXISTING METHODS

The proposed methods are compared with independent component analysis (ICA) and automatic wavelet ICA. AWCCR outperformed all methods in removing artifacts from semi- simulated data. The results also showed that both AWCCR and CCR methods outperformed the existing methods in removing artifacts from the real EEG data to improve the accuracy of motor imaginary classification.

CONCLUSIONS

The findings show that in ordinary or motor imaginary EEG when signatures of artifacts are shared among EEG channels, AWCCR and CCR can identify and remove the artifacts.

Individual-fMRI-approaches reveal cerebellum and visual communities to be functionally connected in obsessive compulsive disorder

There is significant interest in understanding the pathophysiology of Obsessive-Compulsive Disorder (OCD) using resting-state fMRI (rsfMRI). Previous studies acknowledge abnormalities within and beyond the fronto-striato-limbic circuit in OCD that require further clarifications. However, limited information could be inferred from the conventional way of investigating the functional connectivity differences between OCD and healthy controls. Here, we identified altered brain organization in patients with OCD by applying individual-based approaches to maximize the identification of underlying network-based features specific to the OCD group. rsfMRI of 20 patients with OCD and 22 controls were preprocessed, and individual-fMRI-subspace was derived for each subject within each group. We evaluated group differences in functional connectivity using individual-fMRI-subspace and established its advantage over conventional-fMRI methodology. We applied prediction-based approaches to highlight the group differences by evaluating the differences in functional connections that predicted the clinical scores (namely, the Obsessive-Compulsive Inventory-Revised (OCI-R) and Hamilton Anxiety Rating Scale). Then, we explored the brain network organization of both groups by estimating the subject-specific communities within each group. Lastly, we evaluated associations between the inter-individual variation of nodes in the communities to clinical measures using linear regression. Functional connectivity analysis using individual-fMRI-subspace detected 83 connections that were different between OCD and control groups, compared to none found using conventional-fMRI methodology. Connectome-based prediction analysis did not show significant overlap between the two groups in the functional connections that predicted the clinical scores. This suggests that the functional architecture in patients with OCD may be different compared to controls. Seven communities were found in both groups. Interestingly, within the OCD group but not controls, we observed functional connectivity between cerebellar and visual regions, and lack of connectivity between striato-limbic and frontal areas. Inter-individual variations in the community-size of these two communities were also associated with the OCI-R score (p < .005). Due to our small sample size, we further validated our results by (i) accounting for head motion, (ii) applying global signal regression (GSR) in data processing, and (iii) using an alternate atlas for parcellation. While the main results were consistently observed with accounting for head motion and using another atlas, the key findings were not reproduced with GSR application. The study demonstrated the existence of disconnectedness in fronto-striato-limbic community and connectedness between cerebellar and visual areas in OCD patients, which was also related to the clinical symptomatology of OCD.

Generalized integrative principal component analysis for multi-type data with block-wise missing structure

High-dimensional multi-source data are encountered in many fields. Despite recent developments on the integrative dimension reduction of such data, most existing methods cannot easily accommodate data of multiple types (e.g. binary or count-valued). Moreover, multi-source data often have block-wise missing structure, i.e. data in one or more sources may be completely unobserved for a sample. The heterogeneous data types and presence of block-wise missing data pose significant challenges to the integration of multi-source data and further statistical analyses. In this article, we develop a low-rank method, called generalized integrative principal component analysis (GIPCA), for the simultaneous dimension reduction and imputation of multi-source block-wise missing data, where different sources may have different data types. We also devise an adapted Bayesian information criterion (BIC) criterion for rank estimation. Comprehensive simulation studies demonstrate the efficacy of the proposed method in terms of rank estimation, signal recovery, and missing data imputation. We apply GIPCA to a mortality study. We achieve accurate block-wise missing data imputation and identify intriguing latent mortality rate patterns with sociological relevance.

D-CCA: A Decomposition-based Canonical Correlation Analysis for High-Dimensional Datasets

A typical approach to the joint analysis of two high-dimensional datasets is to decompose each data matrix into three parts: a low-rank common matrix that captures the shared information across datasets, a low-rank distinctive matrix that characterizes the individual information within a single dataset, and an additive noise matrix. Existing decomposition methods often focus on the orthogonality between the common and distinctive matrices, but inadequately consider the more necessary orthogonal relationship between the two distinctive matrices. The latter guarantees that no more shared information is extractable from the distinctive matrices. We propose decomposition-based canonical correlation analysis (D-CCA), a novel decomposition method that defines the common and distinctive matrices from the L 2 space of random variables rather than the conventionally used Euclidean space, with a careful construction of the orthogonal relationship between distinctive matrices. D-CCA represents a natural generalization of the traditional canonical correlation analysis. The proposed estimators of common and distinctive matrices are shown to be consistent and have reasonably better performance than some state-of-the-art methods in both simulated data and the real data analysis of breast cancer data obtained from The Cancer Genome Atlas.

Brain anatomical covariation patterns linked to binge drinking and age at first full drink

•

We identified a reproducible cortical and subcortical brain structural covariation pattern.

•

A novel pattern discovery method Joint and Individual Variance Explained (JIVE) was used.

•

The cortical and subcortical structural covariation pattern is related to alcohol use initiation.

•

The identified pattern is dominated by covariation among brainstem, thalamus and PFC.

•

A thalamic-PFC-brainstem circuitry might be related to alcohol use initiation.

Binge drinking and age at first full drink (AFD) of alcohol prior to 21 years (AFD < 21) have been linked to neuroanatomical differences in cortical and subcortical grey matter (GM) volume, cortical thickness, and surface area. Despite the importance of understanding network-level relationships, structural covariation patterns among these morphological measures have yet to be examined in relation to binge drinking and AFD < 21. Here, we used the Joint and Individual Variance Explained (JIVE) method to characterize structural covariation patterns common across and specific to morphological measures in 293 participants (149 individuals with past-12-month binge drinking and 144 healthy controls) from the Human Connectome Project (HCP). An independent dataset (Nathan Kline Institute Rockland Sample; NKI-RS) was used to examine reproducibility/generalizability. We identified a reproducible joint component dominated by structural covariation between GM volume in the brainstem and thalamus proper, and GM volume and surface area in prefrontal cortical regions. Using linear mixed regression models, we found that participants with AFD < 21 showed lower joint component scores in both the HCP (beta = 0.059, p-value = 0.016; Cohen’s d = 0.441) and NKI-RS (beta = 0.023, p-value = 0.040, Cohen’s d = 0.216) datasets, whereas the individual thickness component associated with binge drinking (p-value = 0.02) and AFD < 21 (p-value < 0.001) in the HCP dataset was not statistically significant in the NKI-RS sample. Our findings were also generalizable to the HCP full sample (n = 880 participants). Taken together, our results show that use of JIVE analysis in high-dimensional, large-scale, psychiatry-related datasets led to discovery of a reproducible cortical and subcortical structural covariation pattern involving brain regions relevant to thalamic-PFC-brainstem neural circuitry which is related to AFD < 21 and suggests a possible extension of existing addiction neurocircuitry in humans.

Error-Robust Distributed Denial of Service Attack Detection Based on an Average Common Feature Extraction Technique

In recent years, advanced threats against Cyber–Physical Systems (CPSs), such as Distributed Denial of Service (DDoS) attacks, are increasing. Furthermore, traditional machine learning-based intrusion detection systems (IDSs) often fail to efficiently detect such attacks when corrupted datasets are used for IDS training. To face these challenges, this paper proposes a novel error-robust multidimensional technique for DDoS attack detection. By applying the well-known Higher Order Singular Value Decomposition (HOSVD), initially, the average value of the common features among instances is filtered out from the dataset. Next, the filtered data are forwarded to machine learning classification algorithms in which traffic information is classified as a legitimate or a DDoS attack. In terms of results, the proposed scheme outperforms traditional low-rank approximation techniques, presenting an accuracy of 98.94%, detection rate of 97.70% and false alarm rate of 4.35% for a dataset corruption level of 30% with a random forest algorithm applied for classification. In addition, for error-free conditions, it is found that the proposed approach outperforms other related works, showing accuracy, detection rate and false alarm rate of 99.87%, 99.86% and 0.16%, respectively, for the gradient boosting classifier.

Multi-subject Task-related fMRI Data Analysis via Generalized Canonical Correlation Analysis

Functional magnetic resonance imaging (fMRI) is one of the most popular methods for studying the human brain. It measures brain activity, by detecting local changes of Blood Oxygen Level Dependent (BOLD) signal in the brain, over time, and can be used in both task-related and resting-state studies. In task-related studies, our aim is to determine which brain areas are activated when a specific task is performed. Various unsupervised multivariate statistical methods are being increasingly employed in fMRI data analysis. Their main goal is to extract information from a dataset, often with no prior knowledge of the experimental conditions. Generalized canonical correlation analysis (gCCA) is a well known statistical method that can be considered as a way to estimate a linear subspace, which is "common" to multiple random linear subspaces. We propose a new fMRI data generating model which takes into consideration the existence of common task-related and resting-state components. We estimate the common spatial task-related component via a two-stage gCCA. We test our theoretical results using real-world fMRI data. Our experimental findings corroborate our theoretical results, rendering our approach a very good candidate for multi-subject task-related fMRI processing.Clinical Relevance-This work provides a set of methods for amplifying and recovering commonalities across subjects that appear in data from multi-subject task-related fMRI experiments.

Semiparametric partial common principal component analysis for covariance matrices

We consider the problem of jointly modeling multiple covariance matrices by partial common principal component analysis (PCPCA), which assumes a proportion of eigenvectors to be shared across covariance matrices and the rest to be individual-specific. This paper proposes consistent estimators of the shared eigenvectors in the PCPCA as the number of matrices or the number of samples to estimate each matrix goes to infinity. We prove such asymptotic results without making any assumptions on the ranks of eigenvalues that are associated with the shared eigenvectors. When the number of samples goes to infinity, our results do not require the data to be Gaussian distributed. Furthermore, this paper introduces a sequential testing procedure to identify the number of shared eigenvectors in the PCPCA. In simulation studies, our method shows higher accuracy in estimating the shared eigenvectors than competing methods. Applied to a motor-task functional magnetic resonance imaging data set, our estimator identifies meaningful brain networks that are consistent with current scientific understandings of motor networks during a motor paradigm.

Hi-Net: Hybrid-Fusion Network for Multi-Modal MR Image Synthesis

Magnetic resonance imaging (MRI) is a widely used neuroimaging technique that can provide images of different contrasts (i.e., modalities). Fusing this multi-modal data has proven particularly effective for boosting model performance in many tasks. However, due to poor data quality and frequent patient dropout, collecting all modalities for every patient remains a challenge. Medical image synthesis has been proposed as an effective solution, where any missing modalities are synthesized from the existing ones. In this paper, we propose a novel Hybrid-fusion Network (Hi-Net) for multi-modal MR image synthesis, which learns a mapping from multi-modal source images (i.e., existing modalities) to target images (i.e., missing modalities). In our Hi-Net, a modality-specific network is utilized to learn representations for each individual modality, and a fusion network is employed to learn the common latent representation of multi-modal data. Then, a multi-modal synthesis network is designed to densely combine the latent representation with hierarchical features from each modality, acting as a generator to synthesize the target images. Moreover, a layer-wise multi-modal fusion strategy effectively exploits the correlations among multiple modalities, where a Mixed Fusion Block (MFB) is proposed to adaptively weight different fusion strategies. Extensive experiments demonstrate the proposed model outperforms other state-of-the-art medical image synthesis methods.

Covariate-driven factorization by thresholding for multiblock data

Multiblock data, where multiple groups of variables from different sources are observed for a common set of subjects, are routinely collected in many areas of science. Methods for joint factorization of such multiblock data are being developed to explore the potentially joint variation structure of the data. While most of the existing work focuses on delineating joint components, shared across all data blocks, from individual components, which is only relevant to a single data block, we propose to model and estimate partially joint components across some, but not all, data blocks. If covariates, with potential multiblock structures, are available, then the components are further modeled to be driven by the covariate information. To estimate such a covariate-driven, block-structured factor model, we propose an iterative algorithm based on thresholding, by transforming the problem of signal segmentation into a grouped variable selection problem. The proposed factorization provides accurate estimation of individual and (partially) joint structures in multiblock data, as confirmed by simulation studies. In the analysis of a real multiblock genomic dataset from the Cancer Genome Atlas project, we demonstrate that the estimated block structures provide straightforward interpretation and facilitate subsequent analyses.

Robust Multiview Subspace Learning With Nonindependently and Nonidentically Distributed Complex Noise

Multiview Subspace Learning (MSL), which aims at obtaining a low-dimensional latent subspace from multiview data, has been widely used in practical applications. Most recent MSL approaches, however, only assume a simple independent identically distributed (i.i.d.) Gaussian or Laplacian noise for all views of data, which largely underestimates the noise complexity in practical multiview data. Actually, in real cases, noises among different views generally have three specific characteristics. First, in each view, the data noise always has a complex configuration beyond a simple Gaussian or Laplacian distribution. Second, the noise distributions of different views of data are generally nonidentical and with evident distinctiveness. Third, noises among all views are nonindependent but obviously correlated. Based on such understandings, we elaborately construct a new MSL model by more faithfully and comprehensively considering all these noise characteristics. First, the noise in each view is modeled as a Dirichlet process (DP) Gaussian mixture model (DPGMM), which can fit a wider range of complex noise types than conventional Gaussian or Laplacian. Second, the DPGMM parameters in each view are different from one another, which encodes the "nonidentical" noise property. Third, the DPGMMs on all views share the same high-level priors by using the technique of hierarchical DP, which encodes the "nonindependent" noise property. All the aforementioned ideas are incorporated into an integrated graphics model which can be appropriately solved by the variational Bayes algorithm. The superiority of the proposed method is verified by experiments on 3-D reconstruction simulations, multiview face modeling, and background subtraction, as compared with the current state-of-the-art MSL methods.

Maximizing dissimilarity in resting state detects heterogeneous subtypes in healthy population associated with high substance use and problems in antisocial personality

Patterns in resting-state fMRI (rs-fMRI) are widely used to characterize the trait effects of brain function. In this aspect, multiple rs-fMRI scans from single subjects can provide interesting clues about the rs-fMRI patterns, though scan-to-scan variability pose challenges. Therefore, rs-fMRI's are either concatenated or the functional connectivity is averaged. This leads to loss of information. Here, we use an alternative way to extract the rs-fMRI features that are common across all the scans by applying common-and-orthogonal-basis-extraction (COBE) technique. To address this, we employed rs-fMRI of 788 subjects from the human connectome project and estimated the common-COBE-component of each subject from the four rs-fMRI runs. Since the common-COBE-component is specific to a subject, the pattern was used to classify the subjects based on the similarity/dissimilarity of the features. The subset of subjects (n = 107) with maximal-COBE-dissimilarity (MCD) was extracted and the remaining subjects (n = 681) formed the COBE-similarity (CS) group. The distribution of weights of the common-COBE-component for the two groups across rs-fMRI networks and subcortical regions was evaluated. We found the weights in the default mode network to be lower in the MCD compared to the CS. We compared the scores of 69 behavioral measures and found six behaviors related to the use of marijuana, illicit drugs, alcohol, and tobacco; and including a measure of antisocial personality to differentiate the two groups. Gender differences were also significant. Altogether the findings suggested that subtypes exist even in healthy control population, and comparison studies (case vs. control) need to be mindful of it.

Integrative factorization of bidimensionally linked matrices

Advances in molecular "omics" technologies have motivated new methodologies for the integration of multiple sources of high-content biomedical data. However, most statistical methods for integrating multiple data matrices only consider data shared vertically (one cohort on multiple platforms) or horizontally (different cohorts on a single platform). This is limiting for data that take the form of bidimensionally linked matrices (eg, multiple cohorts measured on multiple platforms), which are increasingly common in large-scale biomedical studies. In this paper, we propose bidimensional integrative factorization (BIDIFAC) for integrative dimension reduction and signal approximation of bidimensionally linked data matrices. Our method factorizes data into (a) globally shared, (b) row-shared, (c) column-shared, and (d) single-matrix structural components, facilitating the investigation of shared and unique patterns of variability. For estimation, we use a penalized objective function that extends the nuclear norm penalization for a single matrix. As an alternative to the complicated rank selection problem, we use results from the random matrix theory to choose tuning parameters. We apply our method to integrate two genomics platforms (messenger RNA and microRNA expression) across two sample cohorts (tumor samples and normal tissue samples) using the breast cancer data from the Cancer Genome Atlas. We provide R code for fitting BIDIFAC, imputing missing values, and generating simulated data.

Eliminating the Permutation Ambiguity of Convolutive Blind Source Separation by Using Coupled Frequency Bins

Blind source separation (BSS) is a typical unsupervised learning method that extracts latent components from their observations. In the meanwhile, convolutive BSS (CBSS) is particularly challenging as the observations are the mixtures of latent components as well as their delayed versions. CBSS is usually solved in frequency domain since convolutive mixtures in time domain is just instantaneous mixtures in frequency domain, which allows to recover source frequency components independently of each frequency bin by running ordinary BSS, and then concatenate them to form the Fourier transformation of source signals. Because BSS has inherent permutation ambiguity, this category of CBSS methods suffers from a common drawback: it is very difficult to choose the frequency components belonging to a specific source as they are estimated from different frequency bins using BSS. This paper presents a tensor framework that can completely eliminate the permutation ambiguity. By combining each frequency bin with an anchor frequency bin that is chosen arbitrarily in advance, we establish a new virtual BSS model where the corresponding correlation matrices comply with a block tensor decomposition (BTD) model. The essential uniqueness of BTD and the sparse structure of coupled mixing parameters allow the estimation of the mixing matrices free of permutation ambiguity. Extensive simulation results confirmed that the proposed algorithm could achieve higher separation accuracy compared with the state-of-the-art methods.

Preliminary fusion of EEG and MRI with phenotypic scores in children with epilepsy based on the Canonical Polyadic Decomposition

Cognitive and behavioural impairments in early-onset epilepsy affect the children and families' quality of life. Our ability to detect these impairments is limited, and it requires laborious questionnaires. Here, we describe a pilot study exploring the fusion of resting-state EEG, volumetric MRI, and phenotypic scores of child development based on the Canonical Polyadic Decomposition, expanding the recently presented Joint EEG-Development Inference (JEDI) model. Pilot data fusion was performed on functional, structural and developmental brain features of 29 preschool children diagnosed with epilepsy. The results suggest that combining multimodal brain data towards a comprehensive analysis of brain development in young children is plausible.

Spatial and temporal joint, partially-joint and individual sources in independent component analysis: Application to social brain fMRI dataset

absectionBackground Three types of sources can be considered in the analysis of multi-subject datasets: (i) joint sources which are common among all subjects, (ii) partially-joint sources which are common only among a subset of subjects, and (iii) individual sources which belong to each subject and represent the specific conditions of that subject. Extracting spatial and temporal joint, partially-joint, and individual sources of multi-subject datasets is of significant importance to analyze common and cross information of multiple subjects.

NEW METHOD

We present a new framework to extract these three types of spatial and temporal sources in multi-subject functional magnetic resonance imaging (fMRI) datasets. In this framework, temporal and spatial independent component analysis are utilized, and a weighted sum of higher-order cumulants is maximized.

RESULTS

We evaluate the presented algorithm by analyzing simulated data and one real multi-subject fMRI dataset. Our results on the real dataset are consistent with the existing meta-analysis studies. We show that spatial and temporal jointness of extracted joint and partially-joint sources in the theory of mind regions of brain increase with the age of subjects.

COMPARISON WITH EXISTING METHOD

In Richardson et al. (2018), predefined regions of interest (ROI) have been used to analyze the real dataset, whereas our unified algorithm simultaneously extracts activated and uncorrelated ROIs, and determines their spatial and temporal jointness without additional computations.

CONCLUSIONS

Extracting temporal and spatial joint and partially-joint sources in a unified algorithm improves the accuracy of joint analysis of the multi-subject fMRI dataset.

Structural learning and integrative decomposition of multi-view data

The increased availability of multi-view data (data on the same samples from multiple sources) has led to strong interest in models based on low-rank matrix factorizations. These models represent each data view via shared and individual components, and have been successfully applied for exploratory dimension reduction, association analysis between the views, and consensus clustering. Despite these advances, there remain challenges in modeling partially-shared components and identifying the number of components of each type (shared/partially-shared/individual). We formulate a novel linked component model that directly incorporates partially-shared structures. We call this model SLIDE for Structural Learning and Integrative DEcomposition of multi-view data. The proposed model-fitting and selection techniques allow for joint identification of the number of components of each type, in contrast to existing sequential approaches. In our empirical studies, SLIDE demonstrates excellent performance in both signal estimation and component selection. We further illustrate the methodology on the breast cancer data from The Cancer Genome Atlas repository.

Discriminative Low-Rank Subspace Learning with Nonconvex Penalty

Subspace learning has been widely utilized to extract discriminative features for classification task, such as face recognition, even when facial images are occluded or corrupted. However, the performance of most existing methods would be degraded significantly in the scenario of that data being contaminated with severe noise, especially when the magnitude of the gross corruption can be arbitrarily large. To this end, in this paper, a novel discriminative subspace learning method is proposed based on the well-known low-rank representation (LRR). Specifically, a discriminant low-rank representation and the projecting subspace are learned simultaneously, in a supervised way. To avoid the deviation from the original solution by using some relaxation, we adopt the Schatten [Formula: see text]-norm and [Formula: see text]-norm, instead of the nuclear norm and [Formula: see text]-norm, respectively. Experimental results on two famous databases, i.e. PIE and ORL, demonstrate that the proposed method achieves better classification scores than the state-of-the-art approaches.

Temporally Constrained Sparse Group Spatial Patterns for Motor Imagery BCI

Common spatial pattern (CSP)-based spatial filtering has been most popularly applied to electroencephalogram (EEG) feature extraction for motor imagery (MI) classification in brain-computer interface (BCI) application. The effectiveness of CSP is highly affected by the frequency band and time window of EEG segments. Although numerous algorithms have been designed to optimize the spectral bands of CSP, most of them selected the time window in a heuristic way. This is likely to result in a suboptimal feature extraction since the time period when the brain responses to the mental tasks occurs may not be accurately detected. In this paper, we propose a novel algorithm, namely temporally constrained sparse group spatial pattern (TSGSP), for the simultaneous optimization of filter bands and time window within CSP to further boost classification accuracy of MI EEG. Specifically, spectrum-specific signals are first derived by bandpass filtering from raw EEG data at a set of overlapping filter bands. Each of the spectrum-specific signals is further segmented into multiple subseries using sliding window approach. We then devise a joint sparse optimization of filter bands and time windows with temporal smoothness constraint to extract robust CSP features under a multitask learning framework. A linear support vector machine classifier is trained on the optimized EEG features to accurately identify the MI tasks. An experimental study is implemented on three public EEG datasets (BCI Competition III dataset IIIa, BCI Competition IV datasets IIa, and BCI Competition IV dataset IIb) to validate the effectiveness of TSGSP in comparison to several other competing methods. Superior classification performance (averaged accuracies are 88.5%, 83.3%, and 84.3% for the three datasets, respectively) based on the experimental results confirms that the proposed algorithm is a promising candidate for performance improvement of MI-based BCIs.