Tensor-on-tensor regression

Tensor response quantile regression with neuroimaging data

Collecting neuroimaging data in the form of tensors (i.e. multidimensional arrays) has become more common in mental health studies, driven by an increasing interest in studying the associations between neuroimaging phenotypes and clinical disease manifestation. Motivated by a neuroimaging study of post-traumatic stress disorder (PTSD) from the Grady Trauma Project, we study a tensor response quantile regression framework, which enables novel analyses that confer a detailed view of the potentially heterogeneous association between a neuroimaging phenotype and relevant clinical predictors. We adopt a sensible low-rank structure to represent the association of interest, and propose a simple two-step estimation procedure which is easy to implement with existing software. We provide rigorous theoretical justifications for the intuitive two-step procedure. Simulation studies demonstrate good performance of the proposed method with realistic sample sizes in neuroimaging studies. We conduct the proposed tensor response quantile regression analysis of the motivating PTSD study to investigate the association between fMRI resting-state functional connectivity and PTSD symptom severity. Our results uncover non-homogeneous effects of PTSD symptoms on brain functional connectivity, which cannot be captured by existing tensor response methods.

Imputed mean tensor regression for near-sited spatial temporal data

Modern spatial temporal data are often collected from sensor networks. Missing data problems are common to this kind of data. Making accurate imputation is important for many applications. In the unsupervised setting, one technique is to minimize the rank of a tensor or matrix. If we add related covariates, can we get more accurate imputation results? To address this, we transform the original sensor×time measurements to high order tensors by adding additional temporal dimensions and then integrate tensor regression with tensor completion using nuclear norm penalty. One advantage is we can simultaneously estimate parameters and impute missing values due to clear spatial consistency for near-sited spatial-temporal data. The proposed method doesn't assume missing mechanism of the response. Theoretical properties of the proposed estimator are investigated. Simulation studies and real data analysis are conducted to verify the efficiency of the estimation procedure.

Reduced-Rank Tensor-on-Tensor Regression and Tensor-Variate Analysis of Variance

Fitting regression models with many multivariate responses and covariates can be challenging, but such responses and covariates sometimes have tensor-variate structure. We extend the classical multivariate regression model to exploit such structure in two ways: first, we impose four types of low-rank tensor formats on the regression coefficients. Second, we model the errors using the tensor-variate normal distribution that imposes a Kronecker separable format on the covariance matrix. We obtain maximum likelihood estimators via block-relaxation algorithms and derive their computational complexity and asymptotic distributions. Our regression framework enables us to formulate tensor-variate analysis of variance (TANOVA) methodology. This methodology, when applied in a one-way TANOVA layout, enables us to identify cerebral regions significantly associated with the interaction of suicide attempters or non-attemptor ideators and positive-, negative- or death-connoting words in a functional Magnetic Resonance Imaging study. Another application uses three-way TANOVA on the Labeled Faces in the Wild image dataset to distinguish facial characteristics related to ethnic origin, age group and gender. A R package totr implements the methodology.

Multiway sparse distance weighted discrimination

Modern data often take the form of a multiway array. However, most classification methods are designed for vectors, i.e., 1-way arrays. Distance weighted discrimination (DWD) is a popular high-dimensional classification method that has been extended to the multiway context, with dramatic improvements in performance when data have multiway structure. However, the previous implementation of multiway DWD was restricted to classification of matrices, and did not account for sparsity. In this paper, we develop a general framework for multiway classification which is applicable to any number of dimensions and any degree of sparsity. We conducted extensive simulation studies, showing that our model is robust to the degree of sparsity and improves classification accuracy when the data have multiway structure. For our motivating application, magnetic resonance spectroscopy (MRS) was used to measure the abundance of several metabolites across multiple neurological regions and across multiple time points in a mouse model of Friedreich's ataxia, yielding a four-way data array. Our method reveals a robust and interpretable multi-region metabolomic signal that discriminates the groups of interest. We also successfully apply our method to gene expression time course data for multiple sclerosis treatment. An R implementation is available in the package MultiwayClassification at http://github.com/lockEF/MultiwayClassification.

Tensor envelope mixture model for simultaneous clustering and multiway dimension reduction

In the form of multidimensional arrays, tensor data have become increasingly prevalent in modern scientific studies and biomedical applications such as computational biology, brain imaging analysis, and process monitoring system. These data are intrinsically heterogeneous with complex dependencies and structure. Therefore, ad-hoc dimension reduction methods on tensor data may lack statistical efficiency and can obscure essential findings. Model-based clustering is a cornerstone of multivariate statistics and unsupervised learning; however, existing methods and algorithms are not designed for tensor-variate samples. In this article, we propose a tensor envelope mixture model (TEMM) for simultaneous clustering and multiway dimension reduction of tensor data. TEMM incorporates tensor-structure-preserving dimension reduction into mixture modeling and drastically reduces the number of free parameters and estimative variability. An expectation-maximization-type algorithm is developed to obtain likelihood-based estimators of the cluster means and covariances, which are jointly parameterized and constrained onto a series of lower dimensional subspaces known as the tensor envelopes. We demonstrate the encouraging empirical performance of the proposed method in extensive simulation studies and a real data application in comparison with existing vector and tensor clustering methods.

Gene-Set Integrative Analysis of Multi-Omics Data Using Tensor-based Association Test

MOTIVATION

Facilitated by technological advances and the decrease in costs, it is feasible to gather subject data from several omics platforms. Each platform assesses different molecular events, and the challenge lies in efficiently analyzing these data to discover novel disease genes or mechanisms. A common strategy is to regress the outcomes on all omics variables in a gene set. However, this approach suffers from problems associated with high-dimensional inference.

RESULTS

We introduce a tensor-based framework for variable-wise inference in multi-omics analysis. By accounting for the matrix structure of an individual's multi-omics data, the proposed tensor methods incorporate the relationship among omics effects, reduce the number of parameters, and boost the modeling efficiency. We derive the variable-specific tensor test and enhance computational efficiency of tensor modeling. Using simulations and data applications on the Cancer Cell Line Encyclopedia (CCLE), we demonstrate our method performs favorably over baseline methods and will be useful for gaining biological insights in multi-omics analysis.

AVAILABILITY AND IMPLEMENTATION

R function and instruction are available from the authors' website: https://www4.stat.ncsu.edu/∼jytzeng/Software/TR.omics/TRinstruction.pdf.

SUPPLEMENTARY INFORMATION

Supplementary materials are available at Bioinformatics online.

Low-Rank Tensor Train Coefficient Array Estimation for Tensor-on-Tensor Regression

The tensor-on-tensor regression can predict a tensor from a tensor, which generalizes most previous multilinear regression approaches, including methods to predict a scalar from a tensor, and a tensor from a scalar. However, the coefficient array could be much higher dimensional due to both high-order predictors and responses in this generalized way. Compared with the current low CANDECOMP/PARAFAC (CP) rank approximation-based method, the low tensor train (TT) approximation can further improve the stability and efficiency of the high or even ultrahigh-dimensional coefficient array estimation. In the proposed low TT rank coefficient array estimation for tensor-on-tensor regression, we adopt a TT rounding procedure to obtain adaptive ranks, instead of selecting ranks by experience. Besides, an l₂ constraint is imposed to avoid overfitting. The hierarchical alternating least square is used to solve the optimization problem. Numerical experiments on a synthetic data set and two real-life data sets demonstrate that the proposed method outperforms the state-of-the-art methods in terms of prediction accuracy with comparable computational complexity, and the proposed method is more computationally efficient when the data are high dimensional with small size in each mode.

Multiway generalized canonical correlation analysis

Regularized generalized canonical correlation analysis (RGCCA) is a general multiblock data analysis framework that encompasses several important multivariate analysis methods such as principal component analysis, partial least squares regression, and several versions of generalized canonical correlation analysis. In this article, we extend RGCCA to the case where at least one block has a tensor structure. This method is called multiway generalized canonical correlation analysis (MGCCA). Convergence properties of the MGCCA algorithm are studied, and computation of higher-level components are discussed. The usefulness of MGCCA is shown on simulation and on the analysis of a cognitive study in human infants using electroencephalography (EEG).