Products of rectangular random matrices: singular values and progressive scattering

Quadratic Forms in Random Matrices with Applications in Spectrum Sensing

Quadratic forms with random kernel matrices are ubiquitous in applications of multivariate statistics, ranging from signal processing to time series analysis, biomedical systems design, wireless communications performance analysis, and other fields. Their statistical characterization is crucial to both design guideline formulation and efficient computation of performance indices. To this end, random matrix theory can be successfully exploited. In particular, recent advancements in spectral characterization of finite-dimensional random matrices from the so-called polynomial ensembles allow for the analysis of several scenarios of interest in wireless communications and signal processing. In this work, we focus on the characterization of quadratic forms in unit-norm vectors, with unitarily invariant random kernel matrices, and we also provide some approximate but numerically accurate results concerning a non-unitarily invariant kernel matrix. Simulations are run with reference to a peculiar application scenario, the so-called spectrum sensing for wireless communications. Closed-form expressions for the moment generating function of the quadratic forms of interest are provided; this will pave the way to an analytical performance analysis of some spectrum sensing schemes, and will potentially assist in the rate analysis of some multi-antenna systems.

Random density matrices: Analytical results for mean fidelity and variance of squared Bures distance

One of the key issues in problems related to quantum information theory is concerned with the distinguishability of quantum states. In this context, Bures distance serves as one of the foremost choices among various distance measures. It also relates to fidelity, which is another quantity of immense importance in quantum information theory. In this work we derive exact results for the average fidelity and variance of the squared Bures distance between a fixed density matrix and a random density matrix and also between two independent random density matrices. These results go beyond the recently obtained results for the mean root fidelity and mean of the squared Bures distance. The availability of both mean and variance also enables us to provide a gamma-distribution-based approximation for the probability density of the squared Bures distance. The analytical results are corroborated using Monte Carlo simulations. Furthermore, we compare our analytical results with the mean and variance of the squared Bures distance between reduced density matrices generated using coupled kicked tops and a correlated spin chain system in a random magnetic field. In both cases, we find good agreement.

Statistical limits of dictionary learning: Random matrix theory and the spectral replica method

We consider increasingly complex models of matrix denoising and dictionary learning in the Bayes-optimal setting, in the challenging regime where the matrices to infer have a rank growing linearly with the system size. This is in contrast with most existing literature concerned with the low-rank (i.e., constant-rank) regime. We first consider a class of rotationally invariant matrix denoising problems whose mutual information and minimum mean-square error are computable using techniques from random matrix theory. Next, we analyze the more challenging models of dictionary learning. To do so we introduce a combination of the replica method from statistical mechanics together with random matrix theory, coined spectral replica method. This allows us to derive variational formulas for the mutual information between hidden representations and the noisy data of the dictionary learning problem, as well as for the overlaps quantifying the optimal reconstruction error. The proposed method reduces the number of degrees of freedom from Θ(N^{2}) matrix entries to Θ(N) eigenvalues (or singular values), and yields Coulomb gas representations of the mutual information which are reminiscent of matrix models in physics. The main ingredients are a combination of large deviation results for random matrices together with a replica symmetric decoupling ansatz at the level of the probability distributions of eigenvalues (or singular values) of certain overlap matrices and the use of Harish-Chandra-Itzykson-Zuber spherical integrals.

Contrasting random and learned features in deep Bayesian linear regression

Understanding how feature learning affects generalization is among the foremost goals of modern deep learning theory. Here, we study how the ability to learn representations affects the generalization performance of a simple class of models: deep Bayesian linear neural networks trained on unstructured Gaussian data. By comparing deep random feature models to deep networks in which all layers are trained, we provide a detailed characterization of the interplay between width, depth, data density, and prior mismatch. We show that both models display samplewise double-descent behavior in the presence of label noise. Random feature models can also display modelwise double descent if there are narrow bottleneck layers, while deep networks do not show these divergences. Random feature models can have particular widths that are optimal for generalization at a given data density, while making neural networks as wide or as narrow as possible is always optimal. Moreover, we show that the leading-order correction to the kernel-limit learning curve cannot distinguish between random feature models and deep networks in which all layers are trained. Taken together, our findings begin to elucidate how architectural details affect generalization performance in this simple class of deep regression models.

Universality of local spectral statistics of products of random matrices

We derive exact analytical expressions for correlation functions of singular values of the product of M Ginibre matrices of size N in the double scaling limit M,N→∞. The singular value statistics is described by a determinantal point process with a kernel that interpolates between Gaussian unitary ensemble statistic and Dirac-delta (picket-fence) statistic. In the thermodynamic limit N→∞, the interpolation parameter is given by the limiting quotient a=N/M. One of our goals is to find an explicit form of the kernel at the hard edge, in the bulk, and at the soft edge for any a. We find that in addition to the standard scaling regimes, there is a transitional regime which interpolates between the hard edge and the bulk. We conjecture that these results are universal, and that they apply to a broad class of products of random matrices from the Gaussian basin of attraction, including correlated matrices. We corroborate this conjecture by numerical simulations. Additionally, we show that the local spectral statistics of the considered random matrix products is identical with the local statistics of Dyson Brownian motion with the initial condition given by equidistant positions, with the crucial difference that this equivalence holds only locally. Finally, we have identified a mesoscopic spectral scale at the soft edge which is crucial for the unfolding of the spectrum.

On Products of Random Matrices

We introduce a family of models, which we name matrix models associated with children's drawings-the so-called dessin d'enfant. Dessins d'enfant are graphs of a special kind drawn on a closed connected orientable surface (in the sky). The vertices of such a graph are small disks that we call stars. We attach random matrices to the edges of the graph and get multimatrix models. Additionally, to the stars we attach source matrices. They play the role of free parameters or model coupling constants. The answers for our integrals are expressed through quantities that we call the "spectrum of stars". The answers may also include some combinatorial numbers, such as Hurwitz numbers or characters from group representation theory.

Eigenvector statistics of the product of Ginibre matrices

We develop a method to calculate left-right eigenvector correlations of the product of m independent N×N complex Ginibre matrices. For illustration, we present explicit analytical results for the vector overlap for a couple of examples for small m and N. We conjecture that the integrated overlap between left and right eigenvectors is given by the formula O=1+(m/2)(N-1) and support this conjecture by analytical and numerical calculations. We derive an analytical expression for the limiting correlation density as N→∞ for the product of Ginibre matrices as well as for the product of elliptic matrices. In the latter case, we find that the correlation function is independent of the eccentricities of the elliptic laws.

Random matrix ensembles involving Gaussian Wigner and Wishart matrices, and biorthogonal structure

We consider four nontrivial ensembles involving Gaussian Wigner and Wishart matrices. These are relevant to problems ranging from multiantenna communication to random supergravity. We derive the matrix probability density, as well as the eigenvalue densities for these ensembles. In all cases the joint eigenvalue density exhibits a biorthogonal structure. A determinantal representation, based on a generalization of Andréief's integration formula, is used to compactly express the r-point correlation function of eigenvalues. This representation circumvents the complications encountered in the usual approaches, and the answer is obtained immediately by examining the joint density of eigenvalues. We validate our analytical results using Monte Carlo simulations.

Spectral density of generalized Wishart matrices and free multiplicative convolution

We investigate the level density for several ensembles of positive random matrices of a Wishart-like structure, W=XX(†), where X stands for a non-Hermitian random matrix. In particular, making use of the Cauchy transform, we study the free multiplicative powers of the Marchenko-Pastur (MP) distribution, MP(⊠s), which for an integer s yield Fuss-Catalan distributions corresponding to a product of s-independent square random matrices, X=X(1)⋯X(s). New formulas for the level densities are derived for s=3 and s=1/3. Moreover, the level density corresponding to the generalized Bures distribution, given by the free convolution of arcsine and MP distributions, is obtained. We also explain the reason of such a curious convolution. The technique proposed here allows for the derivation of the level densities for several other cases.

Spectral domain of large nonsymmetric correlated Wishart matrices

We study complex eigenvalues of the Wishart model for nonsymmetric correlation matrices. The model is defined for two statistically equivalent but different Gaussian real matrices, as C=AB(t)/T, where B(t) is the transpose of B and both matrices A and B are of dimensions N×T. If A and B are uncorrelated, or equivalently if C vanishes on average, it is known that at large matrix dimension the domain of the eigenvalues of C is a circle centered-at-origin and the eigenvalue density depends only on the radial distances. We consider actual correlation in A and B and derive a result for the contour describing the domain of the bulk of the eigenvalues of C in the limit of large N and T where the ratio N/T is finite. In particular, we show that the eigenvalue domain is sensitive to the correlations. For example, when C is diagonal on average with the same element c≠0, the contour is no longer a circle centered at origin but a shifted ellipse. In this case we explicitly derive a result for the spectral density which again depends only on the radial distances. For more general cases, we show that the contour depends on the symmetric and antisymmetric parts of the correlation matrix resulting from the ensemble-averaged C. If the correlation matrix is normal, then the contour depends only on its spectrum. We also provide numerics to justify our analytics.

Weak commutation relations and eigenvalue statistics for products of rectangular random matrices

We study the joint probability density of the eigenvalues of a product of rectangular real, complex, or quaternion random matrices in a unified way. The random matrices are distributed according to arbitrary probability densities, whose only restriction is the invariance under left and right multiplication by orthogonal, unitary, or unitary symplectic matrices, respectively. We show that a product of rectangular matrices is statistically equivalent to a product of square matrices. Hereby we prove a weak commutation relation of the random matrices at finite matrix sizes, which previously has been discussed for infinite matrix size. Moreover, we derive the joint probability densities of the eigenvalues. To illustrate our results, we apply them to a product of random matrices drawn from Ginibre ensembles and Jacobi ensembles as well as a mixed version thereof. For these weights, we show that the product of complex random matrices yields a determinantal point process, while the real and quaternion matrix ensembles correspond to Pfaffian point processes. Our results are visualized by numerical simulations. Furthermore, we present an application to a transport on a closed, disordered chain coupled to a particle bath.