Simulation-based hypothesis testing of high dimensional means under covariance heterogeneity

Nonparametric two-sample tests of high dimensional mean vectors via random integration

Testing the equality of the means in two samples is a fundamental statistical inferential problem. Most of the existing methods are based on the sum-of-squares or supremum statistics. They are possibly powerful in some situations, but not in others, and they do not work in a unified way. Using random integration of the difference, we develop a framework that includes and extends many existing methods, especially in high-dimensional settings, without restricting the same covariance matrices or sparsity. Under a general multivariate model, we can derive the asymptotic properties of the proposed test statistic without specifying a relationship between the data dimension and sample size explicitly. Specifically, the new framework allows us to better understand the test's properties and select a powerful procedure accordingly. For example, we prove that our proposed test can achieve the power of 1 when nonzero signals in the true mean differences are weakly dense with nearly the same sign. In addition, we delineate the conditions under which the asymptotic relative Pitman efficiency of our proposed test to its competitor is greater than or equal to 1. Extensive numerical studies and a real data example demonstrate the potential of our proposed test.

Projection Test for Mean Vector in High Dimensions

This paper studies the projection test for high-dimensional mean vectors via optimal projection. The idea of projection test is to project high-dimensional data onto a space of low dimension such that traditional methods can be applied. We first propose a new estimation for the optimal projection direction by solving a constrained and regularized quadratic programming. Then two tests are constructed using the estimated optimal projection direction. The first one is based on a data-splitting procedure, which achieves an exact t -test under normality assumption. To mitigate the power loss due to data-splitting, we further propose an online framework, which iteratively updates the estimation of projection direction when new observations arrive. We show that this online-style projection test asymptotically converges to the standard normal distribution. Various simulation studies as well as a real data example show that the proposed online-style projection test retains the type I error rate well and is more powerful than other existing tests.

An approximate randomization test for the high-dimensional two-sample Behrens–Fisher problem under arbitrary covariances

This paper is concerned with the problem of comparing the population means of two groups of independent observations. An approximate randomization test procedure based on the test statistic of? is proposed. The asymptotic behaviour of the test statistic as well as the randomized statistic is studied under weak conditions. In our theoretical framework, observations are not assumed to be identically distributed even within groups. No condition on the eigenstructure of the covariance matrices is imposed. And the sample sizes of the two groups are allowed to be unbalanced. Under general conditions, all possible asymptotic distributions of the test statistic are obtained. We derive the asymptotic level and local power of the approximate randomization test procedure. Our theoretical results show that the proposed test procedure can adapt to all possible asymptotic distributions of the test statistic, always has correct test level asymptotically, and has good power behaviour. Our numerical experiments show that the proposed test procedure has favourable performance compared with several alternative test procedures.

Testing for unit roots based on sample autocovariances

We propose a new unit-root test for a stationary null hypothesis $H_0$ against a unit-root alternative $H_1$. Our approach is nonparametric as $H_0$ assumes only that the process concerned is $I(0)$, without specifying any parametric forms. The new test is based on the fact that the sample autocovariance function converges to the finite population autocovariance function for an $I(0)$ process, but diverges to infinity for a process with unit roots. Therefore, the new test rejects $H_0$ for large values of the sample autocovariance function. To address the technical question of how large is large, we split the sample and establish an appropriate normal approximation for the null distribution of the test statistic. The substantial discriminative power of the new test statistic is due to the fact that it takes finite values under $H_0$ and diverges to infinity under $H_1$. This property allows one to truncate the critical values of the test so that it has asymptotic power 1; it also alleviates the loss of power due to the sample-splitting. The test is implemented in $\texttt{R}$.

Drug screening with zebrafish visual behavior identifies carvedilol as a potential treatment for an autosomal dominant form of retinitis pigmentosa

Retinitis Pigmentosa (RP) is a mostly incurable inherited retinal degeneration affecting approximately 1 in 4000 individuals globally. The goal of this work was to identify drugs that can help patients suffering from the disease. To accomplish this, we screened drugs on a zebrafish autosomal dominant RP model. This model expresses a truncated human rhodopsin transgene (Q344X) causing significant rod degeneration by 7 days post-fertilization (dpf). Consequently, the larvae displayed a deficit in visual motor response (VMR) under scotopic condition. The diminished VMR was leveraged to screen an ENZO SCREEN-WELL REDOX library since oxidative stress is postulated to play a role in RP progression. Our screening identified a beta-blocker, carvedilol, that ameliorated the deficient VMR of the RP larvae and increased their rod number. Carvedilol may directly on rods as it affected the adrenergic pathway in the photoreceptor-like human Y79 cell line. Since carvedilol is an FDA-approved drug, our findings suggest that carvedilol can potentially be repurposed to treat autosomal dominant RP patients.

Small sample sizes: A big data problem in high-dimensional data analysis

In many experiments and especially in translational and preclinical research, sample sizes are (very) small. In addition, data designs are often high dimensional, i.e. more dependent than independent replications of the trial are observed. The present paper discusses the applicability of max t-test-type statistics (multiple contrast tests) in high-dimensional designs (repeated measures or multivariate) with small sample sizes. A randomization-based approach is developed to approximate the distribution of the maximum statistic. Extensive simulation studies confirm that the new method is particularly suitable for analyzing data sets with small sample sizes. A real data set illustrates the application of the methods.

High-dimensional empirical likelihood inference

High-dimensional statistical inference with general estimating equations is challenging and remains little explored. We study two problems in the area: confidence set estimation for multiple components of the model parameters, and model specifications tests. First, we propose to construct a new set of estimating equations such that the impact from estimating the high-dimensional nuisance parameters becomes asymptotically negligible. The new construction enables us to estimate a valid confidence region by empirical likelihood ratio. Second, we propose a test statistic as the maximum of the marginal empirical likelihood ratios to quantify data evidence against the model specification. Our theory establishes the validity of the proposed empirical likelihood approaches, accommodating over-identification and exponentially growing data dimensionality. Numerical studies demonstrate promising performance and potential practical benefits of the new methods.

Specification tests for covariance structures in high-dimensional statistical models

We consider testing the covariance structure in statistical models. We focus on developing such tests when the random vectors of interest are not directly observable and have to be derived via estimated models. Additionally, the covariance specification may involve extra nuisance parameters which also need to be estimated. In a generic additive model setting, we develop and investigate test statistics based on the maximum discrepancy measure calculated from the residuals. To approximate the distributions of the test statistics under the null hypothesis, new multiplier bootstrap procedures with dedicated adjustments that incorporate the model and nuisance parameter estimation errors are proposed. Our theoretical development elucidates the impact due to the estimation errors with high-dimensional data and demonstrates the validity of our tests. Simulations and real data examples confirm our theory and demonstrate the performance of the proposed tests.

Better-than-chance classification for signal detection

The estimated accuracy of a classifier is a random quantity with variability. A common practice in supervised machine learning, is thus to test if the estimated accuracy is significantly better than chance level. This method of signal detection is particularly popular in neuroimaging and genetics. We provide evidence that using a classifier's accuracy as a test statistic can be an underpowered strategy for finding differences between populations, compared to a bona fide statistical test. It is also computationally more demanding than a statistical test. Via simulation, we compare test statistics that are based on classification accuracy, to others based on multivariate test statistics. We find that the probability of detecting differences between two distributions is lower for accuracy-based statistics. We examine several candidate causes for the low power of accuracy-tests. These causes include: the discrete nature of the accuracy-test statistic, the type of signal accuracy-tests are designed to detect, their inefficient use of the data, and their suboptimal regularization. When the purpose of the analysis is the evaluation of a particular classifier, not signal detection, we suggest several improvements to increase power. In particular, to replace V-fold cross-validation with the Leave-One-Out Bootstrap.

ARE DISCOVERIES SPURIOUS? DISTRIBUTIONS OF MAXIMUM SPURIOUS CORRELATIONS AND THEIR APPLICATIONS

Over the last two decades, many exciting variable selection methods have been developed for finding a small group of covariates that are associated with the response from a large pool. Can the discoveries by such data mining approaches be spurious due to high dimensionality and limited sample size? Can our fundamental assumptions on exogeneity of covariates needed for such variable selection be validated with the data? To answer these questions, we need to derive the distributions of the maximum spurious correlations given certain number of predictors, namely, the distribution of the correlation of a response variable Y with the best s linear combinations of p covariates X, even when X and Y are independent. When the covariance matrix of X possesses the restricted eigenvalue property, we derive such distributions for both finite s and diverging s, using Gaussian approximation and empirical process techniques. However, such a distribution depends on the unknown covariance matrix of X. Hence, we use the multiplier bootstrap procedure to approximate the unknown distributions and establish the consistency of such a simple bootstrap approach. The results are further extended to the situation where residuals are from regularized fits. Our approach is then applied to construct the upper confidence limit for the maximum spurious correlation and testing exogeneity of covariates. The former provides a baseline for guarding against false discoveries due to data mining and the latter tests whether our fundamental assumptions for high-dimensional model selection are statistically valid. Our techniques and results are illustrated by both numerical examples and real data analysis.