Peak p-values and false discovery rate inference in neuroimaging

Evaluation of resampling-based inference for topological features of neuroimages

Many recent studies have demonstrated the inflated type 1 error rate of the original Gaussian random field (GRF) methods for inference of neuroimages and identified resampling (permutation and bootstrapping) methods that have better performance. There has been no evaluation of resampling procedures when using robust (sandwich) statistical images with different topological features (TF) used for neuroimaging inference. Here, we consider estimation of distributions TFs of a statistical image and evaluate resampling procedures that can be used when exchangeability is violated. We compare the methods using realistic simulations and study sex differences in life-span age-related changes in gray matter volume in the Nathan Kline Institute Rockland sample. We find that our proposed wild bootstrap and the commonly used permutation procedure perform well in sample sizes above 50 under realistic simulations with heteroskedasticity. The Rademacher wild bootstrap has fewer assumptions than the permutation and performs similarly in samples of 100 or more, so is valid in a broader range of conditions. We also evaluate the GRF-based pTFCE method and show that it has inflated error rates in samples less than 200. Our R package, pbj , is available on Github and allows the user to reproducibly implement various resampling-based group level neuroimage analyses.

Controlling the familywise error rate in widefield optical neuroimaging of functional connectivity in mice

SIGNIFICANCE

Statistical inference in functional neuroimaging is complicated by the multiple testing problem and spatial autocorrelation. Common methods in functional magnetic resonance imaging to control the familywise error rate (FWER) include random field theory (RFT) and permutation testing. The ability of these methods to control the FWER in optical neuroimaging has not been evaluated.

AIM

We attempt to control the FWER in optical intrinsic signal imaging resting-state functional connectivity using both RFT and permutation inference at a nominal value of 0.05. The FWER was derived using a mass empirical analysis of real data in which the null is known to be true.

APPROACH

Data from normal mice were repeatedly divided into two groups, and differences between functional connectivity maps were calculated with pixel-wise t -tests. As the null hypothesis was always true, all positives were false positives.

RESULTS

Gaussian RFT resulted in a higher than expected FWER with either cluster-based (0.15) or pixel-based (0.62) methods. t -distribution RFT could achieve FWERs of 0.05 (cluster-based or pixel-based). Permutation inference always controlled the FWER.

CONCLUSIONS

RFT can lead to highly inflated FWERs. Although t -distribution RFT can be accurate, it is sensitive to statistical assumptions. Permutation inference is robust to statistical errors and accurately controls the FWER.

Bayes estimate of primary threshold in clusterwise functional magnetic resonance imaging inferences

Clusterwise statistical inference is the most widely used technique for functional magnetic resonance imaging (fMRI) data analyses. Clusterwise statistical inference consists of two steps: (i) primary thresholding that excludes less significant voxels by a prespecified cut-off (eg, p < . 001 ); and (ii) clusterwise thresholding that controls the familywise error rate caused by clusters consisting of false positive suprathreshold voxels. The selection of the primary threshold is critical because it determines both statistical power and false discovery rate (FDR). However, in most existing statistical packages, the primary threshold is selected based on prior knowledge (eg, p < . 001 ) without taking into account the information in the data. In this article, we propose a data-driven approach to algorithmically select the optimal primary threshold based on an empirical Bayes framework. We evaluate the proposed model using extensive simulation studies and real fMRI data. In the simulation, we show that our method can effectively increase statistical power by 20% to over 100% while effectively controlling the FDR. We then investigate the brain response to the dose-effect of chlorpromazine in patients with schizophrenia by analyzing fMRI scans and generate consistent results.

A comparison of random-field-theory and false-discovery-rate inference results in the analysis of registered one-dimensional biomechanical datasets

Background

The inflation of falsely rejected hypotheses associated with multiple hypothesis testing is seen as a threat to the knowledge base in the scientific literature. One of the most recently developed statistical constructs to deal with this problem is the false discovery rate (FDR), which aims to control the proportion of the falsely rejected null hypotheses among those that are rejected. FDR has been applied to a variety of problems, especially for the analysis of 3-D brain images in the field of Neuroimaging, where the predominant form of statistical inference involves the more conventional control of false positives, through Gaussian random field theory (RFT). In this study we considered FDR and RFT as alternative methods for handling multiple testing in the analysis of 1-D continuum data. The field of biomechanics has recently adopted RFT, but to our knowledge FDR has not previously been used to analyze 1-D biomechanical data, nor has there been a consideration of how FDR vs. RFT can affect biomechanical interpretations.

Methods

We reanalyzed a variety of publicly available experimental datasets to understand the characteristics which contribute to the convergence and divergence of RFT and FDR results. We also ran a variety of numerical simulations involving smooth, random Gaussian 1-D data, with and without true signal, to provide complementary explanations for the experimental results.

Results

Our results suggest that RFT and FDR thresholds (the critical test statistic value used to judge statistical significance) were qualitatively identical for many experimental datasets, but were highly dissimilar for others, involving non-trivial changes in data interpretation. Simulation results clarified that RFT and FDR thresholds converge as the true signal weakens and diverge when the signal is broad in terms of the proportion of the continuum size it occupies. Results also showed that, while sample size affected the relation between RFT and FDR results for small sample sizes (<15), this relation was stable for larger sample sizes, wherein only the nature of the true signal was important.

Discussion

RFT and FDR thresholds are both computationally efficient because both are parametric, but only FDR has the ability to adapt to the signal features of particular datasets, wherein the threshold lowers with signal strength for a gain in sensitivity. Additional advantages and limitations of these two techniques as discussed further. This article is accompanied by freely available software for implementing FDR analyses involving 1-D data and scripts to replicate our results.