A Study of Multivariate Permutation Tests Which May Replace Hotelling's T2 Test in Prescribed Circumstances

Resolution of impaired multisensory processing in autism and the cost of switching sensory modality

Children with autism spectrum disorders (ASD) exhibit alterations in multisensory processing, which may contribute to the prevalence of social and communicative deficits in this population. Resolution of multisensory deficits has been observed in teenagers with ASD for complex, social speech stimuli; however, whether this resolution extends to more basic multisensory processing deficits remains unclear. Here, in a cohort of 364 participants we show using simple, non-social audiovisual stimuli that deficits in multisensory processing observed in high-functioning children and teenagers with ASD are not evident in adults with the disorder. Computational modelling indicated that multisensory processing transitions from a default state of competition to one of facilitation, and that this transition is delayed in ASD. Further analysis revealed group differences in how sensory channels are weighted, and how this is impacted by preceding cross-sensory inputs. Our findings indicate that there is a complex and dynamic interplay among the sensory systems that differs considerably in individuals with ASD.

Crosse et al. study a cohort of 364 participants with autism spectrum disorders (ASD) and matched controls, and show that deficits in multisensory processing observed in high-functioning children and teenagers with ASD are not evident in adults with the disorder. Using computational modelling they go on to demonstrate that there is a delayed transition of multisensory processing from a default state of competition to one of facilitation in ASD, as well as differences in sensory weighting and the ability to switch between sensory modalities, which sheds light on the interplay among sensory systems that differ in ASD individuals.

Human visual gamma for color stimuli

Strong gamma-band oscillations in primate early visual cortex can be induced by homogeneous color surfaces (Peter et al., 2019; Shirhatti and Ray, 2018). Compared to other hues, particularly strong gamma oscillations have been reported for red stimuli. However, precortical color processing and the resultant strength of input to V1 have often not been fully controlled for. Therefore, stronger responses to red might be due to differences in V1 input strength. We presented stimuli that had equal luminance and cone contrast levels in a color coordinate system based on responses of the lateral geniculate nucleus, the main input source for area V1. With these stimuli, we recorded magnetoencephalography in 30 human participants. We found gamma oscillations in early visual cortex which, contrary to previous reports, did not differ between red and green stimuli of equal L-M cone contrast. Notably, blue stimuli with contrast exclusively on the S-cone axis induced very weak gamma responses, as well as smaller event-related fields and poorer change-detection performance. The strength of human color gamma responses for stimuli on the L-M axis could be well explained by L-M cone contrast and did not show a clear red bias when L-M cone contrast was properly equalized.

Stimulus-specific plasticity in human visual gamma-band activity and functional connectivity

Under natural conditions, the visual system often sees a given input repeatedly. This provides an opportunity to optimize processing of the repeated stimuli. Stimulus repetition has been shown to strongly modulate neuronal-gamma band synchronization, yet crucial questions remained open. Here we used magnetoencephalography in 30 human subjects and find that gamma decreases across ≈10 repetitions and then increases across further repetitions, revealing plastic changes of the activated neuronal circuits. Crucially, increases induced by one stimulus did not affect responses to other stimuli, demonstrating stimulus specificity. Changes partially persisted when the inducing stimulus was repeated after 25 minutes of intervening stimuli. They were strongest in early visual cortex and increased interareal feedforward influences. Our results suggest that early visual cortex gamma synchronization enables adaptive neuronal processing of recurring stimuli. These and previously reported changes might be due to an interaction of oscillatory dynamics with established synaptic plasticity mechanisms.

Crucial role for CA2 inputs in the sequential organization of CA1 time cells supporting memory

There is considerable evidence for hippocampal time cells that briefly activate in succession to represent the temporal structure of memories. Previous studies have shown that time cells can be disrupted while leaving place cells intact, indicating that spatial and temporal information can be coded in parallel. However, the circuits in which spatial and temporal information are coded have not been clearly identified. Here we investigated temporal and spatial coding by dorsal hippocampal CA1 (dCA1) neurons in mice trained on a classic spatial working-memory task. On each trial, the mice approached the same choice point on a maze but were trained to alternate between traversing one of two distinct spatial routes (spatial coding phase). In between trials, there was a 10-s mnemonic delay during which the mouse continuously ran in a fixed location (temporal coding phase). Using cell-type-specific optogenetic methods, we found that inhibiting dorsal CA2 (dCA2) inputs into dCA1 degraded time cell coding during the mnemonic delay and impaired the mouse's subsequent memory-guided choice. Conversely, inhibiting dCA2 inputs during the spatial coding phase had a negligible effect on place cell activity in dCA1 and no effect on behavior. Collectively, our work demonstrates that spatial and temporal coding in dCA1 is largely segregated with respect to the dCA2-dCA1 circuit and suggests that CA2 plays a critical role in representing the flow of time in memory within the hippocampal network.

Operating in a Multisensory Context: Assessing the Interplay Between Multisensory Reaction Time Facilitation and Inter-sensory Task-switching Effects

Individuals respond faster to presentations of bisensory stimuli (e.g. audio-visual targets) than to presentations of either unisensory constituent in isolation (i.e. to the auditory-alone or visual-alone components of an audio-visual stimulus). This well-established multisensory speeding effect, termed the redundant signals effect (RSE), is not predicted by simple linear summation of the unisensory response time probability distributions. Rather, the speeding is typically faster than this prediction, leading researchers to ascribe the RSE to a so-called co-activation account. According to this account, multisensory neural processing occurs whereby the unisensory inputs are integrated to produce more effective sensory-motor activation. However, the typical paradigm used to test for RSE involves random sequencing of unisensory and bisensory inputs in a mixed design, raising the possibility of an alternate attention-switching account. This intermixed design requires participants to switch between sensory modalities on many task trials (e.g. from responding to a visual stimulus to an auditory stimulus). Here we show that much, if not all, of the RSE under this paradigm can be attributed to slowing of reaction times to unisensory stimuli resulting from modality switching, and is not in fact due to speeding of responses to AV stimuli. As such, the present data do not support a co-activation account, but rather suggest that switching and mixing costs akin to those observed during classic task-switching paradigms account for the observed RSE.

Non-parametric combination and related permutation tests for neuroimaging

In this work, we show how permutation methods can be applied to combination analyses such as those that include multiple imaging modalities, multiple data acquisitions of the same modality, or simply multiple hypotheses on the same data. Using the well-known definition of union-intersection tests and closed testing procedures, we use synchronized permutations to correct for such multiplicity of tests, allowing flexibility to integrate imaging data with different spatial resolutions, surface and/or volume-based representations of the brain, including non-imaging data. For the problem of joint inference, we propose and evaluate a modification of the recently introduced non-parametric combination (NPC) methodology, such that instead of a two-phase algorithm and large data storage requirements, the inference can be performed in a single phase, with reasonable computational demands. The method compares favorably to classical multivariate tests (such as MANCOVA), even when the latter is assessed using permutations. We also evaluate, in the context of permutation tests, various combining methods that have been proposed in the past decades, and identify those that provide the best control over error rate and power across a range of situations. We show that one of these, the method of Tippett, provides a link between correction for the multiplicity of tests and their combination. Finally, we discuss how the correction can solve certain problems of multiple comparisons in one-way ANOVA designs, and how the combination is distinguished from conjunctions, even though both can be assessed using permutation tests. We also provide a common algorithm that accommodates combination and correction.

Permutation Tests in the Educational and Behavioral Sciences

Over the past two decades, permutation tests (PTs) have received much attention in the educational and behavioral sciences. The aim of this article is to review the theoretical developments of PTs, the active areas in the educational and behavioral research using PTs, and the types of analysis under which PTs have been applied. We obtained 224 published articles, which included 141 theoretical articles and 83 application articles. After scrutinizing each article, we are happy to see that (1) some researchers began to advocate introducing PTs into basic statistics training; (2) computing load for PTs may be reduced dramatically by some intelligent algorithms; (3) PTs began to be applied in new areas such as studies on the relationship between brain and behavior and the relationship between gene and behavior; (4) besides simple types of analysis such as independent two-group comparison, PTs can also be carried out under more complex situations such as multivariate analysis. However, we should also notice that PTs are still mostly used for simple analyses (e.g., randomness analysis).

Measured and perceived environmental comfort: field monitoring in an Italian school

Microclimatic conditions were recorded in an Italian school and Fanger's indexes PMV and PPD were calculated under different conditions. Students' sensations were investigated four times by means of two surveys, one related to actual microclimatic conditions and one on overall satisfaction, interaction occupant-building and reactions to discomfort. Pupils' classroom position was considered to look for possible influence on thermal comfort: a difference emerged from PMV and the survey, but the results obtained from the two approaches differ for both the entity of discomfort and its distribution within each classroom. Innovative multivariate nonparametric statistical techniques were applied to compare and rank the classrooms in accordance with students' subjective perceptions; a global ranking has been also calculated, considering thermal and visual comfort and air quality. Comparing pupil-sensation-based ranking with environmental parameters no clear correspondence was found, except for mid-season, where PMV, CO2 concentration and desk illuminance were similar in all the classrooms.

Permutation inference for the general linear model

Permutation methods can provide exact control of false positives and allow the use of non-standard statistics, making only weak assumptions about the data. With the availability of fast and inexpensive computing, their main limitation would be some lack of flexibility to work with arbitrary experimental designs. In this paper we report on results on approximate permutation methods that are more flexible with respect to the experimental design and nuisance variables, and conduct detailed simulations to identify the best method for settings that are typical for imaging research scenarios. We present a generic framework for permutation inference for complex general linear models (GLMS) when the errors are exchangeable and/or have a symmetric distribution, and show that, even in the presence of nuisance effects, these permutation inferences are powerful while providing excellent control of false positives in a wide range of common and relevant imaging research scenarios. We also demonstrate how the inference on GLM parameters, originally intended for independent data, can be used in certain special but useful cases in which independence is violated. Detailed examples of common neuroimaging applications are provided, as well as a complete algorithm - the "randomise" algorithm - for permutation inference with the GLM.

Iterated combination-based paired permutation tests to determine shape effects of chemotherapy in patients with esophageal cancer

The nonparametric combination of dependent permutation tests method is a useful general tool when a testing problem can be broken down into a set of different k > 1 partial tests. These partial tests, after adjustment of p-values to control for multiplicity, can be marginally analyzed, but jointly considered they can provide information on an overall hypothesis, which might represent the true goal of the testing problem. On the one hand, independence among the partial tests is usually an unrealistic assumption; on the other, even when the underlying dependence relations are known quite often they are difficult to cope with properly. Therefore this combination must be achieved nonparametrically, by implicitly taking into account the dependence structure of tests without explicitly describing it. An important property of the tests based on nonparametric combination methodology, when the number of response variables is high compared to the sample sizes, consists in the finite sample consistency. A practical problem involves choosing the most suitable combining function for each specific testing problem given that the final result can be affected by this crucial choice. The purpose of this article is to present an nonparametric combination solution based on the iterated combination of partial tests, evaluate its power behavior using a Monte Carlo simulation study and apply it to a real medical problem, namely the evaluation of the effects of chemotherapy on the shape of esophageal tumors. R code has been implemented to carry out the analyses.

A permutation test for the race model inequality

When participants are asked to respond in the same way to several stimulus identities, responses are often observed to be faster if two stimuli are presented simultaneously as opposed to when a single stimulus is presented (redundant signals effect; Miller, 1982). An important issue of such experiments is whether the observed redundancy gains can be explained by parallel processing of the two stimuli in a race-like fashion. To test the parallel processing model, Miller derived the well-known race model inequality which has become a routine test for behavioral data in experiments with redundant signals. Several statistical procedures have been used for testing the race model inequality. However, the commonly employed procedure does not control the Type I error. In this article a permutation test is described that keeps the Type I error at the desired level. Simulations show that the power of the test is reasonable even for small samples. The scripts discussed in this article may be downloaded as supplemental materials from http://brm.psychonomic-journals.org/content/supplemental.

Detecting change in biological rhythms: a multivariate permutation test approach to Fourier-transformed data

Treatment-related changes in neurobiological rhythms are of increasing interest to psychologists, psychiatrists, and biological rhythms researchers. New methods for analyzing change in rhythms are needed, as most common methods disregard the rich complexity of biological processes. Large time series data sets reflect the intricacies of underlying neurobiological processes, but can be difficult to analyze. We propose the use of Fourier methods with multivariate permutation test (MPT) methods for analyzing change in rhythms from time series data. To validate the use of MPT for Fourier-transformed data, we performed Monte Carlo simulations and compared statistical power and family-wise error for MPT to Bonferroni-corrected and uncorrected methods. Results show that MPT provides greater statistical power than Bonferroni-corrected tests, while appropriately controlling family-wise error. We applied this method to human, pre- and post-treatment, serially-sampled neurotransmitter data to confirm the utility of this method using real data. Together, Fourier with MPT methods provides a statistically powerful approach for detecting change in biological rhythms from time series data.

Performance of Monte Carlo Permutation and Approximate Tests for Multivariate Means Comparisons With Small Sample Sizes When Parametric Assumptions are Violated

Permutation testing has been suggested as an alternative to the standard F approximate tests used in multivariate analysis of variance (MANOVA). These approximate tests, such as Wilks’ Lambda and Pillai’s Trace, have been shown to perform poorly when assumptions of normally distributed dependent variables and homogeneity of group covariance matrices were violated. Because Monte Carlo permutation tests do not rely on distributional assumptions, they may be expected to work better than their approximate cousins when the data do not conform to the assumptions described above. The current simulation study compared the performance of four standard MANOVA test statistics with their Monte Carlo permutation-based counterparts under a variety of conditions with small samples, including conditions when the assumptions were met and when they were not. Results suggest that for sample sizes of 50 subjects, power is very low for all the statistics. In addition, Type I error rates for both the approximate F and Monte Carlo tests were inflated under the condition of nonnormal data and unequal covariance matrices. In general, the performance of the Monte Carlo permutation tests was slightly better in terms of Type I error rates and power when both assumptions of normality and homogeneous covariance matrices were not met. It should be noted that these simulations were based upon the case with three groups only, and as such results presented in this study can only be generalized to similar situations.

Multivariate permutation analysis associates multiple polymorphisms with subphenotypes of major depression

Unipolar major depressive disorder (MDD) is a prevalent, disabling condition with multiple genetic and environmental factors impacting disease risk. The diagnosis of MDD relies on a cumulative measure derived from multiple trait dimensions and alone is limited in elucidating MDD genetic determinants. We and others have proposed that MDD may be better dissected using paradigms that assess how specific genes associate with component features of MDD. This within-disease design requires both a well-phenotyped cohort and a robust statistical approach that retains power with multiple tests of genetic association. In the present study, common polymorphic variants of genes related to central monoaminergic and cholinergic pathways that previous studies align with functional change in vitro or depression associations in vivo were genotyped in 110 individuals with unipolar MDD. Subphenotypic characteristics were examined using responses to individual items assessed with the Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders (DSM IV), the 17-item Hamilton Rating Scale for Depression (HAM-D) and the NEO Five Factor Inventory. Multivariate Permutation Testing (MPT) was used to infer genotype-phenotype relationships underlying dimensional findings within clinical categories. MPT analyses show significant associations of the norepinephrine transporter (NET, SLC6A2) -182 T/C (rs2242446) with recurrent depression [odds ratio, OR = 4.15 (1.91-9.02)], NET -3081 A/T (rs28386840) with increase in appetite [OR = 3.58 (1.53-8.39)] and the presynaptic choline transporter (CHT, SLC5A7) Ile89Val (rs1013940) with HAM-D-17 total score {i.e. overall depression severity [OR = 2.74 (1.05-7.18)]}. These relationships illustrate an approach to the elucidation of gene influences on trait components of MDD and with replication, may help identify MDD subpopulations that can benefit from more targeted pharmacotherapy.

Characteristic functional networks in high- versus low-proficiency second language speakers detected also during native language processing: an explorative EEG coherence study in 6 frequency bands

An EEG coherence study was performed with a twofold objective: first, to scrutinize the theoretical concept of "cortical efficiency" in connection with second language (L2) acquisition and, second, to detect cooperations between cortical areas in specific frequency bands indicative for highly proficient L2 processing. Two groups differing only in their level of L2 proficiency were contrasted during presentation of natural language videos in English (L2) and German (native language, L1), with explorative coherence analysis in 6 frequency bands (0.5-31.5 Hz). The coherence brain maps revealed more pronounced and widespread increases in coherences in the alpha1-band (8-10 Hz) in low-proficiency than in the high-proficiency L2 speakers. Surprisingly, this difference was obtained also during L1 processing and corroborated for both languages by multivariate permutation tests. These tests revealed additional differences between the low- and the high-proficiency group also for coherences within the beta1- (13-18 Hz) and the beta2-band (18.5-31.5 Hz), again during L2 and L1 processing. Since the same group differences were observed during L1 and L2 processing, our high-proficiency group might have profited from a more generic advantage in language or text processing strategy. This strategic advantage was most evident at alpha1 frequencies, possibly related to a specific way of processing internal mental states (top-down processing).

Decreased EEG coherence between prefrontal electrodes: a correlate of high language proficiency?

To investigate the influence of proficiency level on the cortical organization of foreign language processing, two groups of German speaking students, differing only in their proficiency in English as a second language, were subjected to EEG coherence analysis during foreign and native language processing (news reports, alpha1 frequency band). In the group with minor experience with English, coherence increase was observed with all electrode combinations, with left hemisphere (LH) predominance. In the high proficiency group, coherence increase was limited to temporal electrodes over LH. In the latter group only, coherence between prefrontal electrodes was significantly lower during the language tasks than during the baseline task (silence, noisy screen). Both results were obtained with foreign as well as native language processing. We suggest that reduced EEG coherence in highly proficient foreign language speakers reflects a more efficient operating strategy not only for their second, but also for their native language.

Hypothesis testing with nonlinear shape models

We present a method for two-sample hypothesis testing for statistical shape analysis using nonlinear shape models. Our approach uses a true multivariate permutation test that is invariant to the scale of different model parameters and that explicitly accounts for the dependencies between variables. We apply our method to m-rep models of the lateral ventricles to examine the amount of shape variability in twins with different degrees of genetic similarity.

Multivariate tests for the evaluation of high-dimensional EEG data

In this paper several multivariate tests are presented, in particular permutation tests, which can be used in multiple endpoint problems as for example in comparisons of high-dimensional vectors of EEG data. We have investigated the power of these tests using artificial data in simulations and real EEG data. It is obvious that no one multivariate test is uniformly most powerful. The power of the different methods depends in different ways on the correlation between the endpoints, on the number of endpoints for which differences exist and on other factors. Based on our findings, we have derived rules of thumb regarding under which configurations a particular test should be used. In order to demonstrate the properties of different multivariate tests we applied them to EEG coherence data. As an example for the paired samples case, we compared the 171-dimensional coherence vectors observed for the alpha1-band while processing either concrete or abstract nouns and obtained significant global differences for some sections of time. As an example for the unpaired samples case, we compared the coherence vectors observed for language students and non-language students who processed an English text and found a significant global difference.