Penalized discriminant analysis of [15O]-water PET brain images with prediction error selection of smoothness and regularization hyperparameters

Pattern classification of fMRI data: applications for analysis of spatially distributed cortical networks

The field of fMRI data analysis is rapidly growing in sophistication, particularly in the domain of multivariate pattern classification. However, the interaction between the properties of the analytical model and the parameters of the BOLD signal (e.g. signal magnitude, temporal variance and functional connectivity) is still an open problem. We addressed this problem by evaluating a set of pattern classification algorithms on simulated and experimental block-design fMRI data. The set of classifiers consisted of linear and quadratic discriminants, linear support vector machine, and linear and nonlinear Gaussian naive Bayes classifiers. For linear discriminant, we used two methods of regularization: principal component analysis, and ridge regularization. The classifiers were used (1) to classify the volumes according to the behavioral task that was performed by the subject, and (2) to construct spatial maps that indicated the relative contribution of each voxel to classification. Our evaluation metrics were: (1) accuracy of out-of-sample classification and (2) reproducibility of spatial maps. In simulated data sets, we performed an additional evaluation of spatial maps with ROC analysis. We varied the magnitude, temporal variance and connectivity of simulated fMRI signal and identified the optimal classifier for each simulated environment. Overall, the best performers were linear and quadratic discriminants (operating on principal components of the data matrix) and, in some rare situations, a nonlinear Gaussian naïve Bayes classifier. The results from the simulated data were supported by within-subject analysis of experimental fMRI data, collected in a study of aging. This is the first study that systematically characterizes interactions between analysis model and signal parameters (such as magnitude, variance and correlation) on the performance of pattern classifiers for fMRI.

Nonlinear denoising and analysis of neuroimages with kernel principal component analysis and pre-image estimation

We investigate the use of kernel principal component analysis (PCA) and the inverse problem known as pre-image estimation in neuroimaging: i) We explore kernel PCA and pre-image estimation as a means for image denoising as part of the image preprocessing pipeline. Evaluation of the denoising procedure is performed within a data-driven split-half evaluation framework. ii) We introduce manifold navigation for exploration of a nonlinear data manifold, and illustrate how pre-image estimation can be used to generate brain maps in the continuum between experimentally defined brain states/classes. We base these illustrations on two fMRI BOLD data sets - one from a simple finger tapping experiment and the other from an experiment on object recognition in the ventral temporal lobe.

ODVBA: optimally-discriminative voxel-based analysis

Gaussian smoothing of images prior to applying voxel-based statistics is an important step in voxel-based analysis and statistical parametric mapping (VBA-SPM) and is used to account for registration errors, to Gaussianize the data and to integrate imaging signals from a region around each voxel. However, it has also become a limitation of VBA-SPM based methods, since it is often chosen empirically and lacks spatial adaptivity to the shape and spatial extent of the region of interest, such as a region of atrophy or functional activity. In this paper, we propose a new framework, named optimally-discriminative voxel-based analysis (ODVBA), for determining the optimal spatially adaptive smoothing of images, followed by applying voxel-based group analysis. In ODVBA, nonnegative discriminative projection is applied regionally to get the direction that best discriminates between two groups, e.g., patients and controls; this direction is equivalent to local filtering by an optimal kernel whose coefficients define the optimally discriminative direction. By considering all the neighborhoods that contain a given voxel, we then compose this information to produce the statistic for each voxel. Finally, permutation tests are used to obtain a statistical parametric map of group differences. ODVBA has been evaluated using simulated data in which the ground truth is known and with data from an Alzheimer's disease (AD) study. The experimental results have shown that the proposed ODVBA can precisely describe the shape and location of structural abnormality.

Data-driven optimization and evaluation of 2D EPI and 3D PRESTO for BOLD fMRI at 7 Tesla: I. Focal coverage

Blood oxygenation level dependent (BOLD) functional magnetic resonance imaging (fMRI) is commonly performed using 2D single-shot echo-planar imaging (EPI). However, single-shot EPI at 7 Tesla (T) often suffers from significant geometric distortions (due to low bandwidth (BW) in the phase-encode (PE) direction) and amplified physiological noise. Recent studies have suggested that 3D multi-shot sequences such as PRESTO may offer comparable BOLD contrast-to-noise ratio with increased volume coverage and decreased geometric distortions. Thus, a four-way group-level comparison was performed between 2D and 3D acquisition sequences at two in-plane resolutions. The quality of fMRI data was evaluated via metrics of prediction and reproducibility using NPAIRS (Non-parametric Prediction, Activation, Influence and Reproducibility re-Sampling). Group activation maps were optimized for each acquisition strategy by selecting the number of principal components that jointly maximized prediction and reproducibility, and showed good agreement in sensitivity and specificity for positive BOLD changes. High-resolution EPI exhibited the highest z-scores of the four acquisition sequences; however, it suffered from the lowest BW in the PE direction (resulting in the worst geometric distortions) and limited spatial coverage, and also caused some subject discomfort through peripheral nerve stimulation (PNS). In comparison, PRESTO also had high z-scores (higher than EPI for a matched in-plane resolution), the highest BW in the PE direction (producing images with superior geometric fidelity), the potential for whole-brain coverage, and no reported PNS. This study provides evidence to support the use of 3D multi-shot acquisition sequences in lieu of single-shot EPI for ultra high field BOLD fMRI at 7T.

Comparing Classification Methods for Longitudinal fMRI Studies

We compare 10 methods of classifying fMRI volumes by applying them to data from a longitudinal study of stroke recovery: adaptive Fisher's linear and quadratic discriminant; gaussian naive Bayes; support vector machines with linear, quadratic, and radial basis function (RBF) kernels; logistic regression; two novel methods based on pairs of restricted Boltzmann machines (RBM); and K-nearest neighbors. All methods were tested on three binary classification tasks, and their out-of-sample classification accuracies are compared. The relative performance of the methods varies considerably across subjects and classification tasks. The best overall performers were adaptive quadratic discriminant, support vector machines with RBF kernels, and generatively trained pairs of RBMs.

Dimensionality estimation for optimal detection of functional networks in BOLD fMRI data

Estimation of the intrinsic dimensionality of fMRI data is an important part of data analysis that helps to separate the signal of interest from noise. We have studied multiple methods of dimensionality estimation proposed in the literature and used these estimates to select a subset of principal components that was subsequently processed by linear discriminant analysis (LDA). Using simulated multivariate Gaussian data, we show that the dimensionality that optimizes signal detection (in terms of the receiver operating characteristic (ROC) metric) goes through a transition from many dimensions to a single dimension as a function of the signal-to-noise ratio. This transition happens when the loci of activation are organized into a spatial network and the variance of the networked, task-related signals is high enough for the signal to be easily detected in the data. We show that reproducibility of activation maps is a metric that captures this switch in intrinsic dimensionality. Except for reproducibility, all of the methods of dimensionality estimation we considered failed to capture this transition: optimization of Bayesian evidence, minimum description length, supervised and unsupervised LDA prediction, and Stein's unbiased risk estimator. This failure results in sub-optimal ROC performance of LDA in the presence of a spatially distributed network, and may have caused LDA to underperform in many of the reported comparisons in the literature. Using real fMRI data sets, including multi-subject group and within-subject longitudinal analysis we demonstrate the existence of these dimensionality transitions in real data.

Robustness of multivariate image analysis assessed by resampling techniques and applied to FDG-PET scans of patients with Alzheimer's disease

For finite and noisy samples extraction of robust features or patterns which are representative of the population is a formidable task in which over-interpretation is not uncommon. In this work, resampling techniques have been applied to a sample of 42 FDG PET brain images of 19 healthy volunteers (HVs) and 23 Alzheimer's disease (AD) patients to assess the robustness of image features extracted through principal component analysis (PCA) and Fisher discriminant analysis (FDA). The objective of this work is to: 1) determine the relative variance described by the PCA to the population variance; 2) assess the robustness of the PCA to the population sample using the largest principal angle between PCA subspaces; 3) assess the robustness and accuracy of the FDA. Since the sample does not have histopathological data the impact of possible clinical misdiagnosis on the discrimination analysis is investigated. The PCA can describe up to 40% of the total population variability. Not more than the first three or four PCs can be regarded as robust on which a robust FDA can be build. Standard error images showed that regions close to the falx and around ventricles are less stable. Using the first three PCs, sensitivity and specificity were 90.5% and 96.9% respectively. The use of resampling techniques in the evaluation of the robustness of many multivariate image analysis methods enables researchers to avoid over-analysis when using these methods applied to many different neuroimaging studies often with small sample sizes.

Linking functional and structural brain images with multivariate network analyses: a novel application of the partial least square method

In this article, we introduce a multimodal multivariate network analysis to characterize the linkage between the patterns of information from the same individual's complementary brain images, and illustrate its potential by showing its ability to distinguish older from younger adults with greater power than several previously established methods. Our proposed method uses measurements from every brain voxel in each person's complementary co-registered images and uses the partial least square (PLS) algorithm to form a combined latent variable that maximizes the covariance among all of the combined variables. It represents a new way to calculate the singular value decomposition from the high-dimensional covariance matrix in a computationally feasible way. Analyzing fluorodeoxyglucose positron emission tomography (PET) and volumetric magnetic resonance imaging (MRI) images, this method distinguished 14 older adults from 15 younger adults (p=4e-12) with no overlap between groups, no need to correct for multiple comparisons, and greater power than the univariate Statistical Parametric Mapping (SPM), multimodal SPM or multivariate PLS analysis of either imaging modality alone. This technique has the potential to link patterns of information among any number of complementary images from an individual, to use other kinds of complementary complex datasets besides brain images, and to characterize individual state- or trait-dependent brain patterns in a more powerful way.

Evaluation and optimization of fMRI single-subject processing pipelines with NPAIRS and second-level CVA

In functional magnetic resonance imaging (fMRI) analysis, although the univariate general linear model (GLM) is currently the dominant approach to brain activation detection, there is growing interest in multivariate approaches such as principal component analysis, canonical variate analysis (CVA), independent component analysis and cluster analysis, which have the potential to reveal neural networks and functional connectivity in the brain. To understand the effect of processing options on performance of multivariate model-based fMRI processing pipelines with real fMRI data, we investigated the impact of commonly used fMRI preprocessing steps and optimized the associated multivariate CVA-based, single-subject processing pipelines with the NPAIRS (nonparametric prediction, activation, influence and reproducibility resampling) performance metrics [prediction accuracy and statistical parametric image (SPI) reproducibility] on the Fiswidgets platform. We also compared the single-subject SPIs of univariate GLM with multivariate CVA-based processing pipelines from SPM, FSL.FEAT, NPAIRS.GLM and NPAIRS.CVA software packages (or modules) using a novel second-level CVA. We found that for the block-design data, (a) slice timing correction and global intensity normalization have little consistent impact on the fMRI processing pipeline, but spatial smoothing, temporal detrending or high-pass filtering, and motion correction significantly improved pipeline performance across all subjects; (b) the combined optimization of spatial smoothing, temporal detrending and CVA model parameters on average improved between-subject reproducibility; and (c) the most important pipeline choices include univariate or multivariate statistical models and spatial smoothing. This study suggests that considering options other than simply using GLM with a fixed spatial filter may be of critical importance in determining activation patterns in BOLD fMRI studies.

Evaluation and comparison of GLM- and CVA-based fMRI processing pipelines with Java-based fMRI processing pipeline evaluation system

Activation patterns identified by fMRI processing pipelines or fMRI software packages are usually determined by the preprocessing options, parameters, and statistical models used. Previous studies that evaluated options of GLM (general linear model)--based fMRI processing pipelines are mainly based on simulated data with receiver operating characteristics (ROC) analysis, but evaluation of such fMRI processing pipelines on real fMRI data is rare. To understand the effect of processing options on performance of GLM-based fMRI processing pipelines with real fMRI data, we investigated the impact of commonly-used fMRI preprocessing steps; optimized the associated GLM-based single-subject processing pipelines; and quantitatively compared univariate GLM (in FSL.FEAT and NPAIRS.GLM) and multivariate CVA (canonical variates analysis) (in NPAIRS.CVA)-based analytic models in single-subject analysis with a recently developed fMRI processing pipeline evaluation system based on prediction accuracy (classification accuracy) and reproducibility performance metrics. For block-design data, we found that with GLM analysis (1) slice timing correction and global intensity normalization have little consistent impact on fMRI processing pipelines, spatial smoothing and high-pass filtering or temporal detrending significantly increases pipeline performance and thus are essential for robust fMRI statistical analysis; (2) combined optimization of spatial smoothing and temporal detrending improves pipeline performance; and (3) in general, the prediction performance of multivariate CVA is higher than that of the univariate GLM, while univariate GLM is more reproducible than multivariate CVA. Because of the different bias-variance trade-offs of univariate and multivariate models, it may be necessary to consider a consensus approach to obtain more accurate activation patterns in fMRI data.

Real-time fMRI using brain-state classification

We have implemented a real-time functional magnetic resonance imaging system based on multivariate classification. This approach is distinctly different from spatially localized real-time implementations, since it does not require prior assumptions about functional localization and individual performance strategies, and has the ability to provide feedback based on intuitive translations of brain state rather than localized fluctuations. Thus this approach provides the capability for a new class of experimental designs in which real-time feedback control of the stimulus is possible-rather than using a fixed paradigm, experiments can adaptively evolve as subjects receive brain-state feedback. In this report, we describe our implementation and characterize its performance capabilities. We observed approximately 80% classification accuracy using whole brain, block-design, motor data. Within both left and right motor task conditions, important differences exist between the initial transient period produced by task switching (changing between rapid left or right index finger button presses) and the subsequent stable period during sustained activity. Further analysis revealed that very high accuracy is achievable during stable task periods, and that the responsiveness of the classifier to changes in task condition can be much faster than signal time-to-peak rates. Finally, we demonstrate the versatility of this implementation with respect to behavioral task, suggesting that our results are applicable across a spectrum of cognitive domains. Beyond basic research, this technology can complement electroencephalography-based brain computer interface research, and has potential applications in the areas of biofeedback rehabilitation, lie detection, learning studies, virtual reality-based training, and enhanced conscious awareness.

Contrasting single-trial ERPs between experimental manipulations: Improving differentiability by blind source separation

Contrasting event-related potentials (ERPs) generated under different experimental conditions and inferring differential brain responses is widely practiced in cognitive neuroscience research. Traditionally, these contrasts and subsequent inferences have proceeded directly from ERPs measured at the scalp. For certain tasks, it is not unusual that ERPs from a subset of channels are given particular emphasis in data analysis, such as the channels displaying the maximum peak amplitude in regions of interest ("best sensors") or channels showing the largest averaged ERP waveform differences. With the aid of a blind source separation (BSS) algorithm, second-order blind identification (SOBI), which has been recently validated for its ability to recover correlated neuronal sources, we show that single-trial ERPs from previously validated neuronal sources were more distinguishable among different experimental manipulations than the single-trial ERPs measured at the comparable "best sensors". This suggests that by using validated SOBI-recovered neuronal sources, ERP researchers can improve the ability to detect differences in neuronal responses induced by experimental manipulations. Critically, our observations were made at the level of single trials, as opposed to the averaged ERP. Therefore, our conclusions are particularly relevant to phenomena involving trial-to-trial changes in brain activation, for example, rapid induction of brain plasticity and perceptual learning, as well as to the development of brain-computer interfaces. Similar advantages would also apply to analogous situations with magnetoencephalography (MEG).

Functional connectivity during Stroop task performance

Using covariance-based multivariate analysis, we examined patterns of functional connectivity in rCBF on a practice-extended version of the Stroop color-word paradigm. Color-word congruent and incongruent conditions were presented in six AB trials to healthy subjects during 12 H2(15)O PET scans. Analyses identified two reproducible canonical eigenimages (CE) from the PET data, which were converted to a standard Z score scale after cross-validation resampling and correction for random subject effects. The first CE corresponded to practice-dependent changes in covarying rCBF that occurred over early task repetitions and correlated with improved behavioral performance. This included many regions previously implicated by PET and fMRI studies of this task, which we suggest may represent two "parallel" networks: (i) a cingulo-frontal system that was initially engaged in selecting and mapping a task-relevant response (color naming) when the attentional demands of the task were greatest; and (ii) a ventral visual processing stream whose concurrent decrease in activity represented the task-irrelevant inhibition of word reading. The second CE corresponded to a consistent paradigmatic effect of Stroop interference on covarying rCBF. Coactivations were located in dorsal and ventral prefrontal regions as well as frontopolar cortex. This pattern supports existing evidence that prefrontal regions are involved in maintaining attentional control over conflicting response systems. Taken together, these findings may be more in line with theoretical models that emphasize a role for practice in the emergence of Stroop phenomena. These findings may also provide some additional insight into the nature of anterior cingulate- and prefrontal cortical contributions to implementing cognitive control in the brain.

Optimizing the fMRI data-processing pipeline using prediction and reproducibility performance metrics: I. A preliminary group analysis

We argue that published results demonstrate that new insights into human brain function may be obscured by poor and/or limited choices in the data-processing pipeline, and review the work on performance metrics for optimizing pipelines: prediction, reproducibility, and related empirical Receiver Operating Characteristic (ROC) curve metrics. Using the NPAIRS split-half resampling framework for estimating prediction/reproducibility metrics (Strother et al., 2002), we illustrate its use by testing the relative importance of selected pipeline components (interpolation, in-plane spatial smoothing, temporal detrending, and between-subject alignment) in a group analysis of BOLD-fMRI scans from 16 subjects performing a block-design, parametric-static-force task. Large-scale brain networks were detected using a multivariate linear discriminant analysis (canonical variates analysis, CVA) that was tuned to fit the data. We found that tuning the CVA model and spatial smoothing were the most important processing parameters. Temporal detrending was essential to remove low-frequency, reproducing time trends; the number of cosine basis functions for detrending was optimized by assuming that separate epochs of baseline scans have constant, equal means, and this assumption was assessed with prediction metrics. Higher-order polynomial warps compared to affine alignment had only a minor impact on the performance metrics. We found that both prediction and reproducibility metrics were required for optimizing the pipeline and give somewhat different results. Moreover, the parameter settings of components in the pipeline interact so that the current practice of reporting the optimization of components tested in relative isolation is unlikely to lead to fully optimized processing pipelines.

Spatio-temporal autoregressive models defined over brain manifolds

Multivariate Autoregressive time series models (MAR) are an increasingly used tool for exploring functional connectivity in Neuroimaging. They provide the framework for analyzing the Granger Causality of a given brain region on others. In this article, we shall limit our attention to linear MAR models, in which a set of matrices of autoregressive coefficients Ak (k = 1,...,p) describe the dependence of present values of the image on lagged values of its past. Methods for estimating the Ak and determining which elements that are zero are well-known and are the basis for directed measures of influence. However, to date, MAR models are limited in the number of time series they can handle, forcing the a priori selection of a (small) number of voxels or regions of interest for analysis. This ignores the full spatio-temporal nature of functional brain data which are, in fact, collections of time series sampled over an underlying continuous spatial manifold the brain. A fully spatio-temporal MAR model (ST-MAR) is developed within the framework of functional data analysis. For spatial data, each row of a matrix Ak is the influence field of a given voxel. A Bayesian ST-MAR model is specified in which the influence fields for all voxels are required to vary smoothly over space. This requirement is enforced by penalizing the spatial roughness of the influence fields. This roughness is calculated with a discrete version of the spatial Laplacian operator. A massive reduction in dimensionality of computations is achieved via the singular value decomposition, making an interactive exploration of the model feasible. Use of the model is illustrated with an fMRI time series that was gathered concurrently with EEG in order to analyze the origin of resting brain rhythms.

A post-processing/region of interest (ROI) method for discriminating patterns of activity in statistical maps of fMRI data

To combine functional neuroimaging studies across subjects, anatomical and functional data are typically either transformed to a common space or averaged across regions of interest (ROIs). However, if there are (1) anatomical variations within the subject pool (as in clinical or aging populations), (2) non-Gaussian distributions of task-related activity within a typical ROI or, (3) more ROIs than subjects, neither spatial transformation of the data to a common space nor averaging across all subjects' ROIs is suitable for standard discriminant analysis. To solve these problems, we describe a post-processing method that uses voxel-based statistics representing task-related activity (pooled within ROIs) to establish combinations of ROIs that maximally differentiate tasks across all subjects. The method involves randomized resampling from multiple ROIs within each subject, multivariate linear discriminant analysis across all subjects and validation with bootstrapping techniques. When applied to experimental data from healthy subjects performing two motor tasks, the method detected some brain regions, including the supplementary motor area (SMA), that participated in a distributed network differentially active between tasks. However there was not a significant difference in SMA activity when this region was examined in isolation. We suggest this method is a practical means to combine voxel-based statistics within anatomically defined ROIs across subjects.

Evaluating subject specific preprocessing choices in multisubject fMRI data sets using data-driven performance metrics

This study investigated the possible benefit of subject specific optimization of preprocessing strategies in functional magnetic resonance imaging (fMRI) experiments. The optimization was performed using the data-driven performance metrics developed recently [Neuroimage 15 (2002), 747]. We applied numerous preprocessing strategies and a multivariate statistical analysis to each of the 20 subjects in our two example fMRI data sets. We found that the optimal preprocessing strategy varied, in general, from subject to subject. For example, in one data set, optimum smoothing levels varied from 16 mm (4 subjects), 10 mm (5 subjects), to no smoothing at all (1 subject). This strongly suggests that group-specific preprocessing schemes may not give optimum results. For both studies, optimizing the preprocessing for each subject resulted in an increased number of suprathresholded voxels in within-subject analyses. Furthermore, we demonstrated that we were able to aggregate the optimized data with a random effects group analysis, resulting in improved sensitivity in one study and the detection of interesting, previously undetected results in the other.

Patterns of Activity in the Categorical Representations of Objects

Object perception has been a subject of extensive fMRI studies in recent years. Yet the nature of the cortical representation of objects in the human brain remains controversial. Analyses of fMRI data have traditionally focused on the activation of individual voxels associated with presentation of various stimuli. The current analysis approaches functional imaging data as collective information about the stimulus. Linking activity in the brain to a stimulus is treated as a pattern-classification problem. Linear discriminant analysis was used to reanalyze a set of data originally published by Ishai et al. (2000), available from the fMRIDC (accession no. 2-20001113D). Results of the new analysis reveal that patterns of activity that distinguish one category of objects from other categories are largely independent of one another, both in terms of the activity and spatial overlap. The information used to detect objects from phase-scrambled control stimuli is not essential in distinguishing one object category from another. Furthermore, performing an object-matching task during the scan significantly improved the ability to predict objects from controls, but had minimal effect on object classification, suggesting that the task-based attentional benefit was nonspecific to object categories.

The evaluation of preprocessing choices in single-subject BOLD fMRI using NPAIRS performance metrics

This work proposes an alternative to simulation-based receiver operating characteristic (ROC) analysis for assessment of fMRI data analysis methodologies. Specifically, we apply the rapidly developing nonparametric prediction, activation, influence, and reproducibility resampling (NPAIRS) framework to obtain cross-validation-based model performance estimates of prediction accuracy and global reproducibility for various degrees of model complexity. We rely on the concept of an analysis chain meta-model in which all parameters of the preprocessing steps along with the final statistical model are treated as estimated model parameters. Our ROC analog, then, consists of plotting prediction vs. reproducibility results as curves of model complexity for competing meta-models. Two theoretical underpinnings are crucial to utilizing this new validation technique. First, we explore the relationship between global signal-to-noise and our reproducibility estimates as derived previously. Second, we submit our model complexity curves in the prediction versus reproducibility space as reflecting classic bias-variance tradeoffs. Among the particular analysis chains considered, we found little impact in performance metrics with alignment, some benefit with temporal detrending, and greatest improvement with spatial smoothing.

An evaluation of methods for detecting brain activations from functional neuroimages

Brain activation studies based on PET or fMRI seek to explore neuroscience questions by statistically analyzing the acquired images to produce statistical parametric images (SPIs). An increasingly wide range of univariate and multivariate analysis techniques are used to generate SPIs in order to detect mean-signal activations and/or long-range spatial interactions. However, little is known about the comparative detection performance of even simple techniques in finite data sets. Our aims are (1) to empirically compare the detection performance of a range of techniques using simulations of a simple image phantom and receiver operating characteristics (ROC) analysis, and (2) to construct two near-optimal detectors, both generalized likelihood ratio tests as upper performance bounds. We found that for finite samples of (10-100) images, even when the t-test with single-voxel variance estimates (single-voxel t-test) is the "correct" (i.e. unbiased) model for simple local additive signals, better detection performance is obtained using pooled variance estimates or adaptive, multivariate covariance-based detectors. Normalization by voxel-based variance estimates causes significantly decreased detection performance using either single-voxel t-tests or correlation-coefficient thresholding compared to pooled-variance t-tests or covariance thresholding, respectively. Moreover, we found that SVD by itself, or followed by an adaptive Fisher linear discriminant, provides a detector that is (1) more sensitive to mean differences than a single-voxel t-test, (2) insensitive to the large local signal variances detected by covariance thresholding, and (3) much more sensitive to signal correlations than correlation-coefficient thresholding. Adaptive, multivariate covariance-based approaches and pooled-variance t-tests represent promising directions for obtaining optimal signal detection in functional neuroimaging studies.

The quantitative evaluation of functional neuroimaging experiments: the NPAIRS data analysis framework

We introduce a data-analysis framework and performance metrics for evaluating and optimizing the interaction between activation tasks, experimental designs, and the methodological choices and tools for data acquisition, preprocessing, data analysis, and extraction of statistical parametric maps (SPMs). Our NPAIRS (nonparametric prediction, activation, influence, and reproducibility resampling) framework provides an alternative to simulations and ROC curves by using real PET and fMRI data sets to examine the relationship between prediction accuracy and the signal-to-noise ratios (SNRs) associated with reproducible SPMs. Using cross-validation resampling we plot training-test set predictions of the experimental design variables (e.g., brain-state labels) versus reproducibility SNR metrics for the associated SPMs. We demonstrate the utility of this framework across the wide range of performance metrics obtained from [(15)O]water PET studies of 12 age- and sex-matched data sets performing different motor tasks (8 subjects/set). For the 12 data sets we apply NPAIRS with both univariate and multivariate data-analysis approaches to: (1) demonstrate that this framework may be used to obtain reproducible SPMs from any data-analysis approach on a common Z-score scale (rSPM[Z]); (2) demonstrate that the histogram of a rSPM[Z] image may be modeled as the sum of a data-analysis-dependent noise distribution and a task-dependent, Gaussian signal distribution that scales monotonically with our reproducibility performance metric; (3) explore the relation between prediction and reproducibility performance metrics with an emphasis on bias-variance tradeoffs for flexible, multivariate models; and (4) measure the broad range of reproducibility SNRs and the significant influence of individual subjects. A companion paper describes learning curves for four of these 12 data sets, which describe an alternative mutual-information prediction metric and NPAIRS reproducibility as a function of training-set sizes from 2 to 18 subjects. We propose the NPAIRS framework as a validation tool for testing and optimizing methodological choices and tools in functional neuroimaging.

The quantitative evaluation of functional neuroimaging experiments: mutual information learning curves

Learning curves are presented as an unbiased means for evaluating the performance of models for neuroimaging data analysis. The learning curve measures the predictive performance in terms of the generalization or prediction error as a function of the number of independent examples (e.g., subjects) used to determine the parameters in the model. Cross-validation resampling is used to obtain unbiased estimates of a generic multivariate Gaussian classifier, for training set sizes from 2 to 16 subjects. We apply the framework to four different activation experiments, in this case [(15)O]water data sets, although the framework is equally valid for multisubject fMRI studies. We demonstrate how the prediction error can be expressed as the mutual information between the scan and the scan label, measured in units of bits. The mutual information learning curve can be used to evaluate the impact of different methodological choices, e.g., classification label schemes, preprocessing choices. Another application for the learning curve is to examine the model performance using bias/variance considerations enabling the researcher to determine if the model performance is limited by statistical bias or variance. We furthermore present the sensitivity map as a general method for extracting activation maps from statistical models within the probabilistic framework and illustrate relationships between mutual information and pattern reproducibility as derived in the NPAIRS framework described in a companion paper.

Abnormal functional connectivity in posttraumatic stress disorder

This study investigated the efficacy of a combined multivariate/resampling procedure for the analysis of PET activation studies. The covariance-based multivariate analysis was used to investigate distributed brain systems in posttraumatic stress disorder (PTSD) patients and matched controls during performance of a working memory task. The results were compared to univariate results obtained in an earlier study. We also examined whether the PTSD patients demonstrated a breakdown in functional connectivity that may be associated with working memory difficulties often experienced by these patients. A resampling procedure was used specifically to test the reliability of measured between-group effects, to avoid mistaken inference on the basis of random intersubject differences. Significant and reproducible differences in network connectivity were obtained for the two groups. The functional connectivity pattern of the patient group was characterized by relatively more activation in the bilateral inferior parietal lobes and the left precentral gyrus than the control group, and less activation in the inferior medial frontal lobe, bilateral middle frontal gyri and right inferior temporal gyrus. The resampling procedure provided direct evidence that working memory updating was abnormal in PTSD patients relative to matched controls. This work focuses on the need to identify extended brain networks (in addition to regionally specific changes) for the full characterization of brain responses in neuroimaging experiments. Our multivariate analysis explicitly measures the reliability of the patterns of functional connectivity we obtain and demonstrates the potential of such analyses for the study of brain network dysfunction in psychopathology.