Iterative Gaussianization: from ICA to random rotations

Beyond Pairwise Connections in Complex Systems: Insights into the Human Multiscale Psychotic Brain

Complex biological systems, like the brain, exhibit intricate multiway and multiscale interactions that drive emergent behaviors. In psychiatry, neural processes extend beyond pairwise connectivity, involving higher-order interactions critical for understanding mental disorders. Conventional brain network studies focus on pairwise links, offering insights into basic connectivity but failing to capture the complexity of neural dysfunction in psychiatric conditions. This study aims to bridge this gap by applying a matrix-based entropy functional to estimate total correlation, a mathematical framework that incorporates multivariate information measures extending beyond pairwise interactions. We apply this framework to fMRI-ICA-derived multiscale brain networks, enabling the investigation of interactions beyond pairwise relationships in the human multiscale brain. Additionally, this approach holds promise for psychiatric studies, providing a new lens through which to explore beyond pairwise brain network interactions. By examining both triple interactions and the latent factors underlying the triadic relationships among intrinsic brain connectivity networks through tensor decomposition, our study presents a novel approach to understanding higher-order brain dynamics. This framework not only enhances our understanding of complex brain functions but also offers new opportunities for investigating pathophysiology, potentially informing more targeted diagnostic and therapeutic strategies. Moreover, the methodology of analyzing multiway interactions beyond pairwise connections can be applied to any signal analysis. In this study, we specifically explore its application to neural signals, demonstrating its power in uncovering complex multiway interaction patterns of brain activity, which provide a window to explore connectivity beyond pairwise interactions in the multiscale functionality of the brain.

Estimating global phase synchronization by quantifying multivariate mutual information and detecting network structure

In neuroscience, phase synchronization (PS) is a crucial mechanism that facilitates information processing and transmission between different brain regions. Specifically, global phase synchronization (GPS) characterizes the degree of PS among multivariate neural signals. In recent years, several GPS methods have been proposed. However, they primarily focus on the collective synchronization behavior of multivariate neural signals, while neglecting the structural difference between oscillator networks. Therefore, in this paper, we introduce a method named total correlation-based synchronization (TCS) to quantify GPS intensity by examining network organization. To evaluate the performance of TCS, we conducted simulations using the Rössler model and compared it to three existing methods: circular omega complexity, hyper-torus synchrony, and symbolic phase difference and permutation entropy. The results indicate that TCS outperforms the other methods at distinguishing the GPS intensity between networks with similar structures. And it offers insight into the separation and integration behavior of signals during synchronization. Furthermore, to validate this method with experimental data, TCS was applied to analyze the GPS variation of multichannel stereo-electroencephalography (SEEG) signals recorded from onset zones of patients with temporal lobe epilepsy. It was observed that the termination of seizures was associated with the increased GPS and the integration of brain regions. Taken together, TCS offers an alternative way to measure GPS of multivariate signals, which may shed new lights on the mechanism of brain functions and neurological disorders, such as learning, memory, epilepsy, and Alzheimer's disease.

Estimating Information Theoretic Measures via Multidimensional Gaussianization

Information theory is an outstanding framework for measuring uncertainty, dependence, and relevance in data and systems. It has several desirable properties for real-world applications: naturally deals with multivariate data, can handle heterogeneous data, and the measures can be interpreted. However, it has not been adopted by a wider audience because obtaining information from multidimensional data is a challenging problem due to the curse of dimensionality. We propose an indirect way of estimating information based on a multivariate iterative Gaussianization transform. The proposed method has a multivariate-to-univariate property: it reduces the challenging estimation of multivariate measures to a composition of marginal operations applied in each iteration of the Gaussianization. Therefore, the convergence of the resulting estimates depends on the convergence of well-understood univariate entropy estimates, and the global error linearly depends on the number of times the marginal estimator is invoked. We introduce Gaussianization-based estimates for Total Correlation, Entropy, Mutual Information, and Kullback-Leibler Divergence. Results on artificial data show that our approach is superior to previous estimators, particularly in high-dimensional scenarios. We also illustrate the method's performance in different fields to obtain interesting insights. We make the tools and datasets publicly available to provide a test bed for analyzing future methodologies.

Subband Independent Component Analysis for Coherence Enhancement

OBJECTIVE

Cortico-muscular coherence (CMC) is becoming a common technique for detection and characterization of functional coupling between the motor cortex and muscle activity. It is typically evaluated between surface electromyogram (sEMG) and electroencephalogram (EEG) signals collected synchronously during controlled movement tasks. However, the presence of noise and activities unrelated to observed motor tasks in sEMG and EEG results in low CMC levels, which often makes functional coupling difficult to detect.

METHODS

In this paper, we introduce Coherent Subband Independent Component Analysis (CoSICA) to enhance synchronous cortico-muscular components in mixtures captured by sEMG and EEG. The methodology relies on filter bank processing to decompose sEMG and EEG signals into frequency bands. Then, it applies independent component analysis along with a component selection algorithm for re-synthesis of sEMG and EEG designed to maximize CMC levels.

RESULTS

We demonstrate the effectiveness of the proposed method in increasing CMC levels across different signal-to-noise ratios first using simulated data. Using neurophysiological data, we then illustrate that CoSICA processing achieves a pronounced enhancement of original CMC.

CONCLUSION

Our findings suggest that the proposed technique provides an effective framework for improving coherence detection.

SIGNIFICANCE

The proposed methodologies will eventually contribute to understanding of movement control and has high potential for translation into clinical practice.

Gaussianization of LA-ICP-MS features to improve calibration in forensic glass comparison

The forensic comparison of glass aims to compare a glass sample of an unknown source with a control glass sample of a known source. In this work, we use multi-elemental features from Laser Ablation Inductively Coupled Plasma with Mass Spectrometry (LA-ICP-MS) to compute a likelihood ratio. This calculation is a complex procedure that generally requires a probabilistic model including the within-source and between-source variabilities of the features. Assuming the within-source variability to be normally distributed is a practical premise with the available data. However, the between-source variability is generally assumed to follow a much more complex distribution, typically described with a kernel density function. In this work, instead of modeling distributions with complex densities, we propose the use of simpler models and the introduction of a data pre-processing step consisting on the Gaussianization of the glass features. In this context, to obtain a better fit of the features with the Gaussian model assumptions, we explore the use of different normalization techniques of the LA-ICP-MS glass features, namely marginal Gaussianization based on histogram matching, marginal Gaussianization based on Yeo-Johnson transformation and a more complex joint Gaussianization using normalizing flows. We report an improvement in the performance of the Likelihood Ratios computed with the previously Gaussianized feature vectors, particularly relevant in their calibration, which implies a more reliable forensic glass comparison.

On the Comparisons of Decorrelation Approaches for Non-Gaussian Neutral Vector Variables

As a typical non-Gaussian vector variable, a neutral vector variable contains nonnegative elements only, and its l₁ -norm equals one. In addition, its neutral properties make it significantly different from the commonly studied vector variables (e.g., the Gaussian vector variables). Due to the aforementioned properties, the conventionally applied linear transformation approaches [e.g., principal component analysis (PCA) and independent component analysis (ICA)] are not suitable for neutral vector variables, as PCA cannot transform a neutral vector variable, which is highly negatively correlated, into a set of mutually independent scalar variables and ICA cannot preserve the bounded property after transformation. In recent work, we proposed an efficient nonlinear transformation approach, i.e., the parallel nonlinear transformation (PNT), for decorrelating neutral vector variables. In this article, we extensively compare PNT with PCA and ICA through both theoretical analysis and experimental evaluations. The results of our investigations demonstrate the superiority of PNT for decorrelating the neutral vector variables.

COVID-19 therapy optimization by AI-driven biomechanical simulations

The COVID-19 disease causes pneumonia in many patients that in the most serious cases evolves into the Acute Distress Respiratory Syndrome (ARDS), requiring assisted ventilation and intensive care. In this context, identification of patients at high risk of developing ARDS is a key point for early clinical management, better clinical outcome and optimization in using the limited resources available in the intensive care units. We propose an AI-based prognostic system that makes predictions of oxygen exchange with arterial blood by using as input lung Computed Tomography (CT), the air flux in lungs obtained from biomechanical simulations and Arterial Blood Gas (ABG) analysis. We developed and investigated the feasibility of this system on a small clinical database of proven COVID-19 cases where the initial CT and various ABG reports were available for each patient. We studied the time evolution of the ABG parameters and found correlation with the morphological information extracted from CT scans and disease outcome. Promising results of a preliminary version of the prognostic algorithm are presented. The ability to predict the evolution of patients' respiratory efficiency would be of crucial importance for disease management.

Functional Connectome of the Human Brain with Total Correlation

Recent studies proposed the use of Total Correlation to describe functional connectivity among brain regions as a multivariate alternative to conventional pairwise measures such as correlation or mutual information. In this work, we build on this idea to infer a large-scale (whole-brain) connectivity network based on Total Correlation and show the possibility of using this kind of network as biomarkers of brain alterations. In particular, this work uses Correlation Explanation (CorEx) to estimate Total Correlation. First, we prove that CorEx estimates of Total Correlation and clustering results are trustable compared to ground truth values. Second, the inferred large-scale connectivity network extracted from the more extensive open fMRI datasets is consistent with existing neuroscience studies, but, interestingly, can estimate additional relations beyond pairwise regions. And finally, we show how the connectivity graphs based on Total Correlation can also be an effective tool to aid in the discovery of brain diseases.

Exploring Dimensionality Reduction Techniques in Multilingual Transformers

In scientific literature and industry, semantic and context-aware Natural Language Processing-based solutions have been gaining importance in recent years. The possibilities and performance shown by these models when dealing with complex Human Language Understanding tasks are unquestionable, from conversational agents to the fight against disinformation in social networks. In addition, considerable attention is also being paid to developing multilingual models to tackle the language bottleneck. An increase in size has accompanied the growing need to provide more complex models implementing all these features without being conservative in the number of dimensions required. This paper aims to provide a comprehensive account of the impact of a wide variety of dimensional reduction techniques on the performance of different state-of-the-art multilingual siamese transformers, including unsupervised dimensional reduction techniques such as linear and nonlinear feature extraction, feature selection, and manifold techniques. In order to evaluate the effects of these techniques, we considered the multilingual extended version of Semantic Textual Similarity Benchmark (mSTSb) and two different baseline approaches, one using the embeddings from the pre-trained version of five models and another using their fine-tuned STS version. The results evidence that it is possible to achieve an average reduction of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$91.58\% \pm 2.59\%$$\end{document}91.58%±2.59% in the number of dimensions of embeddings from pre-trained models requiring a fitting time \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$96.68\% \pm 0.68\%$$\end{document}96.68%±0.68% faster than the fine-tuning process. Besides, we achieve \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$54.65\% \pm 32.20\%$$\end{document}54.65%±32.20% dimensionality reduction in embeddings from fine-tuned models. The results of this study will significantly contribute to the understanding of how different tuning approaches affect performance on semantic-aware tasks and how dimensional reduction techniques deal with the high-dimensional embeddings computed for the STS task and their potential for other highly demanding NLP tasks.

Information Flow in Biological Networks for Color Vision

Biological neural networks for color vision (also known as color appearance models) consist of a cascade of linear + nonlinear layers that modify the linear measurements at the retinal photo-receptors leading to an internal (nonlinear) representation of color that correlates with psychophysical experience. The basic layers of these networks include: (1) chromatic adaptation (normalization of the mean and covariance of the color manifold); (2) change to opponent color channels (PCA-like rotation in the color space); and (3) saturating nonlinearities to obtain perceptually Euclidean color representations (similar to dimension-wise equalization). The Efficient Coding Hypothesis argues that these transforms should emerge from information-theoretic goals. In case this hypothesis holds in color vision, the question is what is the coding gain due to the different layers of the color appearance networks? In this work, a representative family of color appearance models is analyzed in terms of how the redundancy among the chromatic components is modified along the network and how much information is transferred from the input data to the noisy response. The proposed analysis is performed using data and methods that were not available before: (1) new colorimetrically calibrated scenes in different CIE illuminations for the proper evaluation of chromatic adaptation; and (2) new statistical tools to estimate (multivariate) information-theoretic quantities between multidimensional sets based on Gaussianization. The results confirm that the efficient coding hypothesis holds for current color vision models, and identify the psychophysical mechanisms critically responsible for gains in information transference: opponent channels and their nonlinear nature are more important than chromatic adaptation at the retina.

Functional connectivity inference from fMRI data using multivariate information measures

Shannon's entropy or an extension of Shannon's entropy can be used to quantify information transmission between or among variables. Mutual information is the pair-wise information that captures nonlinear relationships between variables. It is more robust than linear correlation methods. Beyond mutual information, two generalizations are defined for multivariate distributions: interaction information or co-information and total correlation or multi-mutual information. In comparison to mutual information, interaction information and total correlation are underutilized and poorly studied in applied neuroscience research. Quantifying information flow between brain regions is not explicitly explained in neuroscience by interaction information and total correlation. This article aims to clarify the distinctions between the neuroscience concepts of mutual information, interaction information, and total correlation. Additionally, we proposed a novel method for determining the interaction information between three variables using total correlation and conditional mutual information. On the other hand, how to apply it properly in practical situations. We supplied both simulation experiments and real neural studies to estimate functional connectivity in the brain with the above three higher-order information-theoretic approaches. In order to capture redundancy information for multivariate variables, we discovered that interaction information and total correlation were both robust, and it could be able to capture both well-known and yet-to-be-discovered functional brain connections.

Predictive Coding, Variational Autoencoders, and Biological Connections

We present a review of predictive coding, from theoretical neuroscience, and variational autoencoders, from machine learning, identifying the common origin and mathematical framework underlying both areas. As each area is prominent within its respective field, more firmly connecting these areas could prove useful in the dialogue between neuroscience and machine learning. After reviewing each area, we discuss two possible correspondences implied by this perspective: cortical pyramidal dendrites as analogous to (nonlinear) deep networks and lateral inhibition as analogous to normalizing flows. These connections may provide new directions for further investigations in each field.

A minimal model of the interaction of social and individual learning

Learning is thought to be achieved by the selective, activity dependent, adjustment of synaptic connections. Individual learning can also be very hard and/or slow. Social, supervised, learning from others might amplify individual, possibly mainly unsupervised, learning by individuals, and might underlie the development and evolution of culture. We studied a minimal neural network model of the interaction of individual, unsupervised, and social supervised learning by communicating "agents". Individual agents attempted to learn to track a hidden fluctuating "source", which, linearly mixed with other masking fluctuations, generated observable input vectors. In this model data are generated linearly, facilitating mathematical analysis. Learning was driven either solely by direct observation of input data (unsupervised, Hebbian) or, in addition, by observation of another agent's output (supervised, Delta rule). To make learning more difficult, and to enhance biological realism, the learning rules were made slightly connection-inspecific, so that incorrect individual learning sometimes occurs. We found that social interaction can foster both correct and incorrect learning. Useful social learning therefore presumably involves additional factors some of which we outline.

Distinguishing type II focal cortical dysplasias from normal cortex: A novel normative modeling approach

OBJECTIVE

Focal cortical dysplasias (FCDs) are a common cause of apparently non-lesional drug-resistant focal epilepsy. Visual detection of subtle FCDs on MRI is clinically important and often challenging. In this study, we implement a set of 3D local image filters adapted from computer vision applications to characterize the appearance of normal cortex surrounding the gray-white junction. We create a normative model to serve as the basis for a novel multivariate constrained outlier approach to automated FCD detection.

METHODS

Standardized MPRAGE, T₂ and FLAIR MR images were obtained in 15 patients with radiologically or histologically diagnosed FCDs and 30 healthy volunteers. Multiscale 3D local image filters were computed for each MR contrast then sampled onto the gray-white junction surface. Using an iterative Gaussianization procedure, we created a normative model of cortical variability in healthy volunteers, allowing for identification of outlier regions and estimates of similarity in normal cortex and FCD lesions. We used a constrained outlier approach following local normalization to automatically detect FCD lesions based on projection onto the mean FCD feature vector.

RESULTS

FCDs as well as some normal cortical regions such as primary sensorimotor and paralimbic regions appear as outliers. Regions such as the paralimbic regions and the anterior insula have similar features to FCDs. Our constrained outlier approach allows for automated FCD detection with 80% sensitivity and 70% specificity.

SIGNIFICANCE

A normative model using multiscale local image filters can be used to describe the normal cortical variability. Although FCDs appear similar to some cortical regions such as the anterior insula and paralimbic cortices, they can be identified using a constrained outlier detection approach. Our method for detecting outliers and estimating similarity is generic and could be extended to identification of other types of lesions or atypical cortical areas.

Spatio-chromatic information available from different neural layers via Gaussianization

How much visual information about the retinal images can be extracted from the different layers of the visual pathway? This question depends on the complexity of the visual input, the set of transforms applied to this multivariate input, and the noise of the sensors in the considered layer. Separate subsystems (e.g. opponent channels, spatial filters, nonlinearities of the texture sensors) have been suggested to be organized for optimal information transmission. However, the efficiency of these different layers has not been measured when they operate together on colorimetrically calibrated natural images and using multivariate information-theoretic units over the joint spatio-chromatic array of responses.In this work, we present a statistical tool to address this question in an appropriate (multivariate) way. Specifically, we propose an empirical estimate of the information transmitted by the system based on a recent Gaussianization technique. The total correlation measured using the proposed estimator is consistent with predictions based on the analytical Jacobian of a standard spatio-chromatic model of the retina-cortex pathway. If the noise at certain representation is proportional to the dynamic range of the response, and one assumes sensors of equivalent noise level, then transmitted information shows the following trends: (1) progressively deeper representations are better in terms of the amount of captured information, (2) the transmitted information up to the cortical representation follows the probability of natural scenes over the chromatic and achromatic dimensions of the stimulus space, (3) the contribution of spatial transforms to capture visual information is substantially greater than the contribution of chromatic transforms, and (4) nonlinearities of the responses contribute substantially to the transmitted information but less than the linear transforms.

Extracting Reproducible Time-Resolved Resting State Networks Using Dynamic Mode Decomposition

Resting state networks (RSNs) extracted from functional magnetic resonance imaging (fMRI) scans are believed to reflect the intrinsic organization and network structure of brain regions. Most traditional methods for computing RSNs typically assume these functional networks are static throughout the duration of a scan lasting 5-15 min. However, they are known to vary on timescales ranging from seconds to years; in addition, the dynamic properties of RSNs are affected in a wide variety of neurological disorders. Recently, there has been a proliferation of methods for characterizing RSN dynamics, yet it remains a challenge to extract reproducible time-resolved networks. In this paper, we develop a novel method based on dynamic mode decomposition (DMD) to extract networks from short windows of noisy, high-dimensional fMRI data, allowing RSNs from single scans to be resolved robustly at a temporal resolution of seconds. After validating the method on a synthetic dataset, we analyze data from 120 individuals from the Human Connectome Project and show that unsupervised clustering of DMD modes discovers RSNs at both the group (gDMD) and the single subject (sDMD) levels. The gDMD modes closely resemble canonical RSNs. Compared to established methods, sDMD modes capture individualized RSN structure that both better resembles the population RSN and better captures subject-level variation. We further leverage this time-resolved sDMD analysis to infer occupancy and transitions among RSNs with high reproducibility. This automated DMD-based method is a powerful tool to characterize spatial and temporal structures of RSNs in individual subjects.

Sequential Independent Component Analysis Density Estimation

A problem of multivariate probability density function estimation by exploiting linear independent components analysis (ICA) is addressed. Historically, ICA density estimation was initially proposed under the name projection pursuit density estimation (PPDE) and two basic methods, named forward and backward, were published. We derive a modification of the forward PPDE method, which avoids a computationally demanding optimization involving Monte Carlo sampling of the original method. The results of the experiments show that the proposed method presents an attractive choice for density estimation, which is pronounced for a small number of training observations. Under such conditions, our method usually outperforms model-based Gaussian mixture model. We also found that our method obtained better results than the backward PPDE methods in the situation of nonfactorizable underlying density functions. The proposed method has demonstrated a competitive performance compared with the support vector machine and the extreme learning machine in some real classification tasks.

Decorrelation of Neutral Vector Variables: Theory and Applications

In this paper, we propose novel strategies for neutral vector variable decorrelation. Two fundamental invertible transformations, namely, serial nonlinear transformation and parallel nonlinear transformation, are proposed to carry out the decorrelation. For a neutral vector variable, which is not multivariate-Gaussian distributed, the conventional principal component analysis cannot yield mutually independent scalar variables. With the two proposed transformations, a highly negatively correlated neutral vector can be transformed to a set of mutually independent scalar variables with the same degrees of freedom. We also evaluate the decorrelation performances for the vectors generated from a single Dirichlet distribution and a mixture of Dirichlet distributions. The mutual independence is verified with the distance correlation measurement. The advantages of the proposed decorrelation strategies are intensively studied and demonstrated with synthesized data and practical application evaluations.

Fusing Partial Camera Signals for Noncontact Pulse Rate Variability Measurement

Remote camera-based measurement of physiology has great potential for healthcare and affective computing. Recent advances in computer vision and signal processing have enabled photoplethysmography (PPG) measurement using commercially available cameras. However, there remain challenges in recovering accurate noncontact PPG measurements in the presence of rigid head motion. When a subject is moving, their face may be turned away from one camera, be obscured by an object, or move out of the frame resulting in missing observations. As the calculation of pulse rate variability (PRV) requires analysis over a time window of several minutes, the effect of missing observations on such features is deleterious. We present an approach for fusing partial color-channel signals from an array of cameras that enable physiology measurements to be made from moving subjects, even if they leave the frame of one or more cameras, which would not otherwise be possible with only a single camera. We systematically test our method on subjects ( N=25) using a set of six, 5-min tasks (each repeated twice) involving different levels of head motion. This results in validation across 25 h of measurement. We evaluate pulse rate and PRV parameter estimation including statistical, geometric, and frequency-based measures. The median absolute error in pulse rate measurements was 0.57 beats-per-minute (BPM). In all but two tasks with the greatest motion, the median error was within 0.4 BPM of that from a contact PPG device. PRV estimates were significantly improved using our proposed approach compared to an alternative not designed to handle missing values and multiple camera signals; the error was reduced by over 50%. Without our proposed method, errors in pulse rate would be very high, and estimation of PRV parameters would not be feasible due to significant data loss.

Gaussian-binary restricted Boltzmann machines for modeling natural image statistics

We present a theoretical analysis of Gaussian-binary restricted Boltzmann machines (GRBMs) from the perspective of density models. The key aspect of this analysis is to show that GRBMs can be formulated as a constrained mixture of Gaussians, which gives a much better insight into the model’s capabilities and limitations. We further show that GRBMs are capable of learning meaningful features without using a regularization term and that the results are comparable to those of independent component analysis. This is illustrated for both a two-dimensional blind source separation task and for modeling natural image patches. Our findings exemplify that reported difficulties in training GRBMs are due to the failure of the training algorithm rather than the model itself. Based on our analysis we derive a better training setup and show empirically that it leads to faster and more robust training of GRBMs. Finally, we compare different sampling algorithms for training GRBMs and show that Contrastive Divergence performs better than training methods that use a persistent Markov chain.

Statistical Modeling of Retinal Optical Coherence Tomography

In this paper, a new model for retinal Optical Coherence Tomography (OCT) images is proposed. This statistical model is based on introducing a nonlinear Gaussianization transform to convert the probability distribution function (pdf) of each OCT intra-retinal layer to a Gaussian distribution. The retina is a layered structure and in OCT each of these layers has a specific pdf which is corrupted by speckle noise, therefore a mixture model for statistical modeling of OCT images is proposed. A Normal-Laplace distribution, which is a convolution of a Laplace pdf and Gaussian noise, is proposed as the distribution of each component of this model. The reason for choosing Laplace pdf is the monotonically decaying behavior of OCT intensities in each layer for healthy cases. After fitting a mixture model to the data, each component is gaussianized and all of them are combined by Averaged Maximum A Posterior (AMAP) method. To demonstrate the ability of this method, a new contrast enhancement method based on this statistical model is proposed and tested on thirteen healthy 3D OCTs taken by the Topcon 3D OCT and five 3D OCTs from Age-related Macular Degeneration (AMD) patients, taken by Zeiss Cirrus HD-OCT. Comparing the results with two contending techniques, the prominence of the proposed method is demonstrated both visually and numerically. Furthermore, to prove the efficacy of the proposed method for a more direct and specific purpose, an improvement in the segmentation of intra-retinal layers using the proposed contrast enhancement method as a preprocessing step, is demonstrated.

Visual aftereffects and sensory nonlinearities from a single statistical framework

When adapted to a particular scenery our senses may fool us: colors are misinterpreted, certain spatial patterns seem to fade out, and static objects appear to move in reverse. A mere empirical description of the mechanisms tuned to color, texture, and motion may tell us where these visual illusions come from. However, such empirical models of gain control do not explain why these mechanisms work in this apparently dysfunctional manner. Current normative explanations of aftereffects based on scene statistics derive gain changes by (1) invoking decorrelation and linear manifold matching/equalization, or (2) using nonlinear divisive normalization obtained from parametric scene models. These principled approaches have different drawbacks: the first is not compatible with the known saturation nonlinearities in the sensors and it cannot fully accomplish information maximization due to its linear nature. In the second, gain change is almost determined a priori by the assumed parametric image model linked to divisive normalization. In this study we show that both the response changes that lead to aftereffects and the nonlinear behavior can be simultaneously derived from a single statistical framework: the Sequential Principal Curves Analysis (SPCA). As opposed to mechanistic models, SPCA is not intended to describe how physiological sensors work, but it is focused on explaining why they behave as they do. Nonparametric SPCA has two key advantages as a normative model of adaptation: (i) it is better than linear techniques as it is a flexible equalization that can be tuned for more sensible criteria other than plain decorrelation (either full information maximization or error minimization); and (ii) it makes no a priori functional assumption regarding the nonlinearity, so the saturations emerge directly from the scene data and the goal (and not from the assumed function). It turns out that the optimal responses derived from these more sensible criteria and SPCA are consistent with dysfunctional behaviors such as aftereffects.

Principal polynomial analysis

This paper presents a new framework for manifold learning based on a sequence of principal polynomials that capture the possibly nonlinear nature of the data. The proposed Principal Polynomial Analysis (PPA) generalizes PCA by modeling the directions of maximal variance by means of curves, instead of straight lines. Contrarily to previous approaches, PPA reduces to performing simple univariate regressions, which makes it computationally feasible and robust. Moreover, PPA shows a number of interesting analytical properties. First, PPA is a volume-preserving map, which in turn guarantees the existence of the inverse. Second, such an inverse can be obtained in closed form. Invertibility is an important advantage over other learning methods, because it permits to understand the identified features in the input domain where the data has physical meaning. Moreover, it allows to evaluate the performance of dimensionality reduction in sensible (input-domain) units. Volume preservation also allows an easy computation of information theoretic quantities, such as the reduction in multi-information after the transform. Third, the analytical nature of PPA leads to a clear geometrical interpretation of the manifold: it allows the computation of Frenet-Serret frames (local features) and of generalized curvatures at any point of the space. And fourth, the analytical Jacobian allows the computation of the metric induced by the data, thus generalizing the Mahalanobis distance. These properties are demonstrated theoretically and illustrated experimentally. The performance of PPA is evaluated in dimensionality and redundancy reduction, in both synthetic and real datasets from the UCI repository.

Projection-pursuit-based method for blind separation of nonnegative sources

This paper presents a projection pursuit (PP) based method for blind separation of nonnegative sources. First, the available observation matrix is mapped to construct a new mixing model, in which the inaccessible source matrix is normalized to be column-sum-to-1. Then, the PP method is proposed to solve this new model, where the mixing matrix is estimated column by column through tracing the projections to the mapped observations in specified directions, which leads to the recovery of the sources. The proposed method is much faster than Chan's method, which has similar assumptions to ours, due to the usage of optimal projection. It is also more advantageous in separating cross-correlated sources than the independence- and uncorrelation-based methods, as it does not employ any statistical information of the sources. Furthermore, the new method does not require the mixing matrix to be nonnegative. Simulation results demonstrate the superior performance of our method.

SOMKE: kernel density estimation over data streams by sequences of self-organizing maps

In this paper, we propose a novel method SOMKE, for kernel density estimation (KDE) over data streams based on sequences of self-organizing map (SOM). In many stream data mining applications, the traditional KDE methods are infeasible because of the high computational cost, processing time, and memory requirement. To reduce the time and space complexity, we propose a SOM structure in this paper to obtain well-defined data clusters to estimate the underlying probability distributions of incoming data streams. The main idea of this paper is to build a series of SOMs over the data streams via two operations, that is, creating and merging the SOM sequences. The creation phase produces the SOM sequence entries for windows of the data, which obtains clustering information of the incoming data streams. The size of the SOM sequences can be further reduced by combining the consecutive entries in the sequence based on the measure of Kullback-Leibler divergence. Finally, the probability density functions over arbitrary time periods along the data streams can be estimated using such SOM sequences. We compare SOMKE with two other KDE methods for data streams, the M-kernel approach and the cluster kernel approach, in terms of accuracy and processing time for various stationary data streams. Furthermore, we also investigate the use of SOMKE over nonstationary (evolving) data streams, including a synthetic nonstationary data stream, a real-world financial data stream and a group of network traffic data streams. The simulation results illustrate the effectiveness and efficiency of the proposed approach.

Nonlinearities and adaptation of color vision from sequential principal curves analysis

Mechanisms of human color vision are characterized by two phenomenological aspects: the system is nonlinear and adaptive to changing environments. Conventional attempts to derive these features from statistics use separate arguments for each aspect. The few statistical explanations that do consider both phenomena simultaneously follow parametric formulations based on empirical models. Therefore, it may be argued that the behavior does not come directly from the color statistics but from the convenient functional form adopted. In addition, many times the whole statistical analysis is based on simplified databases that disregard relevant physical effects in the input signal, as, for instance, by assuming flat Lambertian surfaces. In this work, we address the simultaneous statistical explanation of the nonlinear behavior of achromatic and chromatic mechanisms in a fixed adaptation state and the change of such behavior (i.e., adaptation) under the change of observation conditions. Both phenomena emerge directly from the samples through a single data-driven method: the sequential principal curves analysis (SPCA) with local metric. SPCA is a new manifold learning technique to derive a set of sensors adapted to the manifold using different optimality criteria. Here sequential refers to the fact that sensors (curvilinear dimensions) are designed one after the other, and not to the particular (eventually iterative) method to draw a single principal curve. Moreover, in order to reproduce the empirical adaptation reported under D65 and A illuminations, a new database of colorimetrically calibrated images of natural objects under these illuminants was gathered, thus overcoming the limitations of available databases. The results obtained by applying SPCA show that the psychophysical behavior on color discrimination thresholds, discount of the illuminant, and corresponding pairs in asymmetric color matching emerge directly from realistic data regularities, assuming no a priori functional form. These results provide stronger evidence for the hypothesis of a statistically driven organization of color sensors. Moreover, the obtained results suggest that the nonuniform resolution of color sensors at this low abstraction level may be guided by an error-minimization strategy rather than by an information-maximization goal.

Classifiability-based discriminatory projection pursuit

Fisher's linear discriminant (FLD) is one of the most widely used linear feature extraction method, especially in many visual computation tasks. Based on the analysis on several limitations of the traditional FLD, this paper attempts to propose a new computational paradigm for discriminative linear feature extraction, named "classifiability-based discriminatory projection pursuit" (CDPP), which is different from the traditional FLD and its variants. There are two steps in the proposed CDPP: one is the construction of a candidate projection set (CPS), and the other is the pursuit of discriminatory projections. Specifically, in the former step, candidate projections are generated by using the nearest between-class boundary samples, while the latter is efficiently achieved by classifiability-based AdaBoost learning from the CPS. We show that the new "projection pursuit" paradigm not only does not suffer from the limitations of the traditional FLD but also inherits good generalizability from the boundary attribute of candidate projections. Extensive experiments on both synthetic and real datasets validate the effectiveness of CDPP for discriminative linear feature extraction.