Model of multi-modal cortical processing: coherent learning in self-organizing modules

A Dynamic Neural Field Model of Multimodal Merging: Application to the Ventriloquist Effect

Multimodal merging encompasses the ability to localize stimuli based on imprecise information sampled through individual senses such as sight and hearing. Merging decisions are standardly described using Bayesian models that fit behaviors over many trials, encapsulated in a probability distribution. We introduce a novel computational model based on dynamic neural fields able to simulate decision dynamics and generate localization decisions, trial by trial, adapting to varying degrees of discrepancy between audio and visual stimulations. Neural fields are commonly used to model neural processes at a mesoscopic scale-for instance, neurophysiological activity in the superior colliculus. Our model is fit to human psychophysical data of the ventriloquist effect, additionally testing the influence of retinotopic projection onto the superior colliculus and providing a quantitative performance comparison to the Bayesian reference model. While models perform equally on average, a qualitative analysis of free parameters in our model allows insights into the dynamics of the decision and the individual variations in perception caused by noise. We finally show that the increase in the number of free parameters does not result in overfitting and that the parameter space may be either reduced to fit specific criteria or exploited to perform well on more demanding tasks in the future. Indeed, beyond decision or localization tasks, our model opens the door to the simulation of behavioral dynamics, as well as saccade generation driven by multimodal stimulation.

Improved multi-view privileged support vector machine

Multi-view learning (MVL) concentrates on the problem of learning from the data represented by multiple distinct feature sets. The consensus and complementarity principles play key roles in multi-view modeling. By exploiting the consensus principle or the complementarity principle among different views, various successful support vector machine (SVM)-based multi-view learning models have been proposed for performance improvement. Recently, a framework of learning using privileged information (LUPI) has been proposed to model data with complementary information. By bridging connections between the LUPI paradigm and multi-view learning, we have presented a privileged SVM-based two-view classification model, named PSVM-2V, satisfying both principles simultaneously. However, it can be further improved in these three aspects: (1) fully unleash the power of the complementary information among different views; (2) extend to multi-view case; (3) construct a more efficient optimization solver. Therefore, in this paper, we propose an improved privileged SVM-based model for multi-view learning, termed as IPSVM-MV. It directly follows the standard LUPI model to fully utilize the multi-view complementary information; also it is a general model for multi-view scenario, and an alternating direction method of multipliers (ADMM) is employed to solve the corresponding optimization problem efficiently. Further more, we theoretically analyze the performance of IPSVM-MV from the viewpoints of the consensus principle and the generalization error bound. Experimental results on 75 binary data sets demonstrate the effectiveness of the proposed method; here we mainly concentrate on two-view case to compare with state-of-the-art methods.

Emergence in the central nervous system

"Emergence" is an idea that has received much attention in consciousness literature, but it is difficult to find characterizations of that concept which are both specific and useful. I will precisely define and characterize a type of epistemic ("weak") emergence and show that it is a property of some neural circuits throughout the CNS, on micro-, meso- and macroscopic levels. I will argue that possession of this property can result in profoundly altered neural dynamics on multiple levels in cortex and other systems. I will first describe emergent neural entities (ENEs) abstractly. I will then show how ENEs function specifically and concretely, and demonstrate some implications of this type of emergence for the CNS.

Coupling BCM and neural fields for the emergence of self-organization consensus

Human beings interact with the environment through different modalities, i.e. perceptions and actions, processed in the cortex by dedicated brain areas. These areas are self-organized, so that spatially close neurons are sensitive to close stimuli, providing generalization from previously learned examples. Although perceptive flows are picked up by different spatially separated sensors, their processings are not isolated. On the contrary, they are constantly interacting, as illustrated by the McGurk effect. When the auditory stimulus /ba/ and the /ga/ lip movement are presented simultaneously, people perceive a /da/, which does not correspond to any of the stimuli. Merging several stimuli into one multimodal perception reduces ambiguities and noises and is essential to interact with the environment. This article proposes a model for modality association, inspired by the biological properties of the cortex. The model consists of modality maps interacting through an associative map to raise a consistent multimodal perception of the environment. We propose the coupling of BCM learning rule and neural maps to obtain the decentralized and unsupervised self-organization of each modal map influenced by the multisensory context. We obtain local self-organization of modal maps with various inputs and discretizations.

Implicit and explicit representations

During the past decades, the symbol grounding problem, as has been identified by Harnard [Harnard, S. (1990). The symbol grounding problem. Physica D: Nonlinear Phenomena, 42, 335-346], became a prominent problem in the cognitive science society. The idea that a symbol is much more than a mere meaningless token that can be processed through some algorithm, sheds new light on higher brain functions such as language and cognition. We present in this article a computational framework that may help in our understanding of the nature of grounded representations. Two models are briefly introduced that aim at emphasizing the difference we make between implicit and explicit representations.