Information-theoretic solutions to early visual information processing: Analytic results

A mathematical analysis of the development of oriented receptive fields in Linsker's model

We consider Linsker's neural network model and study how oriented receptive fields (RFs) are developed at layer F-->G mathematically. We concentrate on the Mexican-hat correlation function and show that this function determines the spatial frequency of RFs. We also focus on the role of the arbor function. Two types of arbor functions, a Gaussian type and a step type, are considered, and it is shown that the Gaussian arbor function develops center-surround RFs while the step arbor function enables the development of multi-lobed RFs in addition to center-surround RFs. The roles of Linsker's parameters k1, k2 are also discussed.

Linsker-type Hebbian Learning: A Qualitative Analysis on the Parameter Space

We developed a new method to relate the choice of system parameters to the outcomes of the unsupervised learning process in Linsker's multi-layer network model. The behavior of this model is determined by the underlying nonlinear dynamics that are parameterized by a set of parameters originating from the Hebb rule and the arbor density of the synapses. These parameters determine the presence or absence of a specific receptive field (or connection pattern) as a saturated fixed point attractor of the model. We derived a necessary and sufficient condition to test whether a given saturated weight vector is stable or not for any given set of system parameters, and used this condition to determine the whole regime in the parameter space over which the given connection pattern is stable. The parameter space approach allows us to investigate the relative stability of the major receptive fields reported in Linsker's simulation, and to demonstrate the crucial role played by the localized arbor density of synapses between adjacent layers. The method presented here can be employed to analyze other learning and retrieval models that use the limiter function as the constraint controlling the magnitude of the weight or state vectors. Copyright 1997 Elsevier Science Ltd.

On Neurodynamics with Limiter Function and Linsker's Developmental Model

The limiter function is used in many learning and retrieval models as the constraint controlling the magnitude of the weight or state vectors. In this paper, we developed a new method to relate the set of saturated fixed points to the set of system parameters of the models that use the limiter function, and then, as a case study, applied this method to Linsker's Hebbian learning network. We derived a necessary and sufficient condition to test whether a given saturated weight or state vector is stable or not for any given set of system parameters, and used this condition to determine the whole regime in the parameter space over which the given state is stable. This approach allows us to investigate the relative stability of the major receptive fields reported in Linsker's simulations, and to demonstrate the crucial role played by the synaptic density functions.

Temporal Differentiation and Violation of Time-Reversal Invariance in Neurocomputation of Visual Information

Information-theoretic techniques have been employed to study the time-dependent connection strength of a three-layer feedforward neural network. The analysis shows (1) there is a natural emergence of time-dependent receptive field that performs temporal differentiation and (2) the result is shown to be a consequence of a mechanism based on violation of the time-reversal invariance in the visual information processing system. Both analytic and numerical studies are presented.