LGN-CNN: A biologically inspired CNN architecture

In this paper we introduce a biologically inspired Convolutional Neural Network (CNN) architecture called LGN-CNN that has a first convolutional layer composed of a single filter that mimics the role of the Lateral Geniculate Nucleus (LGN). The first layer of the neural network shows a rotational symmetric pattern justified by the structure of the net itself that turns up to be an approximation of a Laplacian of Gaussian (LoG). The latter function is in turn a good approximation of the receptive field profiles (RFPs) of the cells in the LGN. The analogy with the visual system is established, emerging directly from the architecture of the neural network. A proof of rotation invariance of the first layer is given on a fixed LGN-CNN architecture and the computational results are shown. Thus, contrast invariance capability of the LGN-CNN is investigated and a comparison between the Retinex effects of the first layer of LGN-CNN and the Retinex effects of a LoG is provided on different images. A statistical study is done on the filters of the second convolutional layer with respect to biological data. In conclusion, the model we have introduced approximates well the RFPs of both LGN and V1 attaining similar behavior as regards long range connections of LGN cells that show Retinex effects.

Identification of complex shapes using a self organizing neural system

We present a multilayer hierarchical neural system for automatic classification of complex contour patterns. The system consists of a neocognitron-like network structure combined with self-organizing maps to automatically determine feature classes. We present results showing that multilayer hierarchical networks are able to tolerate pattern distortion considerably better than standard neural network implementations.

Inductive bias strength in knowledge-based neural networks: application to magnetic resonance spectroscopy of breast tissues

The integration of symbolic knowledge with artificial neural networks is becoming an increasingly popular paradigm for solving real-world applications. The paradigm provides means for using prior knowledge to determine the network architecture, to program a subset of weights to induce a learning bias which guide network training, and to extract knowledge from trained networks. The role of neural networks then becomes that of knowledge refinement. It thus provides a methodology for dealing with uncertainty in the prior knowledge. We address the open question of how to determine the strength of the inductive bias of programmed weights; we present a quantitative solution which takes the network architecture, the prior knowledge, and the training data into consideration. We apply our solution to the difficult problem of analyzing breast tissue from magnetic resonance spectroscopy (MRS); the available database is extremely limited and cannot be adequately explained by expert knowledge alone.

Neurons with two sites of synaptic integration learn invariant representations

Neurons in mammalian cerebral cortex combine specific responses with respect to some stimulus features with invariant responses to other stimulus features. For example, in primary visual cortex, complex cells code for orientation of a contour but ignore its position to a certain degree. In higher areas, such as the inferotemporal cortex, translation-invariant, rotation-invariant, and even view point-invariant responses can be observed. Such properties are of obvious interest to artificial systems performing tasks like pattern recognition. It remains to be resolved how such response properties develop in biological systems. Here we present an unsupervised learning rule that addresses this problem. It is based on a neuron model with two sites of synaptic integration, allowing qualitatively different effects of input to basal and apical dendritic trees, respectively. Without supervision, the system learns to extract invariance properties using temporal or spatial continuity of stimuli. Furthermore, top-down information can be smoothly integrated in the same framework. Thus, this model lends a physiological implementation to approaches of unsupervised learning of invariant-response properties.

A spike based learning rule for generation of invariant representations

For biological realism, models of learning in neuronal networks often assume that synaptic plasticity solely depends on locally available signals, in particular only on the activity of the pre- and post-synaptic cells. As a consequence, synapses influence the plasticity of other synapses exclusively via the post-synaptic activity. Inspired by recent research on the properties of apical dendrites it has been suggested, that a second integration site in the apical dendrite may mediate specific global information. Here we explore this issue considering the example of learning invariant responses by examining a network of spiking neurones with two sites of synaptic integration. We demonstrate that results obtained in networks of units with continuous outputs transfer to the more realistic neuronal model. This allows a number of more specific experimental predictions, and is a necessary step to unified description of learning rules exploiting timing of action potentials.

Neural-network classifiers for automatic real-world aerial image recognition

We describe the application of the multilayer perceptron (MLP) network and a version of the adaptive resonance theory version 2-A (ART 2-A) network to the problem of automatic aerial image recognition (AAIR). The classification of aerial images, independent of their positions and orientations, is required for automatic tracking and target recognition. Invariance is achieved by the use of different invariant feature spaces in combination with supervised and unsupervised neural networks. The performance of neural-network-based classifiers in conjunction with several types of invariant AAIR global features, such as the Fourier-transform space, Zernike moments, central moments, and polar transforms, are examined. The advantages of this approach are discussed. The performance of the MLP network is compared with that of a classical correlator. The MLP neural-network correlator outperformed the binary phase-only filter (BPOF) correlator. It was found that the ART 2-A distinguished itself with its speed and its low number of required training vectors. However, only the MLP classifier was able to deal with a combination of shift and rotation geometric distortions.

Highly constrained neural networks for industrial quality control

In this work we investigate techniques for embedding domain-specific spatial invariances into highly-constrained neural networks. This information is used to drastically reduce the number of weights which have to be determined during the learning phase, thus allowing us to apply artificial neural networks to problems characterized by a relatively small number of available examples. As an application of the proposed methodology, we study the problem of optical inspection of machined parts. More specifically, we have characterized the performance of a network created according to this strategy, which accepts images of parts under inspection at its input and issues a flag at its output which states whether the part is defective. The results obtained so far show that the proposed methodology provides a potentially relevant approach for the quality control of industrial parts, as it offers both accuracy and short software development time, when compared with a classifier implemented using a standard approach.