A model of computation in neocortical architecture

Power Function Error Initialization Can Improve Convergence of Backpropagation Learning in Neural Networks for Classification

Supervised learning corresponds to minimizing a loss or cost function expressing the differences between model predictions yn and the target values tn given by the training data. In neural networks, this means backpropagating error signals through the transposed weight matrixes from the output layer toward the input layer. For this, error signals in the output layer are typically initialized by the difference yn- tn, which is optimal for several commonly used loss functions like cross-entropy or sum of squared errors. Here I evaluate a more general error initialization method using power functions |yn- tn|q for q>0, corresponding to a new family of loss functions that generalize cross-entropy. Surprisingly, experiments on various learning tasks reveal that a proper choice of q can significantly improve the speed and convergence of backpropagation learning, in particular in deep and recurrent neural networks. The results suggest two main reasons for the observed improvements. First, compared to cross-entropy, the new loss functions provide better fits to the distribution of error signals in the output layer and therefore maximize the model's likelihood more efficiently. Second, the new error initialization procedure may often provide a better gradient-to-loss ratio over a broad range of neural output activity, thereby avoiding flat loss landscapes with vanishing gradients.

Feed-forward and noise-tolerant detection of feature homogeneity in spiking networks with a latency code

In studies of the visual system as well as in computer vision, the focus is often on contrast edges. However, the primate visual system contains a large number of cells that are insensitive to spatial contrast and, instead, respond to uniform homogeneous illumination of their visual field. The purpose of this information remains unclear. Here, we propose a mechanism that detects feature homogeneity in visual areas, based on latency coding and spike time coincidence, in a purely feed-forward and therefore rapid manner. We demonstrate how homogeneity information can interact with information on contrast edges to potentially support rapid image segmentation. Furthermore, we analyze how neuronal crosstalk (noise) affects the mechanism's performance. We show that the detrimental effects of crosstalk can be partly mitigated through delayed feed-forward inhibition that shapes bi-phasic post-synaptic events. The delay of the feed-forward inhibition allows effectively controlling the size of the temporal integration window and, thereby, the coincidence threshold. The proposed model is based on single-spike latency codes in a purely feed-forward architecture that supports low-latency processing, making it an attractive scheme of computation in spiking neuronal networks where rapid responses and low spike counts are desired.

GP-Select: Accelerating EM Using Adaptive Subspace Preselection

We propose a nonparametric procedure to achieve fast inference in generative graphical models when the number of latent states is very large. The approach is based on iterative latent variable preselection, where we alternate between learning a selection function to reveal the relevant latent variables and using this to obtain a compact approximation of the posterior distribution for EM. This can make inference possible where the number of possible latent states is, for example, exponential in the number of latent variables, whereas an exact approach would be computationally infeasible. We learn the selection function entirely from the observed data and current expectation-maximization state via gaussian process regression. This is in contrast to earlier approaches, where selection functions were manually designed for each problem setting. We show that our approach performs as well as these bespoke selection functions on a wide variety of inference problems. In particular, for the challenging case of a hierarchical model for object localization with occlusion, we achieve results that match a customized state-of-the-art selection method at a far lower computational cost.

Autonomous Document Cleaning--A Generative Approach to Reconstruct Strongly Corrupted Scanned Texts

We study the task of cleaning scanned text documents that are strongly corrupted by dirt such as manual line strokes, spilled ink, etc. We aim at autonomously removing such corruptions from a single letter-size page based only on the information the page contains. Our approach first learns character representations from document patches without supervision. For learning, we use a probabilistic generative model parameterizing pattern features, their planar arrangements and their variances. The model's latent variables describe pattern position and class, and feature occurrences. Model parameters are efficiently inferred using a truncated variational EM approach. Based on the learned representation, a clean document can be recovered by identifying, for each patch, pattern class and position while a quality measure allows for discrimination between character and non-character patterns. For a full Latin alphabet we found that a single page does not contain sufficiently many character examples. However, even if heavily corrupted by dirt, we show that a page containing a lower number of character types can efficiently and autonomously be cleaned solely based on the structural regularity of the characters it contains. In different example applications with different alphabets, we demonstrate and discuss the effectiveness, efficiency and generality of the approach.

Spontaneous gamma coherence as a possible trait marker of schizophrenia-An explorative study

OBJECTIVES

Integration of sensory information by cortical network binding appears to be crucially involved in sensory processing activity. Studies in schizophrenia using functional neuroimaging, event-related potentials and EEG coherence indicate an impairment of cortical network coupling in this disorder. Previous electrophysiological investigations in animals and humans suggested that gamma activity (oscillations at around 30-100Hz) is essential for cortical network binding. This is the first investigation of spontaneous gamma activity in schizophrenics and their first degree relatives as compared to normal controls.

METHOD

Resting EEG was recorded in 20 drug naïve/drug free male schizophrenic patients, their pair matched male first degree relatives and 20 age-, sex-, education- and handedness-matched normal controls. Power spectrum and coherence of gamma band activity was analysed using MATLAB 6.5.

RESULTS

Schizophrenic patients had significantly less interhemispheric spontaneous gamma coherence and increased gamma power compared to normal controls. But there was no significant difference in gamma coherence between patients and their first degree relatives. Spontaneous gamma coherence had significant correlation with few PANSS subscale scores.

CONCLUSIONS

There is cortical hyperactivation and a reduced spontaneous and induced gamma coherence abnormality in schizophrenia. The abnormal gamma coherence appears explaining the psychopathology and poor performance on cognitive task in schizophrenia. This study has also generated hypotheses that the gamma band abnormality may be a trait abnormality in schizophrenics as seen by the similarity between the patient and their clinically asymptomatic first degree relatives.

Transformation invariant on-line target recognition

Transformation invariant automatic target recognition (ATR) has been an active research area due to its widespread applications in defense, robotics, medical imaging and geographic scene analysis. The primary goal for this paper is to obtain an on-line ATR system for targets in presence of image transformations, such as rotation, translation, scale and occlusion as well as resolution changes. We investigate biologically inspired adaptive critic design (ACD) neural network (NN) models for on-line learning of such transformations. We further exploit reinforcement learning (RL) in ACD framework to obtain transformation invariant ATR. We exploit two ACD designs, such as heuristic dynamic programming (HDP) and dual heuristic dynamic programming (DHP) to obtain transformation invariant ATR. We obtain extensive statistical evaluations of proposed on-line ATR networks using both simulated image transformations and real benchmark facial image database, UMIST, with pose variations. Our simulations show promising results for learning transformations in simulated images and authenticating out-of plane rotated face images. Comparing the two on-line ATR designs, HDP outperforms DHP in learning capability and robustness and is more tolerant to noise. The computational time involved in HDP is also less than that of DHP. On the other hand, DHP achieves a 100% success rate more frequently than HDP for individual targets, and the residual critic error in DHP is generally smaller than that of HDP. Mathematical analyses of both our RL-based on-line ATR designs are also obtained to provide a sufficient condition for asymptotic convergence in a statistical average sense.

Human gamma-band activity: a review on cognitive and behavioral correlates and network models

Gamma-band oscillations (roughly 30-100 Hz) in human and animal EEG have received considerable attention in the past due to their correlations with cognitive processes. Here, we want to sketch how some of the higher cognitive functions can be explained by memory processes which are known to modulate gamma activity. Especially, the function of binding together the multiple features of a perceived object requires a comparison with contents stored in memory. In addition, we review recent findings about the actual behavioral relevance of human gamma-band activity. Interestingly, rather simple models of spiking neurons are not only able to generate oscillatory activity within the gamma-band range, but even show modulations of these oscillations in line with findings from human experiments.

Cortext: a columnar model of bottom-up and top-down processing in the neocortex

Experimental data suggests that a first hypothesis about the content of a complex visual scene is available as early as 150 ms after stimulus presentation. Other evidence suggests that recognition in the visual cortex of mammals is a bidirectional, often top-down driven process. Here, we present a spiking neural network model that demonstrates how the cortex can use both strategies: Faced with a new stimulus, the cortex first tries to catch the gist of the scene. The gist is then fed back as global hypothesis to influence and redirect further bottom-up processing. We propose that these two modes of processing are carried out in different layers of the cortex. A cortical column may, thus, be primarily defined by the specific connectivity that links neurons in different layers into a functional circuit. Given an input, our model generates an initial hypothesis after only a few milliseconds. The first wave of action potentials traveling up the hierarchy activates representations of features and feature combinations. In most cases, the correct feature representation is activated strongest and precedes all other candidates with millisecond precision. Thus, our model codes the reliability of a response in the relative latency of spikes. In the subsequent refinement stage where high-level activity modulates lower stages, this activation dominance is propagated back, influencing its own afferent activity to establish a unique decision. Thus, top-down influence de-activates representations that have contributed to the initial hypothesis about the current stimulus, comparable to predictive coding. Features that do not match the top-down prediction trigger an error signal that can be the basis for learning new representations.

Cortical basis of communication: local computation, coordination, attention

Human communication emerges from cortical processing, known to be implemented on a regular repetitive neuronal substratum. The supposed genericity of cortical processing has elicited a series of modeling works in computational neuroscience that underline the information flows driven by the cortical circuitry. In the minimalist framework underlying the current theories for the embodiment of cognition, such a generic cortical processing is exploited for the coordination of poles of representation, as is reported in this paper for the case of visual attention. Interestingly, this case emphasizes how abstract internal referents are built to conform to memory requirements. This paper proposes that these referents are the basis for communication in humans, which is firstly a coordination and an attentional procedure with regard to their congeners.

Anticipation of natural stimuli modulates EEG dynamics: physiology and simulation

In everyday life we often encounter situations in which we can expect a visual stimulus before we actually see it. Here, we study the impact of such stimulus anticipation on the actual response to a visual stimulus. Participants were to indicate the sex of deer and cattle on photographs of the respective animals. On some trials, participants were cued on the species of the upcoming animal whereas on other trials this was not the case. Time frequency analysis of the simultaneously recorded EEG revealed modulations by this cue stimulus in two time windows. Early [Formula: see text] spectral responses [Formula: see text] displayed strongest stimulus-locking for stimuli that were preceded by a cue if they were sufficiently large. Late [Formula: see text] responses displayed enhanced amplitudes in response to large stimuli and to stimuli that were preceded by a cue. For late responses, however, no interaction between cue and stimulus size was observed. We were able to explain these results in a simulation by prestimulus gain modulations (early response) and by decreased response thresholds (late response). Thus, it seems plausible, that stimulus anticipation results in a pretuning of local neural populations.

Time pressure modulates electrophysiological correlates of early visual processing

BACKGROUND

Reactions to sensory events sometimes require quick responses whereas at other times they require a high degree of accuracy-usually resulting in slower responses. It is important to understand whether visual processing under different response speed requirements employs different neural mechanisms.

METHODOLOGY/PRINCIPAL FINDINGS

We asked participants to classify visual patterns with different levels of detail as real-world or non-sense objects. In one condition, participants were to respond immediately, whereas in the other they responded after a delay of 1 second. As expected, participants performed more accurately in delayed response trials. This effect was pronounced for stimuli with a high level of detail. These behavioral effects were accompanied by modulations of stimulus related EEG gamma oscillations which are an electrophysiological correlate of early visual processing. In trials requiring speeded responses, early stimulus-locked oscillations discriminated real-world and non-sense objects irrespective of the level of detail. For stimuli with a higher level of detail, oscillatory power in a later time window discriminated real-world and non-sense objects irrespective of response speed requirements.

CONCLUSIONS/SIGNIFICANCE

Thus, it seems plausible to assume that different response speed requirements trigger different dynamics of processing.

Modes of memory: early electrophysiological markers of repetition suppression and recognition enhancement predict behavioral performance

Different forms of perceptual memory have opposite physiological effects. Whereas repetition priming often leads to suppression of brain responses, explicit recognition has been found to enhance brain activity. We investigated effects of repetition priming and recognition memory on early gamma-band responses. In a study phase, participants performed a visual discrimination task with task-irrelevant item repetitions. Stimulus repetition suppressed early evoked gamma responses in participants with strong behavioral repetition effects. In a test phase, participants discriminated old from new items. Evoked and induced gamma activity was enhanced for old items. Effects were stronger in participants with better recognition performance. The results demonstrate a modulation of earliest stages of visual information processing by different memory systems, which is dependent on retrieval intention and predicts individual behavioral performance.

From perception to action: phase-locked gamma oscillations correlate with reaction times in a speeded response task

Background

Phase-locked gamma oscillations have so far mainly been described in relation to perceptual processes such as sensation, attention or memory matching. Due to its very short latency (≈90 ms) such oscillations are a plausible candidate for very rapid integration of sensory and motor processes.

Results

We measured EEG in 13 healthy participants in a speeded reaction task. Participants had to press a button as fast as possible whenever a visual stimulus was presented. The stimulus was always identical and did not have to be discriminated from other possible stimuli. In trials in which the participants showed a fast response, a slow negative potential over central electrodes starting approximately 800 ms before the response and highly phase-locked gamma oscillations over central and posterior electrodes between 90 and 140 ms after the stimulus were observed. In trials in which the participants showed a slow response, no slow negative potential was observed and phase-locked gamma oscillations were significantly reduced. Furthermore, for slow response trials the phase-locked gamma oscillations were significantly delayed with respect to fast response trials.

Conclusion

These results indicate the relevance of phase-locked gamma oscillations for very fast (not necessarily detailed) integration processes.

Induced gamma activity and event-related coherence in schizophrenia

Evidence has been provided that high frequency oscillations within the gamma band reflect mechanisms of cortical integration. In the light of recently proposed pathophysiological models of schizophrenia, suggesting a disturbance of the functional connectivity within distributed neural networks, it has been hypothesized that abnormalities in the gamma band underlie perceptual and cognitive dysfunctions in patients with schizophrenia. In the present study we investigated evoked and induced 40-Hz gamma power as well as frontoparietal and frontotemporal event-related coherence in patients with deficit and nondeficit schizophrenia and in matched healthy controls. In patients, correlations between gamma oscillations and psychopathological dimensions were also investigated. A reduction of both induced gamma power and event-related coherence was observed in patients with nondeficit schizophrenia, but not in those with deficit schizophrenia. Our findings support the hypothesis that deficit and nondeficit schizophrenia represent separate disease entities, suggesting the presence of a poor integration of the neuronal activity within distributed neural network only in the subgroup of schizophrenic patients without primary and persistent negative symptoms. Associations between an excess of gamma oscillations and psychopathological dimensions were observed, suggesting that abnormal thoughts, behaviors and perceptions might be related to the formation of inappropriate neural connections.

A Novel Action Selection Architecture in Soccer Simulation Environment Using Neuro-Fuzzy and Bidirectional Neural Networks

Multi-Agent systems have generated lots of excitement in recent years because of its promise as a new paradigm for conceptualizing, designing, and implementing software systems. One of the most important aspects of agent design in AI is the way agent acts or responds to the environment that the agent is acting upon. An effective action selection and behavioral method gives a powerful advantage in overall agent performance. We define a new method of action selection based on probability/priority models, we thereby introduce two efficient ways to determine probabilities using neuro-fuzzy systems and bidirectional neural networks and a new priority based system which maps the human knowledge to the action selection method. Furthermore, a behavior model is introduced to make the model more flexible.

Evoked γ oscillations in human scalp EEG are test–retest reliable

OBJECTIVE

Evoked, phase-locked gamma oscillations of the electroencephalogram (EEG) have been demonstrated to be modulated by both bottom-up as well as top-down factors. However, to date the test-retest reliability of these oscillations has not been studied systematically.

METHODS

We recorded EEG activity of 12 healthy volunteers in response to stimuli of different sizes. Each participant took part in two sessions separated by two weeks in time. To obtain an estimate of the reliability of evoked gamma band responses (GBRs), we compared frequency and magnitude of phase-locked EEG oscillations between sessions.

RESULTS

In response to large stimuli magnitude and frequency of the evoked GBR yielded significant reliability. However, this was not the case for stimuli which were too small to evoke detectable GBRs.

CONCLUSIONS

The results are in accordance with studies demonstrating a dependence of gamma oscillations on stimulus parameters.

SIGNIFICANCE

The current findings suggest that using appropriate stimulation, the evoked gamma response has sufficient test-retest reliability for use in assessing clinical changes in neurophysiological status.

Stable concurrent synchronization in dynamic system networks

In a network of dynamical systems, concurrent synchronization is a regime where multiple groups of fully synchronized elements coexist. In the brain, concurrent synchronization may occur at several scales, with multiple "rhythms" interacting and functional assemblies combining neural oscillators of many different types. Mathematically, stable concurrent synchronization corresponds to convergence to a flow-invariant linear subspace of the global state space. We derive a general condition for such convergence to occur globally and exponentially. We also show that, under mild conditions, global convergence to a concurrently synchronized regime is preserved under basic system combinations such as negative feedback or hierarchies, so that stable concurrently synchronized aggregates of arbitrary size can be constructed. Robustnesss of stable concurrent synchronization to variations in individual dynamics is also quantified. Simple applications of these results to classical questions in systems neuroscience and robotics are discussed.

A hierarchical model of operational anticipation windows in driving an automobile

Driving an automobile is an example of a goal-directed activity with high complexity in which different behavioral elements have to be integrated and brought into a sequential order. On the basis of the reafference principle and experimental results on temporal perception and cognitive control, we propose a hierarchical model of driving behavior, which can also be adapted to other goal-directed activities. Driving is conceived of as being controlled by anticipatory neuronal programs; if these programs are disrupted by unpredictable stimuli, which require an instantaneous reaction, behavioral control returns after completion of the reactive mode to the anticipatory mode of driving. In the model different levels of anticipation windows are distinguished which, however, are interconnected, in a bi-directional way: (a) Strategic level with a representation of the driving activity from the beginning to reaching the final goal; (b) Segmented tactical level with the sequence of necessary milestones to reach the goal; (c) Maneuver level where actions like passing another car or keeping a lane are controlled; (d) Short-term integration level of a few seconds which allows immediate anticipations; and (e) Synchronization level for sensorimotor control and complexity reduction within neuronal assemblies. A flow diagram schematically describes different driving situations stressing the anticipatory mode of control. In a pilot experiment with 20 subjects using a virtual driving situation in a car simulator predictions of the model could be verified, i.e., subjects showed a significant preference for the anticipatory mode of driving.

An oscillator model of the timing of turn-taking

When humans talk without conventionalized arrangements, they engage in conversation--that is, a continuous and largely nonsimultaneous exchange in which speakers take turns. Turn-taking is ubiquitous in conversation and is the normal case against which alternatives, such as interruptions, are treated as violations that warrant repair. Furthermore, turn-taking involves highly coordinated timing, including a cyclic rise and fall in the probability of initiating speech during brief silences, and involves the notable rarity, especially in two-party conversations, of two speakers' breaking a silence at once. These phenomena, reported by conversation analysts, have been neglected by cognitive psychologists, and to date there has been no adequate cognitive explanation. Here, we propose that, during conversation, endogenous oscillators in the brains of the speaker and the listeners become mutually entrained, on the basis of the speaker's rate of syllable production. This entrained cyclic pattern governs the potential for initiating speech at any given instant for the speaker and also for the listeners (as potential next speakers). Furthermore, the readiness functions of the listeners are counterphased with that of the speaker, minimizing the likelihood of simultaneous starts by a listener and the previous speaker. This mutual entrainment continues for a brief period when the speech stream ceases, accounting for the cyclic property of silences. This model not only captures the timing phenomena observed inthe literature on conversation analysis, but also converges with findings from the literatures on phoneme timing, syllable organization, and interpersonal coordination.

Time-frequency analysis of target detection reveals an early interface between bottom-up and top-down processes in the gamma-band

The early visual gamma-band response is an oscillatory signal evoked approximately 100 ms after stimulation. While some studies have found effects of various cognitive processes on this signal, such effects could not be replicated in other studies. Accordingly, some authors have claimed that evoked gamma-band activity reflects merely sensory functions. To resolve these conflicting positions, we conducted a target detection experiment in which the feature that defined the target could be distributed over a large or a small part of the entire stimulus. Only targets covering a larger area of the entire stimulus evoked stronger gamma-band activity than standards although the over-all stimulus size was identical for all stimuli. This increase in evoked activity resulted from stronger oscillatory power and not exclusively from stronger phase-locking. In contrast, N1 and P3 amplitudes were larger for target stimuli irrespective of the distribution of the relevant stimulus feature. These results are consistent with the notion that early gamma-band activity is generated by feature-selective neural assemblies the activity of which can in fact be modulated by top-down processes. This interaction, however, may be only detectable in scalp-recorded EEG if it affects a sufficient number of neural assemblies.

Bio-computational model of object-recognition: Quantum Hebbian processing with neurally shaped Gabor wavelets

Theoretical and simulational evidence, as well as experimental indications, are accumulating that quantum associative memory and imaging are possible. We compare these data with biological evidence, since we find them to a significant extent compatible. This paper presents a computationally implementable integrative model of appearance-based viewpoint-invariant recognition of objects. The neuro-quantum hybrid model incorporates neural processing up to V1 and quantum associative processing in V1, achieving together an object-recognition result in V2 and ITC. Results of our simulation of the central quantum-like parts of the bio-model, receiving neurally pre-processed inputs, are presented. This part contains our original simulated storage by multiple quantum interference of image-encoding Gabor wavelets done in a Hebbian way, especially using the Griniasty et al. pose-sequence learning rule.

Evolutionary optimization of a hierarchical object recognition model

A major problem in designing artificial neural networks is the proper choice of the network architecture. Especially for vision networks classifying three-dimensional (3-D) objects this problem is very challenging, as these networks are necessarily large and therefore the search space for defining the needed networks is of a very high dimensionality. This strongly increases the chances of obtaining only suboptimal structures from standard optimization algorithms. We tackle this problem in two ways. First, we use biologically inspired hierarchical vision models to narrow the space of possible architectures and to reduce the dimensionality of the search space. Second, we employ evolutionary optimization techniques to determine optimal features and nonlinearities of the visual hierarchy. Here, we especially focus on higher order complex features in higher hierarchical stages. We compare two different approaches to perform an evolutionary optimization of these features. In the first setting, we directly code the features into the genome. In the second setting, in analogy to an ontogenetical development process, we suggest the new method of an indirect coding of the features via an unsupervised learning process, which is embedded into the evolutionary optimization. In both cases the processing nonlinearities are encoded directly into the genome and are thus subject to optimization. The fitness of the individuals for the evolutionary selection process is computed by measuring the network classification performance on a benchmark image database. Here, we use a nearest-neighbor classification approach, based on the hierarchical feature output. We compare the found solutions with respect to their ability to generalize. We differentiate between a first- and a second-order generalization. The first-order generalization denotes how well the vision system, after evolutionary optimization of the features and nonlinearities using a database A, can classify previously unseen test views of objects from this database A. As second-order generalization, we denote the ability of the vision system to perform classification on a database B using the features and nonlinearities optimized on database A. We show that the direct feature coding approach leads to networks with a better first-order generalization, whereas the second-order generalization is on an equally high level for both direct and indirect coding. We also compare the second-order generalization results with other state-of-the-art recognition systems and show that both approaches lead to optimized recognition systems, which are highly competitive with recent recognition algorithms.

A neural network model of memory and higher cognitive functions

I first describe a neural network model of associative memory in a small region of the brain. The model depends, unconventionally, on disinhibition of inhibitory links between excitatory neurons rather than long-term potentiation (LTP) of excitatory projections. The model may be shown to have advantages over traditional neural network models both in terms of information storage capacity and biological plausibility. The learning and recall algorithms are independent of network architecture, and require no thresholds or finely graded synaptic strengths. Several copies of this local network are then connected by means of many, weak, reciprocal, excitatory projections that allow one region to control the recall of information in another to produce behaviors analogous to serial memory, classical and operant conditioning, secondary reinforcement, refabrication of memory, and fabrication of possible future events. The network distinguishes between perceived and recalled events, and can predicate its response on the absence as well as the presence of particular stimuli. Some of these behaviors are achieved in ways that seem to provide instances of self-awareness and imagination, suggesting that consciousness may emerge as an epiphenomenon in simple brains.

Hierarchical self-organization of minicolumnar receptive fields

We study self-organization of receptive fields (RFs) of cortical minicolumns. Input driven self-organization is induced by Hebbian synaptic plasticity of afferent fibers to model minicolumns based on spiking neurons and background oscillations. If input in the form of spike patterns is presented during learning, the RFs of minicolumns hierarchically specialize to increasingly small groups of similar RFs in a series of nested group subdivisions. In a number of experiments we show that the system finds clusters of similar spike patterns, that it is capable of evenly cover the input space if the input is continuously distributed, and that it extracts basic features from input consisting of superpositions of spike patterns. With a continuous version of the bars test we, furthermore, demonstrate the system's ability to evenly cover the space of extracted basic input features. The hierarchical nature and its flexibility with respect to input distinguishes the presented type of self-organization from others including similar but non-hierarchical self-organization as discussed in [Lucke J., & von der Malsburg, C. (2004). Rapid processing and unsupervised learning in a model of the cortical macrocolumn. Neural Computation 16, 501-533]. The capabilities of the presented system match crucial properties of the plasticity of cortical RFs and we suggest it as a model for their hierarchical formation.

Learning optimized features for hierarchical models of invariant object recognition

There is an ongoing debate over the capabilities of hierarchical neural feedforward architectures for performing real-world invariant object recognition. Although a variety of hierarchical models exists, appropriate supervised and unsupervised learning methods are still an issue of intense research. We propose a feedforward model for recognition that shares components like weight sharing, pooling stages, and competitive nonlinearities with earlier approaches but focuses on new methods for learning optimal feature-detecting cells in intermediate stages of the hierarchical network. We show that principles of sparse coding, which were previously mostly applied to the initial feature detection stages, can also be employed to obtain optimized intermediate complex features. We suggest a new approach to optimize the learning of sparse features under the constraints of a weight-sharing or convolutional architecture that uses pooling operations to achieve gradual invariance in the feature hierarchy. The approach explicitly enforces symmetry constraints like translation invariance on the feature set. This leads to a dimension reduction in the search space of optimal features and allows determining more efficiently the basis representatives, which achieve a sparse decomposition of the input. We analyze the quality of the learned feature representation by investigating the recognition performance of the resulting hierarchical network on object and face databases. We show that a hierarchy with features learned on a single object data set can also be applied to face recognition without parameter changes and is competitive with other recent machine learning recognition approaches. To investigate the effect of the interplay between sparse coding and processing nonlinearities, we also consider alternative feedforward pooling nonlinearities such as presynaptic maximum selection and sum-of-squares integration. The comparison shows that a combination of strong competitive nonlinearities with sparse coding offers the best recognition performance in the difficult scenario of segmentation-free recognition in cluttered surround. We demonstrate that for both learning and recognition, a precise segmentation of the objects is not necessary.

A hierarchical dynamical map as a basic frame for cortical mapping and its application to priming

A hierarchical dynamical map is proposed as the basic framework for sensory cortical mapping. To show how the hierarchical dynamical map works in cognitive processes, we applied it to a typical cognitive task known as priming, in which cognitive performance is facilitated as a consequence of prior experience. Prior to the priming task, the network memorizes a sensory scene containing multiple objects presented simultaneously using a hierarchical dynamical map. Each object is composed of different sensory features. The hierarchical dynamical map presented here is formed by random itinerancy among limit-cycle attractors into which these objects are encoded. Each limit-cycle attractor contains multiple point attractors into which elemental features belonging to the same object are encoded. When a feature stimulus is presented as a priming cue, the network state is changed from the itinerant state to a limit-cycle attractor relevant to the priming cue. After a short priming period, the network state reverts to the itinerant state. Under application of the test cue, consisting of some feature belonging to the object relevant to the priming cue and fragments of features belonging to others, the network state is changed to a limit-cycle attractor and finally to a point attractor relevant to the target feature. This process is considered as the identification of the target. The model consistently reproduces various observed results for priming processes such as the difference in identification time between cross-modality and within-modality priming tasks, the effect of interval between priming cue and test cue on identification time, the effect of priming duration on the time, and the effect of repetition of the same priming task on neural activity.

Ionic current and metabolism for brain scanners (a three state-model of modular activation)

The papers of Taylor and Korner [(1999). Neural Networks, 12, 943, 989] describe ambitious and realistic models of the computational platform of the brain. However, in order to correlate E/MEG and MR/PET data, we need additional equivalencies. In this context, we suggest the introduction of three states of the cortical modules.