The time dimension for scene analysis

Fusing Attention Features and Contextual Information for Scene Recognition

Aiming to obtain more discriminative features in scene images and overcome the impacts of intra-class differences and inter-class similarities, the paper proposes a scene recognition method that combines attention and context information. First, we introduce the attention mechanism and build a multi-scale attention model. Discriminative information considers salient objects and regions by means of channel attention and spatial attention. Besides, the central loss function joint supervision strategy is introduced to further reduce the misjudgment of intra-class differences. Second, a model based on multi-level context information is proposed to describe the positional relationship between objects, which can effectively alleviate the influence of the similarity of objects between classes. Finally, the two models are merged to give full play to the compatibility of features, so that the final feature representation not only focuses on the effective discriminant information, but also manifests the relative position relationship between significant objects. Extensive experiments have proved that the method in this paper effectively solves the problem of insufficient feature representation in scene recognition tasks, and improves the accuracy of scene recognition.

Spiking model of fixational eye movements and figure-ground segmentation

We present a model connecting eye movements and cortical state. Its structure includes simulated retinal images, motion detection, feature detectors and layers of spiking neurons. The designed scheme shows how the effect of micro-saccadic scale eye movements can lead to successful figure segregation in a figure-ground paradigm, by inducing changes in the neural dynamics through the time evolution of the inhibition range.

The role of network structure and time delay in a metapopulation Wilson--Cowan model

We study the dynamics of a network Wilson--Cowan model (a system of connected Wilson--Cowan oscillators) for interacting excitatory and inhibitory neuron populations with time delays. Each node in this model corresponds to a population of neurons, including excitatory and inhibitory subpopulations, and hence it can be viewed as a metapopulation model. It is known that information transfer within each cortical area is not instantaneous, and therefore we consider a system of delay differential equations with two different kinds of discrete delays. We account for the time delay in information propagation between individual excitatory and inhibitory subpopulations at each node via intra-node time delays, and we account for time delay in information propagation between neuron populations at different nodes with inter-node time delays. The biologically relevant resting state solutions are oscillatory (stable limit cycles). After determining the influence of the coupling parameters between nodes, the intra-node delays, and the inter-node delays on the dynamics of the two coupled Wilson--Cowan oscillators, we then explore a variety of larger networks of 16 and 100 nodes, in order to determine how the network topology will influence time delayed Wilson--Cowan dynamics. We find that network structure can regularize or deregularize the dynamics, with networks of higher mean degree permitting stable limit cycles and networks with smaller mean degree yielding less regular dynamics (which may range from chaotic solutions, to solutions for which limit cycles collapse into steady states, which are biologically undesirable compared with the preferred stable limit cycles). Furthermore, heterogeneity in the degree distribution of the network (resulting from networks with nodes of varying degree) can result in asynchronous dynamics, even if at each node the local dynamics are that of a limit cycle, in contrast to the synchronization of dynamics between nodes seen when the degree of all nodes is equal. This suggests that homogeneous and well-connected networks permit robust limit cycles under time-delayed Wilson--Cowan dynamics, whereas heterogeneous or poorly connected networks may fail to provide such desirable dynamics, a phenomena akin to structural loss of neuron connections in neurodegenerative diseases.

Neural dynamics of spreading attentional labels in mental contour tracing

Behavioral and neural data suggest that visual attention spreads along contour segments to bind them into a unified object representation. Such attentional labeling segregates the target contour from distractors in a process known as mental contour tracing. A recurrent competitive map is developed to simulate the dynamics of mental contour tracing. In the model, local excitation opposes global inhibition and enables enhanced activity to propagate on the path offered by the contour. The extent of local excitatory interactions is modulated by the output of the multi-scale contour detection network, which constrains the speed of activity spreading in a scale-dependent manner. Furthermore, an L-junction detection network enables tracing to switch direction at the L-junctions, but not at the X- or T-junctions, thereby preventing spillover to a distractor contour. Computer simulations reveal that the model exhibits a monotonic increase in tracing time as a function of the distance to be traced. Also, the speed of tracing increases with decreasing proximity to the distractor contour and with the reduced curvature of the contours. The proposed model demonstrated how an elaborated version of the winner-takes-all network can implement a complex cognitive operation such as contour tracing.

Supervised Speech Separation Based on Deep Learning: An Overview

Speech separation is the task of separating target speech from background interference. Traditionally, speech separation is studied as a signal processing problem. A more recent approach formulates speech separation as a supervised learning problem, where the discriminative patterns of speech, speakers, and background noise are learned from training data. Over the past decade, many supervised separation algorithms have been put forward. In particular, the recent introduction of deep learning to supervised speech separation has dramatically accelerated progress and boosted separation performance. This paper provides a comprehensive overview of the research on deep learning based supervised speech separation in the last several years. We first introduce the background of speech separation and the formulation of supervised separation. Then, we discuss three main components of supervised separation: learning machines, training targets, and acoustic features. Much of the overview is on separation algorithms where we review monaural methods, including speech enhancement (speech-nonspeech separation), speaker separation (multitalker separation), and speech dereverberation, as well as multimicrophone techniques. The important issue of generalization, unique to supervised learning, is discussed. This overview provides a historical perspective on how advances are made. In addition, we discuss a number of conceptual issues, including what constitutes the target source.

A Novel Fractional-Order Chaotic Phase Synchronization Model for Visual Selection and Shifting

Visual information processing is one of the fields of cognitive informatics. In this paper, a two-layer fractional-order chaotic network, which can simulate the mechanism of visual selection and shifting, is established. Unlike other object selection models, the proposed model introduces control units to select object. The first chaotic network layer of the model is used to implement image segmentation. A control layer is added as the second layer, consisting of a central neuron, which controls object selection and shifting. To implement visual selection and shifting, a strategy is proposed that can achieve different subnets corresponding to the objects in the first layer synchronizing with the central neuron at different time. The central unit acting as the central nervous system synchronizes with different subnets (hybrid systems), implementing the mechanism of visual selection and shifting in the human system. The proposed model corresponds better with the human visual system than the typical model of visual information encoding and transmission and provides new possibilities for further analysis of the mechanisms of the human cognitive system. The reasonability of the proposed model is verified by experiments using artificial and natural images.

Global oscillation regime change by gated inhibition

The role of sensory inputs in the modelling of synchrony regimes is exhibited by means of networks of spiking cells where the relative strength of the inhibitory interaction is controlled by the activation of a linear unit working as a gating variable. Adaptation to stimulus size is determined by the value of a changing length scale, modelled by the time-varying radius of a circular receptive field. In this set-up, 'consolidation' time intervals relevant to attentional effects are shown to depend on the dynamics governing the evolution of the introduced length scale.

Feature-Linking Model for Image Enhancement

Inspired by gamma-band oscillations and other neurobiological discoveries, neural networks research shifts the emphasis toward temporal coding, which uses explicit times at which spikes occur as an essential dimension in neural representations. We present a feature-linking model (FLM) that uses the timing of spikes to encode information. The first spiking time of FLM is applied to image enhancement, and the processing mechanisms are consistent with the human visual system. The enhancement algorithm achieves boosting the details while preserving the information of the input image. Experiments are conducted to demonstrate the effectiveness of the proposed method. Results show that the proposed method is effective.

Scene recognition by manifold regularized deep learning architecture

Scene recognition is an important problem in the field of computer vision, because it helps to narrow the gap between the computer and the human beings on scene understanding. Semantic modeling is a popular technique used to fill the semantic gap in scene recognition. However, most of the semantic modeling approaches learn shallow, one-layer representations for scene recognition, while ignoring the structural information related between images, often resulting in poor performance. Modeled after our own human visual system, as it is intended to inherit humanlike judgment, a manifold regularized deep architecture is proposed for scene recognition. The proposed deep architecture exploits the structural information of the data, making for a mapping between visible layer and hidden layer. By the proposed approach, a deep architecture could be designed to learn the high-level features for scene recognition in an unsupervised fashion. Experiments on standard data sets show that our method outperforms the state-of-the-art used for scene recognition.

Phase synchronization of coupled bursting neurons and the generalized Kuramoto model

Bursting neurons fire rapid sequences of action potential spikes followed by a quiescent period. The basic dynamical mechanism of bursting is the slow currents that modulate a fast spiking activity caused by rapid ionic currents. Minimal models of bursting neurons must include both effects. We considered one of these models and its relation with a generalized Kuramoto model, thanks to the definition of a geometrical phase for bursting and a corresponding frequency. We considered neuronal networks with different connection topologies and investigated the transition from a non-synchronized to a partially phase-synchronized state as the coupling strength is varied. The numerically determined critical coupling strength value for this transition to occur is compared with theoretical results valid for the generalized Kuramoto model.

Approximate emergent synchrony in spatially coupled spiking neurons with discrete interaction

Models for perceptual grouping and contour integration are presented. Connection weights depend on distances and angle differences, while neurons evolve following a spiking dynamics (Izhikevich's model in most of the considered cases). Although the studied synapses depend on discrete three-valued functions, simulations display the emergence of approximate synchrony, making these cognitive tasks possible. Noise effects are examined, and the possibility of achieving similar results with a different neuron model is discussed.

A MULTI-LAYER NEURAL-MASS MODEL FOR LEARNING SEQUENCES USING THETA/GAMMA OSCILLATIONS

A neural mass model for the memorization of sequences is presented. It exploits three layers of cortical columns that generate a theta/gamma rhythm. The first layer implements an auto-associative memory working in the theta range; the second segments objects in the gamma range; finally, the feedback interactions between the third and the second layers realize a hetero-associative memory for learning a sequence. After training with Hebbian and anti-Hebbian rules, the network recovers sequences and accounts for the phase-precession phenomenon.

A stereovision matching strategy for images captured with fish-eye lenses in forest environments

We present a novel strategy for computing disparity maps from hemispherical stereo images obtained with fish-eye lenses in forest environments. At a first segmentation stage, the method identifies textures of interest to be either matched or discarded. This is achieved by applying a pattern recognition strategy based on the combination of two classifiers: Fuzzy Clustering and Bayesian. At a second stage, a stereovision matching process is performed based on the application of four stereovision matching constraints: epipolar, similarity, uniqueness and smoothness. The epipolar constraint guides the process. The similarity and uniqueness are mapped through a decision making strategy based on a weighted fuzzy similarity approach, obtaining a disparity map. This map is later filtered through the Hopfield Neural Network framework by considering the smoothness constraint. The combination of the segmentation and stereovision matching approaches makes the main contribution. The method is compared against the usage of simple features and combined similarity matching strategies.

Visual grouping by neural oscillator networks

Distributed synchronization is known to occur at several scales in the brain, and has been suggested as playing a key functional role in perceptual grouping. State-of-the-art visual grouping algorithms, however, seem to give comparatively little attention to neural synchronization analogies. Based on the framework of concurrent synchronization of dynamical systems, simple networks of neural oscillators coupled with diffusive connections are proposed to solve visual grouping problems. The key idea is to embed the desired grouping properties in the choice of the diffusive couplings, so that synchronization of oscillators within each group indicates perceptual grouping of the underlying stimulative atoms, while desynchronization between groups corresponds to group segregation. Compared with state-of-the-art approaches, the same algorithm is shown to achieve promising results on several classical visual grouping problems, including point clustering, contour integration, and image segmentation.

New spiking cortical model for invariant texture retrieval and image processing

Based on the studies of existing local-connected neural network models, in this brief, we present a new spiking cortical neural networks model and find that time matrix of the model can be recognized as a human subjective sense of stimulus intensity. The series of output pulse images of a proposed model represents the segment, edge, and texture features of the original image, and can be calculated based on several efficient measures and forms a sequence as the feature of the original image. We characterize texture images by the sequence for an invariant texture retrieval. The experimental results show that the retrieval scheme is effective in extracting the rotation and scale invariant features. The new model can also obtain good results when it is used in other image processing applications.

A neural model of selective attention and object segmentation in the visual scene: an approach based on partial synchronization and star-like architecture of connections

A brain-inspired computational system is presented that allows sequential selection and processing of objects from a visual scene. The system is comprised of three modules. The selective attention module is designed as a network of spiking neurons of the Hodgkin-Huxley type with star-like connections between the central unit and peripheral elements. The attention focus is represented by those peripheral neurons that generate spikes synchronously with the central neuron while the activity of other peripheral neurons is suppressed. Such dynamics corresponds to the partial synchronization mode. It is shown that peripheral neurons with higher firing rates are preferentially drawn into partial synchronization. We show that local excitatory connections facilitate synchronization, while local inhibitory connections help distinguishing between two groups of peripheral neurons with similar intrinsic frequencies. The module automatically scans a visual scene and sequentially selects regions of interest for detailed processing and object segmentation. The contour extraction module implements standard image processing algorithms for contour extraction. The module computes raw contours of objects accompanied by noise and some spurious inclusions. At the next stage, the object segmentation module designed as a network of phase oscillators is used for precise determination of object boundaries and noise suppression. This module has a star-like architecture of connections. The segmented object is represented by a group of peripheral oscillators working in the regime of partial synchronization with the central oscillator. The functioning of each module is illustrated by an example of processing of the visual scene taken from a visual stream of a robot camera.

Chaotic phase synchronization and desynchronization in an oscillator network for object selection

Object selection refers to the mechanism of extracting objects of interest while ignoring other objects and background in a given visual scene. It is a fundamental issue for many computer vision and image analysis techniques and it is still a challenging task to artificial visual systems. Chaotic phase synchronization takes place in cases involving almost identical dynamical systems and it means that the phase difference between the systems is kept bounded over the time, while their amplitudes remain chaotic and may be uncorrelated. Instead of complete synchronization, phase synchronization is believed to be a mechanism for neural integration in brain. In this paper, an object selection model is proposed. Oscillators in the network representing the salient object in a given scene are phase synchronized, while no phase synchronization occurs for background objects. In this way, the salient object can be extracted. In this model, a shift mechanism is also introduced to change attention from one object to another. Computer simulations show that the model produces some results similar to those observed in natural vision systems.

Distributed processing and temporal codes in neuronal networks

The cerebral cortex presents itself as a distributed dynamical system with the characteristics of a small world network. The neuronal correlates of cognitive and executive processes often appear to consist of the coordinated activity of large assemblies of widely distributed neurons. These features require mechanisms for the selective routing of signals across densely interconnected networks, the flexible and context dependent binding of neuronal groups into functionally coherent assemblies and the task and attention dependent integration of subsystems. In order to implement these mechanisms, it is proposed that neuronal responses should convey two orthogonal messages in parallel. They should indicate (1) the presence of the feature to which they are tuned and (2) with which other neurons (specific target cells or members of a coherent assembly) they are communicating. The first message is encoded in the discharge frequency of the neurons (rate code) and it is proposed that the second message is contained in the precise timing relationships between individual spikes of distributed neurons (temporal code). It is further proposed that these precise timing relations are established either by the timing of external events (stimulus locking) or by internal timing mechanisms. The latter are assumed to consist of an oscillatory modulation of neuronal responses in different frequency bands that cover a broad frequency range from <2 Hz (delta) to >40 Hz (gamma) and ripples. These oscillations limit the communication of cells to short temporal windows whereby the duration of these windows decreases with oscillation frequency. Thus, by varying the phase relationship between oscillating groups, networks of functionally cooperating neurons can be flexibly configurated within hard wired networks. Moreover, by synchronizing the spikes emitted by neuronal populations, the saliency of their responses can be enhanced due to the coincidence sensitivity of receiving neurons in very much the same way as can be achieved by increasing the discharge rate. Experimental evidence will be reviewed in support of the coexistence of rate and temporal codes. Evidence will also be provided that disturbances of temporal coding mechanisms are likely to be one of the pathophysiological mechanisms in schizophrenia.

IMORL: incremental multiple-object recognition and localization

This paper proposes an incremental multiple-object recognition and localization (IMORL) method. The objective of IMORL is to adaptively learn multiple interesting objects in an image. Unlike the conventional multiple-object learning algorithms, the proposed method can automatically and adaptively learn from continuous video streams over the entire learning life. This kind of incremental learning capability enables the proposed approach to accumulate experience and use such knowledge to benefit future learning and the decision making process. Furthermore, IMORL can effectively handle variations in the number of instances in each data chunk over the learning life. Another important aspect analyzed in this paper is the concept drifting issue. In multiple-object learning scenarios, it is a common phenomenon that new interesting objects may be introduced during the learning life. To handle this situation, IMORL uses an adaptive learning principle to autonomously adjust to such new information. The proposed approach is independent of the base learning models, such as decision tree, neural networks, support vector machines, and others, which provide the flexibility of using this method as a general learning methodology in multiple-object learning scenarios. In this paper, we use a neural network with a multilayer perceptron (MLP) structure as the base learning model and test the performance of this method in various video stream data sets. Simulation results show the effectiveness of this method.

Image identification system based on an optical broadcast neural network and a pulse coupled neural network preprocessor stage

We describe the concept of a vision system based on an optoelectronic hardware neural processor. The proposed system is composed of a pulse coupled neural network (PCNN) preprocessor stage that converts an input image into a temporal pulsed pattern. These pulses are inputs to the optical broadcast neural network (OBNN) processor, which classifies the input pattern between a set of reference patterns based on a pattern matching strategy. The PCNN is to provide immunity to the scale, rotation, and translation of objects in the image. The OBNN provides high parallelism and a high speed hardware neural processor.

Unsupervised segmentation with dynamical units

In this paper, we present a novel network to separate mixtures of inputs that have been previously learned. A significant capability of the network is that it segments the components of each input object that most contribute to its classification. The network consists of amplitude-phase units that can synchronize their dynamics, so that separation is determined by the amplitude of units in an output layer, and segmentation by phase similarity between input and output layer units. Learning is unsupervised and based on a Hebbian update, and the architecture is very simple. Moreover, efficient segmentation can be achieved even when there is considerable superposition of the inputs. The network dynamics are derived from an objective function that rewards sparse coding in the generalized amplitude-phase variables. We argue that this objective function can provide a possible formal interpretation of the binding problem and that the implementation of the network architecture and dynamics is biologically plausible.

An oscillatory correlation model of auditory streaming

We present a neurocomputational model for auditory streaming, which is a prominent phenomenon of auditory scene analysis. The proposed model represents auditory scene analysis by oscillatory correlation, where a perceptual stream corresponds to a synchronized assembly of neural oscillators and different streams correspond to desynchronized oscillator assemblies. The underlying neural architecture is a two-dimensional network of relaxation oscillators with lateral excitation and global inhibition, where one dimension represents time and another dimension frequency. By employing dynamic connections along the frequency dimension and a random element in global inhibition, the proposed model produces a temporal coherence boundary and a fissure boundary that closely match those from the psychophysical data of auditory streaming. Several issues are discussed, including how to represent physical time and how to relate shifting synchronization to auditory attention.

A Hopfield Neural Network for Image Change Detection

This paper outlines an optimization relaxation approach based on the analog Hopfield neural network (HNN) for solving the image change detection problem between two images. A difference image is obtained by subtracting pixel by pixel both images. The network topology is built so that each pixel in the difference image is a node in the network. Each node is characterized by its state, which determines if a pixel has changed. An energy function is derived, so that the network converges to stable states. The analog Hopfield's model allows each node to take on analog state values. Unlike most widely used approaches, where binary labels (changed/unchanged) are assigned to each pixel, the analog property provides the strength of the change. The main contribution of this paper is reflected in the customization of the analog Hopfield neural network to derive an automatic image change detection approach. When a pixel is being processed, some existing image change detection procedures consider only interpixel relations on its neighborhood. The main drawback of such approaches is the labeling of this pixel as changed or unchanged according to the information supplied by its neighbors, where its own information is ignored. The Hopfield model overcomes this drawback and for each pixel allows a tradeoff between the influence of its neighborhood and its own criterion. This is mapped under the energy function to be minimized. The performance of the proposed method is illustrated by comparative analysis against some existing image change detection methods.