New accurate and flexible design procedure for a stable KWTA continuous time network

Effect of Time-Varying Multiplicative Noise on DNN-kWTA Model

Among many -winners-take-all ( WTA) models, the dual-neural network (DNN- WTA) model is with significantly less number of connections. However, for analog realization, noise is inevitable and affects the operational correctness of the WTA process. Most existing results focus on the effect of additive noise. This brief studies the effect of time-varying multiplicative input noise. Two scenarios are considered. The first one is the bounded noise case, in which only the noise range is known. Another one is for the general noise distribution case, in which we either know the noise distribution or have noise samples. For each scenario, we first prove the convergence property of the DNN- WTA model under multiplicative input noise and then provide an efficient method to determine whether a noise-affected DNN- WTA network performs the correct WTA process for a given set of inputs. With the two methods, we can efficiently measure the probability of the network performing the correct WTA process. In addition, for the case of the inputs being uniformly distributed, we derive two closed-form expressions, one for each scenario, for estimating the probability of the model having correct operation. Finally, we conduct simulations to verify our theoretical results.

Influence of Imperfections on the Operational Correctness of DNN-kWTA Model

The dual neural network (DNN)-based k -winner-take-all (WTA) model is able to identify the k largest numbers from its m input numbers. When there are imperfections, such as non-ideal step function and Gaussian input noise, in the realization, the model may not output the correct result. This brief analyzes the influence of the imperfections on the operational correctness of the model. Due to the imperfections, it is not efficient to use the original DNN- k WTA dynamics for analyzing the influence. In this regard, this brief first derives an equivalent model to describe the dynamics of the model under the imperfections. From the equivalent model, we derive a sufficient condition for which the model outputs the correct result. Thus, we apply the sufficient condition to design an efficiently estimation method for the probability of the model outputting the correct result. Furthermore, for the inputs with uniform distribution, a closed form expression for the probability value is derived. Finally, we extend our analysis for handling non-Gaussian input noise. Simulation results are provided to validate our theoretical results.

Design of a K-Winners-Take-All Model With a Binary Spike Train

A continuous-time K -winners-take-all (KWTA) neural model that can identify the largest K of N inputs, where command signal is described. The model is given by a differential equation where the spike train is a sum of delta functions. A functional block-diagram of the model includes N feed-forward hard-limiting neurons and one feedback neuron, used to handle input dynamics. The existence and uniqueness of the model steady states are analyzed, the convergence analysis of the state variable trajectories to the KWTA operation is proven, the convergence time and number of spikes required are derived, as well as the processing of time-varying inputs and perturbations of the model nonlinearities are analyzed. The main advantage of the model is that it is not subject to the intrinsic convergence of speed limitations of comparable designs. The model also has an arbitrary finite resolution determined by a given parameter, low complexity, and initial condition independence. Applications of the model for parallel sorting and parallel rank-order filtering are presented. Theoretical results are derived and illustrated with computer-simulated examples that demonstrate the model's performance.

Robustness Analysis on Dual Neural Network-based $k$ WTA With Input Noise

This paper studies the effects of uniform input noise and Gaussian input noise on the dual neural network-based WTA (DNN- WTA) model. We show that the state of the network (under either uniform input noise or Gaussian input noise) converges to one of the equilibrium points. We then derive a formula to check if the network produce correct outputs or not. Furthermore, for the uniformly distributed inputs, two lower bounds (one for each type of input noise) on the probability that the network produces the correct outputs are presented. Besides, when the minimum separation amongst inputs is given, we derive the condition for the network producing the correct outputs. Finally, experimental results are presented to verify our theoretical results. Since random drift in the comparators can be considered as input noise, our results can be applied to the random drift situation.

A Consistent Model for Lazzaro Winner-Take-All Circuit With Invariant Subthreshold Behavior

This paper considers the basic Lazzaro winner-take-all analog computing circuit with N current inputs, N voltage outputs, and a bias current. Motivated by low-power applications, we find a mathematical accurate model of the network when all MOS devices remain inside the subthreshold region all along the transient. This involves analytical inequalities relating the range of admissible input currents to transistor parameters, to supply voltage, and to bias current. The restrictions are sufficiently weak to allow extra demands of functionality or performance. The technical novelty here is that by a slight cut of the maximal subthreshold domain and by choosing proper coordinates, we get a ordinary differential equation invariance problem on a rectangle of ℜ^N+1 space which is analytically tractable. A more precise localization of the asymptotically stable steady state inside the region of interest is also inferred. Although the work is mainly theoretical, an effort to infer simple, interpretable formulas useful for synthesis has been made. The results are numerically verified and their feasibility is discussed in detail. The subject matter is new and it is worth extending it to larger classes of circuits with regional behavior.

Properties and Performance of Imperfect Dual Neural Network-Based kWTA Networks

The dual neural network (DNN)-based k -winner-take-all ( k WTA) model is an effective approach for finding the k largest inputs from n inputs. Its major assumption is that the threshold logic units (TLUs) can be implemented in a perfect way. However, when differential bipolar pairs are used for implementing TLUs, the transfer function of TLUs is a logistic function. This brief studies the properties of the DNN- kWTA model under this imperfect situation. We prove that, given any initial state, the network settles down at the unique equilibrium point. Besides, the energy function of the model is revealed. Based on the energy function, we propose an efficient method to study the model performance when the inputs are with continuous distribution functions. Furthermore, for uniformly distributed inputs, we derive a formula to estimate the probability that the model produces the correct outputs. Finally, for the case that the minimum separation ∆min of the inputs is given, we prove that if the gain of the activation function is greater than 1/4∆min max(ln 2n, 2 ln 1 - ϵ/ϵ ), then the network can produce the correct outputs with winner outputs greater than 1-ϵ and loser outputs less than ϵ, where ϵ is the threshold less than 0.5.