Training Winner-Take-All Simultaneous Recurrent Neural Networks

Admiring the Great Mountain: A Celebration Special Issue in Honor of Stephen Grossberg's 80th Birthday

This editorial summarizes selected key contributions of Prof. Stephen Grossberg and describes the papers in this 80th birthday special issue in his honor. His productivity, creativity, and vision would each be enough to mark a scientist of the first caliber. In combination, they have resulted in contributions that have changed the entire discipline of neural networks. Grossberg has been tremendously influential in engineering, dynamical systems, and artificial intelligence as well. Indeed, he has been one of the most important mentors and role models in my career, and has done so with extraordinary generosity and encouragement. All authors in this special issue have taken great pleasure in hereby commemorating his extraordinary career and contributions.

Design of a K-Winners-Take-All Model With a Binary Spike Train

A continuous-time K -winners-take-all (KWTA) neural model that can identify the largest K of N inputs, where command signal is described. The model is given by a differential equation where the spike train is a sum of delta functions. A functional block-diagram of the model includes N feed-forward hard-limiting neurons and one feedback neuron, used to handle input dynamics. The existence and uniqueness of the model steady states are analyzed, the convergence analysis of the state variable trajectories to the KWTA operation is proven, the convergence time and number of spikes required are derived, as well as the processing of time-varying inputs and perturbations of the model nonlinearities are analyzed. The main advantage of the model is that it is not subject to the intrinsic convergence of speed limitations of comparable designs. The model also has an arbitrary finite resolution determined by a given parameter, low complexity, and initial condition independence. Applications of the model for parallel sorting and parallel rank-order filtering are presented. Theoretical results are derived and illustrated with computer-simulated examples that demonstrate the model's performance.

Quantum inspired PSO for the optimization of simultaneous recurrent neural networks as MIMO learning systems

Training a single simultaneous recurrent neural network (SRN) to learn all outputs of a multiple-input-multiple-output (MIMO) system is a difficult problem. A new training algorithm developed from combined concepts of swarm intelligence and quantum principles is presented. The training algorithm is called particle swarm optimization with quantum infusion (PSO-QI). To improve the effectiveness of learning, a two-step learning approach is introduced in the training. The objective of the learning in the first step is to find the optimal set of weights in the SRN considering all output errors. In the second step, the objective is to maximize the learning of each output dynamics by fine tuning the respective SRN output weights. To demonstrate the effectiveness of the PSO-QI training algorithm and the two-step learning approach, two examples of an SRN learning MIMO systems are presented. The first example is learning a benchmark MIMO system and the second one is the design of a wide area monitoring system for a multimachine power system. From the results, it is observed that SRNs can effectively learn MIMO systems when trained using the PSO-QI algorithm and the two-step learning approach.