Echo State Networks With Orthogonal Pigeon-Inspired Optimization for Image Restoration

Event-Driven Video Restoration With Spiking-Convolutional Architecture

With high temporal resolution, high dynamic range, and low latency, event cameras have made great progress in numerous low-level vision tasks. To help restore low-quality (LQ) video sequences, most existing event-based methods usually employ convolutional neural networks (CNNs) to extract sparse event features without considering the spatial sparse distribution or the temporal relation in neighboring events. It brings about insufficient use of spatial and temporal information from events. To address this problem, we propose a new spiking-convolutional network (SC-Net) architecture to facilitate event-driven video restoration. Specifically, to properly extract the rich temporal information contained in the event data, we utilize a spiking neural network (SNN) to suit the sparse characteristics of events and capture temporal correlation in neighboring regions; to make full use of spatial consistency between events and frames, we adopt CNNs to transform sparse events as an extra brightness prior to being aware of detailed textures in video sequences. In this way, both the temporal correlation in neighboring events and the mutual spatial information between the two types of features are fully explored and exploited to accurately restore detailed textures and sharp edges. The effectiveness of the proposed network is validated in three representative video restoration tasks: deblurring, super-resolution, and deraining. Extensive experiments on synthetic and real-world benchmarks have illuminated that our method performs better than existing competing methods.

Nature-Inspired Intelligent Computing: A Comprehensive Survey

Nature, with its numerous surprising rules, serves as a rich source of creativity for the development of artificial intelligence, inspiring researchers to create several nature-inspired intelligent computing paradigms based on natural mechanisms. Over the past decades, these paradigms have revealed effective and flexible solutions to practical and complex problems. This paper summarizes the natural mechanisms of diverse advanced nature-inspired intelligent computing paradigms, which provide valuable lessons for building general-purpose machines capable of adapting to the environment autonomously. According to the natural mechanisms, we classify nature-inspired intelligent computing paradigms into 4 types: evolutionary-based, biological-based, social-cultural-based, and science-based. Moreover, this paper also illustrates the interrelationship between these paradigms and natural mechanisms, as well as their real-world applications, offering a comprehensive algorithmic foundation for mitigating unreasonable metaphors. Finally, based on the detailed analysis of natural mechanisms, the challenges of current nature-inspired paradigms and promising future research directions are presented.

Evolving Dual-Threshold Bienenstock-Cooper-Munro Learning Rules in Echo State Networks

The strengthening and the weakening of synaptic strength in existing Bienenstock-Cooper-Munro (BCM) learning rule are determined by a long-term potentiation (LTP) sliding modification threshold and the afferent synaptic activities. However, synaptic long-term depression (LTD) even affects low-active synapses during the induction of synaptic plasticity, which may lead to information loss. Biological experiments have found another LTD threshold that can induce either potentiation or depression or no change, even at the activated synapses. In addition, existing BCM learning rules can only select a set of fixed rule parameters, which is biologically implausible and practically inflexible to learn the structural information of input signals. In this article, an evolved dual-threshold BCM learning rule is proposed to regulate the reservoir internal connection weights of the echo-state-network (ESN), which can contribute to alleviating information loss and enhancing learning performance by introducing different optimal LTD thresholds for different postsynaptic neurons. Our experimental results show that the evolved dual-threshold BCM learning rule can result in the synergistic learning of different plasticity rules, effectively improving the learning performance of an ESN in comparison with existing neural plasticity learning rules and some state-of-the-art ESN variants on three widely used benchmark tasks and the prediction of an esterification process.

Computational Modeling of Structural Synaptic Plasticity in Echo State Networks

Most existing studies on computational modeling of neural plasticity have focused on synaptic plasticity. However, regulation of the internal weights in the reservoir based on synaptic plasticity often results in unstable learning dynamics. In this article, a structural synaptic plasticity learning rule is proposed to train the weights and add or remove neurons within the reservoir, which is shown to be able to alleviate the instability of the synaptic plasticity, and to contribute to increase the memory capacity of the network as well. Our experimental results also reveal that a few stronger connections may last for a longer period of time in a constantly changing network structure, and are relatively resistant to decay or disruptions in the learning process. These results are consistent with the evidence observed in biological systems. Finally, we show that an echo state network (ESN) using the proposed structural plasticity rule outperforms an ESN using synaptic plasticity and three state-of-the-art ESNs on four benchmark tasks.

Fault Detection of Wind Turbine Gearboxes Based on IBOA-ERF

As one of the key components of wind turbines, gearboxes are under complex alternating loads for a long time, and the safety and reliability of the whole machine are often affected by the failure of internal gears and bearings. Aiming at the difficulty of optimizing the parameters of wind turbine gearbox fault detection models based on extreme random forest, a fault detection model with extreme random forest optimized by the improved butterfly optimization algorithm (IBOA-ERF) is proposed. The algebraic sum of the false alarm rate and the missing alarm rate of the fault detection model is constructed as the fitness function, and the initial position and position update strategy of the individual are improved. A chaotic mapping strategy is introduced to replace the original population initialization method to enhance the randomness of the initial population distribution. An adaptive inertia weight factor is proposed, combined with the landmark operator of the pigeon swarm optimization algorithm to update the population position iteration equation to speed up the convergence speed and improve the diversity and robustness of the butterfly optimization algorithm. The dynamic switching method of local and global search stages is adopted to achieve dynamic balance between global exploration and local search, and to avoid falling into local optima. The ERF fault detection model is trained, and the improved butterfly optimization algorithm is used to obtain optimal parameters to achieve fast response of the proposed model with good robustness and generalization under high-dimensional data. The experimental results show that, compared with other optimization algorithms, the proposed fault detection method of wind turbine gearboxes has a lower false alarm rate and missing alarm rate.

PID Controller-Guided Attention Neural Network Learning for Fast and Effective Real Photographs Denoising

Real photograph denoising is extremely challenging in low-level computer vision since the noise is sophisticated and cannot be fully modeled by explicit distributions. Although deep-learning techniques have been actively explored for this issue and achieved convincing results, most of the networks may cause vanishing or exploding gradients, and usually entail more time and memory to obtain a remarkable performance. This article overcomes these challenges and presents a novel network, namely, PID controller guide attention neural network (PAN-Net), taking advantage of both the proportional-integral-derivative (PID) controller and attention neural network for real photograph denoising. First, a PID-attention network (PID-AN) is built to learn and exploit discriminative image features. Meanwhile, we devise a dynamic learning scheme by linking the neural network and control action, which significantly improves the robustness and adaptability of PID-AN. Second, we explore both the residual structure and share-source skip connections to stack the PID-ANs. Such a framework provides a flexible way to feature residual learning, enabling us to facilitate the network training and boost the denoising performance. Extensive experiments show that our PAN-Net achieves superior denoising results against the state-of-the-art in terms of image quality and efficiency.

An internet traffic classification method based on echo state network and improved salp swarm algorithm

Internet traffic classification is fundamental to network monitoring, service quality and security. In this paper, we propose an internet traffic classification method based on the Echo State Network (ESN). To enhance the identification performance, we improve the Salp Swarm Algorithm (SSA) to optimize the ESN. At first, Tent mapping with reversal learning, polynomial operator and dynamic mutation strategy are introduced to improve the SSA, which enhances its optimization performance. Then, the advanced SSA are utilized to optimize the hyperparameters of the ESN, including the size of the reservoir, sparse degree, spectral radius and input scale. Finally, the optimized ESN is adopted to classify Internet traffic. The simulation results show that the proposed ESN-based method performs much better than other traditional machine learning algorithms in terms of per-class metrics and overall accuracy.

Collaborative Target Search Algorithm for UAV Based on Chaotic Disturbance Pigeon-Inspired Optimization

Unmanned aerial vehicles (UAVs) have shown their superiority in military and civilian missions. In the face of complex tasks, many UAVs are usually needed to cooperate with each other. Therefore, multi-UAV cooperative target search has attracted more and more scholars’ attention. At present, there are many bionic algorithms for solving the cooperative search problem of multi-UAVs, including particle swarm optimization algorithm (PSO) and differential evolution (DE). Pigeon-inspired optimization (PIO) is a new swarm intelligence optimization algorithm proposed in recent years. It has great advantages over other algorithms in convergence, robustness, and accuracy, and has few parameters to be adjusted. Aiming at the shortcomings of the standard pigeon colony algorithm, such as poor population diversity, slow convergence speed, and the ease of falling into local optimum, we have proposed chaotic disturbance pigeon-inspired optimization (CDPIO) algorithm. The improved tent chaotic map was used to initialize the population and increase the diversity of the population. The disturbance factor is introduced in the iterative update stage of the algorithm to generate new individuals, replace the individuals with poor performance, and carry out disturbance to increase the optimization accuracy. Benchmark functions and UAV target search model were used to test the algorithm performance. The results show that the CDPIO had faster convergence speed, better optimization precision, better robustness, and better performance than PIO.

Tridirectional Transfer Learning for Predicting Gastric Cancer Morbidity

Our previous study has constructed a deep learning model for predicting gastrointestinal infection morbidity based on environmental pollutant indicators in some regions in central China. This article aims to adapt the prediction model for three purposes: 1) predicting the morbidity of a different disease in the same region; 2) predicting the morbidity of the same disease in a different region; and 3) predicting the morbidity of a different disease in a different region. We propose a tridirectional transfer learning approach, which achieves the abovementioned three purposes by: 1) developing a combined univariate regression and multivariate Gaussian model for establishing the relationship between the morbidity of the target disease and that of the source disease together with the high-level pollutant features in the current source region; 2) using mapping-based deep transfer learning to extend the current model to predict the morbidity of the source disease in both source and target regions; and 3) applying the pattern of the combined model in the source region to the extended model to derive a new combined model for predicting the morbidity of the target disease in the target region. We select gastric cancer as the target disease and use the proposed transfer learning approach to predict its morbidity in the source region and three target regions. The results show that, given only a limited number of labeled samples, our approach achieves an average prediction accuracy of over 80% in the source region and up to 78% in the target regions, which can contribute considerably to improving medical preparedness and response.

A Comprehensive Review on Nature Inspired Neural Network based Adaptive Filter for Eliminating Noise in Medical Images

BACKGROUND

Various kind of medical imaging modalities are available for providing noninvasive view and for analyzing any pathological symptoms of human beings. Different noise may appear in those modalities at the time of acquisition, transmission, scanning, or at the time of storing. The removal of noises from the digital medical images without losing any inherent features is always considered a challenging task because a successful diagnosis relies on them. Numerous techniques have been proposed to fulfill this objective, and each having their own benefits and limitations.

DISCUSSION

In this comprehensive review article, more than 65 research articles are investigated to illustrate the applications of Artificial Neural Networks (ANN) in the field of biomedical image denoising. In particular, the zest of this article is to highlight the hybridized filtering model using nature-inspired algorithms and artificial neural networks for suppression of noise. Various other techniques, such as fixed filter, linear adaptive filters and gradient descent learning based neural network filter are also included.

CONCLUSION

This article envisages how to train ANN using derivative free nature-inspired algorithms, and its performance in various medical images modalities and noise conditions.

Evolving Local Plasticity Rules for Synergistic Learning in Echo State Networks

Existing synaptic plasticity rules for optimizing the connections between neurons within the reservoir of echo state networks (ESNs) remain to be global in that the same type of plasticity rule with the same parameters is applied to all neurons. However, this is biologically implausible and practically inflexible for learning the structures in the input signals, thereby limiting the learning performance of ESNs. In this paper, we propose to use local plasticity rules that allow different neurons to use different types of plasticity rules and different parameters, which are achieved by optimizing the parameters of the local plasticity rules using the evolution strategy (ES) with covariance matrix adaptation (CMA-ES). We show that evolving neural plasticity will result in a synergistic learning of different plasticity rules, which plays an important role in improving the learning performance. Meanwhile, we show that the local plasticity rules can effectively alleviate synaptic interferences in learning the structure in sensory inputs. The proposed local plasticity rules are compared with a number of the state-of-the-art ESN models and the canonical ESN using a global plasticity rule on a set of widely used prediction and classification benchmark problems to demonstrate its competitive learning performance.

A Novel Pigeon-Inspired Optimization Based MPPT Technique for PV Systems

The conventional maximum power point tracking (MPPT) method fails in partially shaded conditions, because multiple peaks may appear on the power–voltage characteristic curve. The Pigeon-Inspired Optimization (PIO) algorithm is a new type of meta-heuristic algorithm. Aiming at this situation, this paper proposes a new type of algorithm that combines a new pigeon population algorithm named Parallel and Compact Pigeon-Inspired Optimization (PCPIO) with MPPT, which can solve the problem that MPPT cannot reach the near global maximum power point. This hybrid algorithm is fast, stable, and capable of globally optimizing the maximum power point tracking algorithm. Therefore, the purpose of this article is to study the performance of two optimization techniques. The two algorithms are Particle Swarm Algorithm (PSO) and improved pigeon algorithm. This paper first studies the mechanism of multi-peak output characteristics of photovoltaic arrays in complex environments, and then proposes a multi-peak MPPT algorithm based on a combination of an improved pigeon population algorithm and an incremental conductivity method. The improved pigeon algorithm is used to quickly locate near the maximum power point, and then the variable step size incremental method INC (incremental conductance) is used to accurately locate the maximum power point. A simulation was performed on Matlab/Simulink platform. The results prove that the method can achieve fast and accurate optimization under complex environmental conditions, effectively reduce power oscillations, enhance system stability, and achieve better control results.

A Compact Pigeon-Inspired Optimization for Maximum Short-Term Generation Mode in Cascade Hydroelectric Power Station

Pigeon-inspired optimization (PIO) is a new type of intelligent algorithm. It is proposed that the algorithm simulates the movement of pigeons going home. In this paper, a new pigeon herding algorithm called compact pigeon-inspired optimization (CPIO) is proposed. The challenging task for multiple algorithms is not only combining operations, but also constraining existing devices. The proposed algorithm aims to solve complex scientific and industrial problems with many data packets, including the use of classical optimization problems and the ability to find optimal solutions in many solution spaces with limited hardware resources. A real-valued prototype vector performs probability and statistical calculations, and then generates optimal candidate solutions for CPIO optimization algorithms. The CPIO algorithm was used to evaluate a variety of continuous multi-model functions and the largest model of hydropower short-term generation. The experimental results show that the proposed algorithm is a more effective way to produce competitive results in the case of limited memory devices.

An Improved Pigeon-Inspired Optimisation Algorithm and Its Application in Parameter Inversion

Pre-stack amplitude variation with offset (AVO) elastic parameter inversion is a nonlinear, multi-solution optimisation problem. The techniques that combine intelligent optimisation algorithms and AVO inversion provide an effective identification method for oil and gas exploration. However, these techniques also have shortcomings in solving nonlinear geophysical inversion problems. The evolutionary optimisation algorithms have recognised disadvantages, such as the tendency of convergence to a local optimum resulting in poor local optimisation performance when dealing with multimodal search problems, decreasing diversity and leading to the prematurity of the population as the number of evolutionary iterations increases. The pre-stack AVO elastic parameter inversion is nonlinear with slow convergence, while the pigeon-inspired optimisation (PIO) algorithm has the advantage of fast convergence and better optimisation characteristics. In this study, based on the characteristics of the pre-stack AVO elastic parameter inversion problem, an improved PIO algorithm (IPIO) is proposed by introducing the particle swarm optimisation (PSO) algorithm, an inverse factor, and a Gaussian factor into the PIO algorithm. The experimental comparisons indicate that the proposed IPIO algorithm can achieve better inversion results.

Linear Regression Supporting Vector Machine and Hybrid LOG Filter-Based Image Restoration

The image restoration (IR) technique is a part of image processing to improve the quality of an image that is affected by noise and blur. Thus, IR is required to attain a better quality of image. In this paper, IR is performed using linear regression-based support vector machine (LR-SVM). This LR-SVM has two steps: training and testing. The training and testing stages have a distinct windowing process for extracting blocks from the images. The LR-SVM is trained through a block-by-block training sequence. The extracted block-by-block values of images are used to enhance the classification process of IR. In training, the imperfections on the image are easily identified by setting the target vectors as the original images. Then, the noisy image is given at LR-SVM testing, based on the original image restored from the dictionary. Finally, the image block from the testing stage is enhanced using the hybrid Laplacian of Gaussian (HLOG) filter. The denoising of the HLOG filter provides enhanced results by using block-by-block values. This proposed approach is named as LR-SVM-HLOG. A dataset used in this LR-SVM-HLOG method is the Berkeley Segmentation Database. The performance of LR-SVM-HLOG was analyzed as peak signal-to-noise ratio (PSNR) and structural similarity index. The PSNR values of the house and pepper image (color image) are 40.82 and 36.56 dB, respectively, which are higher compared to the inter- and intra-block sparse estimation method and block matching and three-dimensional filtering for color images at 20% noise.

Online sequential echo state network with sparse RLS algorithm for time series prediction

Recently, the echo state networks (ESNs) have been widely used for time series prediction. To meet the demand of actual applications and avoid the overfitting issue, the online sequential ESN with sparse recursive least squares (OSESN-SRLS) algorithm is proposed. Firstly, the ℓ₀ and ℓ₁ norm sparsity penalty constraints of output weights are separately employed to control the network size. Secondly, the sparse recursive least squares (SRLS) algorithm and the subgradients technique are combined to estimate the output weight matrix. Thirdly, an adaptive selection mechanism for the ℓ₀ or ℓ₁ norm regularization parameter is designed. With the selected regularization parameter, it is proved that the developed SRLS shows comparable or better performance than the regular RLS. Furthermore, the convergence of OSESN-SRLS is theoretically analyzed to guarantee its effectiveness. Simulation results illustrate that the proposed OSESN-SRLS always outperforms other existing ESNs in terms of estimation accuracy and network compactness.

Early fault feature extraction of bearings based on Teager energy operator and optimal VMD

As the fault shock component in vibration signals is extremely sparse and weak, it is difficult to extract the fault features when large-scale, low-speed and heavy-duty mechanical equipment is in the early stage of failure. To solve this problem, an early fault feature extraction method based on the Teager energy operator, combined with optimal variational mode decomposition (VMD) is presented in this study. First, the Teager energy operator was used to strengthen the weak shock component of the original signal. Next, a logistic-sine complex chaotic mapping with variable dimensions was constructed to enhance the global search ability and convergence speed of the pigeon-inspired optimization (PIO) algorithm, which is named the variable dimension chaotic pigeon-inspired optimization (VDCPIO) algorithm. Then, the VDCPIO algorithm is used to search for the optimal combination value of key parameters of VMD. The enhanced vibration signal is decomposed into a set of intrinsic mode functions (IMFs) by the optimized VMD, and then kurtosis for every IMF and mean kurtosis of all IMFs are extracted. According to the average kurtosis, several IMFs, whose kurtosis value is greater than the average kurtosis value, are selected to reconstruct a new signal. Then, envelope spectrum analysis of the reconstructed signal is carried out to extract the early fault features. Finally, experimental verification of the method was performed using the simulated signal and measured signal from a rolling bearing; the experimental results indicate that the method presented in this paper is more effective to extract the early fault features of this kind of mechanical equipment.

Coevolution Pigeon-Inspired Optimization with Cooperation-Competition Mechanism for Multi-UAV Cooperative Region Search

In this paper, a dynamic two-stage closed search (DTSCS) scheme for the unmanned aerial vehicle (UAV) cooperative region search is designed, which satisfies the range constraint (RC) and orientation constraint (OC). The closed trajectory is composed of two coupling stages, the search stage and the return stage. The position and orientation at the end of the search stage are the starting cell and orientation of the return stage. In the first stage, a coevolution pigeon-inspired optimization (CPIO) algorithm based on the cooperation-competition mechanism is proposed for multi-UAV cooperative search. In the return stage, inspired by region searching and trajectory tracking, a search tracking (ST) approach is presented to obtain the lowest-cost path under OC. The simulation results show that: (i) N p = 5 is the best prediction time step. (ii) CPIO algorithm performs better than the compared intelligent algorithms in region searching. (iii) ST has high tracking performance than other algorithms. (iv) The DTSCS scheme enables every UAV to make the best use of its fuel to cover more region and return to the airport within the RC, and the average range utilization of UAVs is 97% under the 3OC.