A Novel Learning Algorithm to Optimize Deep Neural Networks: Evolved Gradient Direction Optimizer (EVGO)

Coevolutionary Neural Solution for Nonconvex Optimization With Noise Tolerance

The existing solutions for nonconvex optimization problems show satisfactory performance in noise-free scenarios. However, they are prone to yield inaccurate results in the presence of noise in real-world problems, which may lead to failures in optimizing nonconvex problems. To this end, in this article, we propose a coevolutionary neural solution (CNS) by combining a simplified neurodynamics (SND) model with the particle swarm optimization (PSO) algorithm. Specifically, the proposed SND model does not leverage the time-derivative information, exhibiting greater stability compared to existing models. Furthermore, due to the noise tolerance capacity and rapid convergence property exhibited by the SND model, the CNS can rapidly achieve the optimal solution even in the presence of various perturbations. Theoretical analyses ensure that the proposed CNS is globally convergent with robustness and probability. In addition, the effectiveness of the CNS is compared with those of the existing solutions by a class of illustrative examples. We further apply the proposed solution to design a finite impulse response (FIR) filter and a pressure vessel to demonstrate its performance.

Data-Knowledge-Driven Self-Organizing Fuzzy Neural Network

Fuzzy neural networks (FNNs) hold the advantages of knowledge leveraging and adaptive learning, which have been widely used in nonlinear system modeling. However, it is difficult for FNNs to obtain the appropriate structure in the situation of insufficient data, which limits its generalization performance. To solve this problem, a data-knowledge-driven self-organizing FNN (DK-SOFNN) with a structure compensation strategy and a parameter reinforcement mechanism is proposed in this article. First, a structure compensation strategy is proposed to mine structural information from empirical knowledge to learn the structure of DK-SOFNN. Then, a complete model structure can be acquired by sufficient structural information. Second, a parameter reinforcement mechanism is developed to determine the parameter evolution direction of DK-SOFNN that is most suitable for the current model structure. Then, a robust model can be obtained by the interaction between parameters and dynamic structure. Finally, the proposed DK-SOFNN is theoretically analyzed on the fixed structure case and dynamic structure case. Then, the convergence conditions can be obtained to guide practical applications. The merits of DK-SOFNN are demonstrated by some benchmark problems and industrial applications.

Efficient Thorax Disease Classification and Localization Using DCNN and Chest X-ray Images

Thorax disease is a life-threatening disease caused by bacterial infections that occur in the lungs. It could be deadly if not treated at the right time, so early diagnosis of thoracic diseases is vital. The suggested study can assist radiologists in more swiftly diagnosing thorax disorders and in the rapid airport screening of patients with a thorax disease, such as pneumonia. This paper focuses on automatically detecting and localizing thorax disease using chest X-ray images. It provides accurate detection and localization using DenseNet-121 which is foundation of our proposed framework, called Z-Net. The proposed framework utilizes the weighted cross-entropy loss function (W-CEL) that manages class imbalance issue in the ChestX-ray14 dataset, which helped in achieving the highest performance as compared to the previous models. The 112,120 images contained in the ChestX-ray14 dataset (60,412 images are normal, and the rest contain thorax diseases) were preprocessed and then trained for classification and localization. This work uses computer-aided diagnosis (CAD) system that supports development of highly accurate and precise computer-aided systems. We aim to develop a CAD system using a deep learning approach. Our quantitative results show high AUC scores in comparison with the latest research works. The proposed approach achieved the highest mean AUC score of 85.8%. This is the highest accuracy documented in the literature for any related model.

The global biogeography and environmental drivers of fairy circles

Fairy circles (FCs) are regular vegetation patterns found in drylands of Namibia and Western Australia. It is virtually unknown whether they are also present in other regions of the world and which environmental factors determine their distribution. We conducted a global systematic survey and found FC-like vegetation patterns in 263 sites from 15 countries and three continents, including the Sahel, Madagascar, and Middle-West Asia. FC-like vegetation patterns are found in environments characterized by a unique combination of soil (including low nutrient levels and high sand content) and climatic (arid regions with high temperatures and high precipitation seasonality) conditions. In addition to these factors, the presence of specific biological elements (termite nests) in certain regions also plays a role in the presence of these patterns. Furthermore, areas with FC-like vegetation patterns also showed more stable temporal productivity patterns than those of surrounding areas. Our study presents a global atlas of FCs and provides unique insights into the ecology and biogeography of these fascinating vegetation patterns.

Multiple Mittag-Leffler Stability of Fractional-Order Complex-Valued Memristive Neural Networks With Delays

This article discusses the coexistence and dynamical behaviors of multiple equilibrium points (Eps) for fractional-order complex-valued memristive neural networks (FCVMNNs) with delays. First, based on the state space partition method, some sufficient conditions are proposed to guarantee that there are multiple Eps in one FCVMNN. Then, the Mittag-Leffler stability of those multiple Eps is proved by using the Lyapunov function. Simultaneously, the enlarged attraction basins are obtained to improve and extend the existing theoretical results in the previous literature. In addition, some existing stability results in the literature are special cases of a new result herein. Finally, two illustrative examples with computer simulations are presented to verify the effectiveness of theoretical analysis.

Forecasting for regulatory credit loss derived from the COVID-19 pandemic: A machine learning approach

The economic onslaught of the COVID-19 pandemic has compromised the risk management of financial institutions. The consequences related to such an unprecedented situation are difficult to foresee with certainty using traditional methods. The regulatory credit loss attached to defaulted mortgages, so-called expected loss best estimate (ELBE), is forecasted using a machine learning technique. The projection of two ELBEs for 2022 and their comparison are presented. One accounts for the outbreak's impact, and the other presumes the nonexistence of the pandemic. Then, it is concluded that the referred crisis surely adversely affects said high-risk portfolios. The proposed method has excellent performance and may serve to estimate future expected and unexpected losses amidst any event of extraordinary magnitude.

Epoch-Evolving Gaussian Process Guided Learning for Classification

The conventional mini-batch gradient descent algorithms are usually trapped in the local batch-level distribution information, resulting in the ``zig-zag'' effect in the learning process. To characterize the correlation information between the batch-level distribution and the global data distribution, we propose a novel learning scheme called epoch-evolving Gaussian process guided learning (GPGL) to encode the global data distribution information in a non-parametric way. Upon a set of class-aware anchor samples, our GP model is built to estimate the class distribution for each sample in mini-batch through label propagation from the anchor samples to the batch samples. The class distribution, also named the context label, is provided as a complement for the ground-truth one-hot label. Such a class distribution structure has a smooth property and usually carries a rich body of contextual information that is capable of speeding up the convergence process. With the guidance of the context label and ground-truth label, the GPGL scheme provides a more efficient optimization through updating the model parameters with a triangle consistency loss. Furthermore, our GPGL scheme can be generalized and naturally applied to the current deep models, outperforming the state-of-the-art optimization methods on six benchmark datasets.