Activated Gradients for Deep Neural Networks

Precise classification of traditional Chinese medicine sources using intelligent fusion of hyperspectral imaging-mass spectrometry data combined with machine learning: A case study of American ginseng

The application of artificial intelligence in traditional Chinese medicine (TCM) has become a hot topic in the scientific community. American ginseng (AG), a perennial herb with a rich history, is widely utilized in clinical settings due to its diverse pharmacological activities and nutritional value. However, the quality of AG in the market is often compromised by the presence of similar-looking adulterants from different regions. Rapid and precise identification of its origin is crucial for consumers. This study proposes a novel approach, employing a Mid-Level-Fusion method that combines hyperspectral imaging (HSI) and ultra performance liquid chromatography-quadrupole linear ion trap mass spectrometry (UPLC-QTRAP-MS/MS) techniques to successfully identify origins of AG. Firstly, the 1D-Gradient-weighted class activation mapping (1D-GradCAM) algorithm was utilized for feature selection on HSI data, visualizing wavelengths contributing significantly to classification results and using the 1D-GradCAM algorithm, the spectral features were reduced from 510 to 91, achieving 105 % of the performance of the full-wavelength model. Simultaneously, redundant data in UPLC-QTRAP-MS/MS were eliminated using Cars-PLS, reducing the number of indicator components from 23 to 11. Subsequently, a Mid-Level Fusion matrix was generated based on the filtered HSI and UPLC-QTRAP-MS/MS data to establish an AG origin tracing model, achieving a detection accuracy of up to 96.15 %. Finally, the established HSI-UPLC-QTRAP-MS/MS-Mid-Level-Fusion model enabled pixel-level recognition of origin tracing. In conclusion, HSI combined with UPLC-QTRAP-MS/MS Mid-Level-Fusion presents a feasible method for tracing AG origins, playing a crucial role in quality control at the source in TCM production.

Leveraging advanced graph neural networks for the enhanced classification of post anesthesia states to aid surgical procedures

Anesthesia plays a pivotal role in modern surgery by facilitating controlled states of unconsciousness. Precise control is crucial for safe and pain-free surgeries. Monitoring anesthesia depth accurately is essential to guide anesthesiologists, optimize drug usage, and mitigate postoperative complications. This study focuses on enhancing the classification performance of anesthesia-induced transitions between wakefulness and deep sleep into eight classes by leveraging advanced graph neural network (GNN). The research combines seven datasets into a single dataset comprising 290 samples and investigates key brain regions, to develop a robust classification framework. Initially, the dataset is augmented using the Synthetic Minority Over-sampling Technique (SMOTE) to expand the sample size to 1197. A graph-based approach is employed to get the intricate relationships between features, constructing a graph dataset with 1197 nodes and 714,610 edges, where nodes represent data samples and edges are the connections between the nodes. The connection (edge weight) is calculated using Spearman correlation coefficient matrix. An optimized GNN model is developed through an ablation study of eight hyperparameters, achieving an accuracy of 92.8%. The model's performance is further evaluated against one-dimensional (1D) CNN, and six machine learning models, demonstrating superior classification capabilities for small and imbalanced datasets. Additionally, we evaluated the proposed model on six different anesthesia datasets, observing no decline in performance. This work advances the understanding and classification of anesthesia states, providing a valuable tool for improved anesthesia management.

Influence of Optimization Algorithms and Computational Complexity on Concrete Compressive Strength Prediction Machine Learning Models for Concrete Mix Design

The proper design of concrete mixtures is a critical task in concrete technology, where optimal strength, eco-friendliness, and production efficiency are increasingly demanded. While traditional analytical methods, such as the Three Equations Method, offer foundational approaches to mix design, they often fall short in handling the complexity of modern concrete technology. Machine learning-based models have demonstrated notable efficacy in predicting concrete compressive strength, addressing the limitations of conventional methods. This study builds on previous research by investigating not only the impact of computational complexity on the predictive performance of machine learning models but also the influence of different optimization algorithms. The study evaluates the effectiveness of three optimization techniques: the Quasi-Newton Method (QNM), the Adaptive Moment Estimation (ADAM) algorithm, and Stochastic Gradient Descent (SGD). A total of forty-five deep neural network models of varying computational complexity were trained and tested using a comprehensive database of concrete mix designs and their corresponding compressive strength test results. The findings reveal a significant interaction between optimization algorithms and model complexity in enhancing prediction accuracy. Models utilizing the QNM algorithm outperformed those using the ADAM and SGD in terms of error reduction (SSE, MSE, RMSE, NSE, and ME) and increased coefficient of determination (R2). These insights contribute to the development of more accurate and efficient AI-driven methods in concrete mix design, promoting the advancement of concrete technology and the potential for future research in this domain.

An Accelerated Approach on Adaptive Gradient Neural Network for Solving Time-Dependent Linear Equations: A State-Triggered Perspective

To improve the acceleration performance, a hybrid state-triggered discretization (HSTD) is proposed for the adaptive gradient neural network (AGNN) for solving time-dependent linear equations (TDLEs). Unlike the existing approaches that use an activation function or a time-varying coefficient for acceleration, the proposed HSTD is uniquely designed from a control theory perspective. It comprises two essential components: adaptive sampling interval state-triggered discretization (ASISTD) and adaptive coefficient state-triggered discretization (ACSTD). The former addresses the gap in acceleration methods related to the variable sampling period, while the latter considers the underlying evolutionary dynamics of the Lyapunov function to determine coefficients greedily. Finally, compared with commonly used discretization methods, the acceleration performance and computational advantages of the proposed HSTD are substantiated by the numerical simulations and applications to robotics.

A Comprehensive Survey of Deep Learning Approaches in Image Processing

The integration of deep learning (DL) into image processing has driven transformative advancements, enabling capabilities far beyond the reach of traditional methodologies. This survey offers an in-depth exploration of the DL approaches that have redefined image processing, tracing their evolution from early innovations to the latest state-of-the-art developments. It also analyzes the progression of architectural designs and learning paradigms that have significantly enhanced the ability to process and interpret complex visual data. Key advancements, such as techniques improving model efficiency, generalization, and robustness, are examined, showcasing DL's ability to address increasingly sophisticated image-processing tasks across diverse domains. Metrics used for rigorous model evaluation are also discussed, underscoring the importance of performance assessment in varied application contexts. The impact of DL in image processing is highlighted through its ability to tackle complex challenges and generate actionable insights. Finally, this survey identifies potential future directions, including the integration of emerging technologies like quantum computing and neuromorphic architectures for enhanced efficiency and federated learning for privacy-preserving training. Additionally, it highlights the potential of combining DL with emerging technologies such as edge computing and explainable artificial intelligence (AI) to address scalability and interpretability challenges. These advancements are positioned to further extend the capabilities and applications of DL, driving innovation in image processing.

Structural prior-driven feature extraction with gradient-momentum combined optimization for convolutional neural network image classification

Recent image classification efforts have achieved certain success by incorporating prior information such as labels and logical rules to learn discriminative features. However, these methods overlook the variability of features, resulting in feature inconsistency and fluctuations in model parameter updates, which further contribute to decreased image classification accuracy and model instability. To address this issue, this paper proposes a novel method combining structural prior-driven feature extraction with gradient-momentum (SPGM), from the perspectives of consistent feature learning and precise parameter updates, to enhance the accuracy and stability of image classification. Specifically, SPGM leverages a structural prior-driven feature extraction (SPFE) approach to calculate gradients of multi-level features and original images to construct structural information, which is then transformed into prior knowledge to drive the network to learn features consistent with the original images. Additionally, an optimization strategy integrating gradients and momentum (GMO) is introduced, dynamically adjusting the direction and step size of parameter updates based on the angle and norm of the sum of gradients and momentum, enabling precise model parameter updates. Extensive experiments on CIFAR10 and CIFAR100 datasets demonstrate that the SPGM method significantly reduces the top-1 error rate in image classification, enhances the classification performance, and outperforms state-of-the-art methods.

Pixel-Level Recognition of Trace Mycotoxins in Red Ginseng Based on Hyperspectral Imaging Combined with 1DCNN-Residual-BiLSTM-Attention Model

Red ginseng is widely used in food and pharmaceuticals due to its significant nutritional value. However, during the processing and storage of red ginseng, it is susceptible to grow mold and produce mycotoxins, generating security issues. This study proposes a novel approach using hyperspectral imaging technology and a 1D-convolutional neural network-residual-bidirectional-long short-term memory attention mechanism (1DCNN-ResBiLSTM-Attention) for pixel-level mycotoxin recognition in red ginseng. The "Red Ginseng-Mycotoxin" (R-M) dataset is established, and optimal parameters for 1D-CNN, residual bidirectional long short-term memory (ResBiLSTM), and 1DCNN-ResBiLSTM-Attention models are determined. The models achieved testing accuracies of 98.75%, 99.03%, and 99.17%, respectively. To simulate real detection scenarios with potential interfering impurities during the sampling process, a "Red Ginseng-Mycotoxin-Interfering Impurities" (R-M-I) dataset was created. The testing accuracy of the 1DCNN-ResBiLSTM-Attention model reached 96.39%, and it successfully predicted pixel-wise classification for other unknown samples. This study introduces a novel method for real-time mycotoxin monitoring in traditional Chinese medicine, with important implications for the on-site quality control of herbal materials.

Deep learning-based facial analysis for predicting difficult videolaryngoscopy: a feasibility study

While videolaryngoscopy has resulted in better overall success rates of tracheal intubation, airway assessment is still an important prerequisite for safe airway management. This study aimed to create an artificial intelligence model to identify difficult videolaryngoscopy using a neural network. Baseline characteristics, medical history, bedside examination and seven facial images were included as predictor variables. ResNet-18 was introduced to recognise images and extract features. Different machine learning algorithms were utilised to develop predictive models. A videolaryngoscopy view of Cormack-Lehane grade of 1 or 2 was classified as 'non-difficult', while grade 3 or 4 was classified as 'difficult'. A total of 5849 patients were included, of whom 5335 had non-difficult and 514 had difficult videolaryngoscopy. The facial model (only including facial images) using the Light Gradient Boosting Machine algorithm showed the highest area under the curve (95%CI) of 0.779 (0.733-0.825) with a sensitivity (95%CI) of 0.757 (0.650-0.845) and specificity (95%CI) of 0.721 (0.626-0.794) in the test set. Compared with bedside examination and multivariate scores (El-Ganzouri and Wilson), the facial model had significantly higher predictive performance (p < 0.001). Artificial intelligence-based facial analysis is a feasible technique for predicting difficulty during videolaryngoscopy, and the model developed using neural networks has higher predictive performance than traditional methods.

Prediction of fruit characteristics of grafted plants of Camellia oleifera by deep neural networks

BACKGROUND

Camellia oleifera, an essential woody oil tree in China, propagates through grafting. However, in production, it has been found that the interaction between rootstocks and scions may affect fruit characteristics. Therefore, it is necessary to predict fruit characteristics after grafting to identify suitable rootstock types.

METHODS

This study used Deep Neural Network (DNN) methods to analyze the impact of 106 6-year-old grafting combinations on the characteristics of C.oleifera, including fruit and seed characteristics, and fatty acids. The prediction of characteristics changes after grafting was explored to provide technical support for the cultivation and screening of specialized rootstocks. After determining the unsaturated fat acids, palmitoleic acid C16:1, cis-11 eicosenoic acid C20:1, oleic acid C18:1, linoleic acid C18:2, linolenic acid C18:3, kernel oil content, fruit height, fruit diameter, fresh fruit weight, pericarp thickness, fresh seed weight, and the number of fresh seeds, the DNN method was used to calculate and analyze the model. The model was screened using the comprehensive evaluation index of Mean Absolute Error (MAPE), determinate correlation R2 and and time consumption.

RESULTS

When using 36 neurons in 3 hidden layers, the deep neural network model had a MAPE of less than or equal to 16.39% on the verification set and less than or equal to 13.40% on the test set. Compared with traditional machine learning methods such as support vector machines and random forests, the DNN method demonstrated more accurate predictions for fruit phenotypic characteristics, with MAPE improvement rates of 7.27 and 3.28 for the 12 characteristics on the test set and maximum R2 improvement values of 0.19 and 0.33. In conclusion, the DNN method developed in this study can effectively predict the oil content and fruit phenotypic characteristics of C. oleifera, providing a valuable tool for predicting the impact of grafting combinations on the fruit of C. oleifera.

A Pseudoinversion-Free Method for Weight Updating in Broad Learning System

Neural networks have evolved into one of the most critical tools in the field of artificial intelligence. As a kind of shallow feedforward neural network, the broad learning system (BLS) uses a training process based on random and pseudoinverse methods, and it does not need to go through a complete training cycle to obtain new parameters when adding nodes. Instead, it performs rapid update iterations on the basis of existing parameters through a series of dynamic update algorithms, which enables BLS to combine high efficiency and accuracy flexibly. The training strategy of BLS is completely different from the existing mainstream neural network training strategy based on the gradient descent algorithm, and the superiority of the former has been proven in many experiments. This article applies an ingenious method of pseudoinversion to the weight updating process in BLS and employs it as an alternative strategy for the dynamic update algorithms in the original BLS. Theoretical analyses and numerical experiments demonstrate the efficiency and effectiveness of BLS aided with this method. The research presented in this article can be regarded as an extended study of the BLS theory, providing an innovative idea and direction for future research on BLS.

Computational Complexity and Its Influence on Predictive Capabilities of Machine Learning Models for Concrete Mix Design

The design of concrete mixtures is crucial in concrete technology, aiming to produce concrete that meets specific quality and performance criteria. Modern standards require not only strength but also eco-friendliness and production efficiency. Based on the Three Equation Method, conventional mix design methods involve analytical and laboratory procedures but are insufficient for contemporary concrete technology, leading to overengineering and difficulty predicting concrete properties. Machine learning-based methods offer a solution, as they have proven effective in predicting concrete compressive strength for concrete mix design. This paper scrutinises the association between the computational complexity of machine learning models and their proficiency in predicting the compressive strength of concrete. This study evaluates five deep neural network models of varying computational complexity in three series. Each model is trained and tested in three series with a vast database of concrete mix recipes and associated destructive tests. The findings suggest a positive correlation between increased computational complexity and the model's predictive ability. This correlation is evidenced by an increment in the coefficient of determination (R2) and a decrease in error metrics (mean squared error, Minkowski error, normalized squared error, root mean squared error, and sum squared error) as the complexity of the model increases. The research findings provide valuable insights for increasing the performance of concrete technical feature prediction models while acknowledging this study's limitations and suggesting potential future research directions. This research paves the way for further refinement of AI-driven methods in concrete mix design, enhancing the efficiency and precision of the concrete mix design process.

A direct discretization recurrent neurodynamics method for time-variant nonlinear optimization with redundant robot manipulators

Discrete time-variant nonlinear optimization (DTVNO) problems are commonly encountered in various scientific researches and engineering application fields. Nowadays, many discrete-time recurrent neurodynamics (DTRN) methods have been proposed for solving the DTVNO problems. However, these traditional DTRN methods currently employ an indirect technical route in which the discrete-time derivation process requires to interconvert with continuous-time derivation process. In order to break through this traditional research method, we develop a novel DTRN method based on the inspiring direct discrete technique for solving the DTVNO problem more concisely and efficiently. To be specific, firstly, considering that the DTVNO problem emerging in the discrete-time tracing control of robot manipulator, we further abstract and summarize the mathematical definition of DTVNO problem, and then we define the corresponding error function. Secondly, based on the second-order Taylor expansion, we can directly obtain the DTRN method for solving the DTVNO problem, which no longer requires the derivation process in the continuous-time environment. Whereafter, such a DTRN method is theoretically analyzed and its convergence is demonstrated. Furthermore, numerical experiments confirm the effectiveness and superiority of the DTRN method. In addition, the application experiments of the robot manipulators are presented to further demonstrate the superior performance of the DTRN method.

An overview of deep learning techniques for COVID-19 detection: methods, challenges, and future works

The World Health Organization (WHO) declared a pandemic in response to the coronavirus COVID-19 in 2020, which resulted in numerous deaths worldwide. Although the disease appears to have lost its impact, millions of people have been affected by this virus, and new infections still occur. Identifying COVID-19 requires a reverse transcription-polymerase chain reaction test (RT-PCR) or analysis of medical data. Due to the high cost and time required to scan and analyze medical data, researchers are focusing on using automated computer-aided methods. This review examines the applications of deep learning (DL) and machine learning (ML) in detecting COVID-19 using medical data such as CT scans, X-rays, cough sounds, MRIs, ultrasound, and clinical markers. First, the data preprocessing, the features used, and the current COVID-19 detection methods are divided into two subsections, and the studies are discussed. Second, the reported publicly available datasets, their characteristics, and the potential comparison materials mentioned in the literature are presented. Third, a comprehensive comparison is made by contrasting the similar and different aspects of the studies. Finally, the results, gaps, and limitations are summarized to stimulate the improvement of COVID-19 detection methods, and the study concludes by listing some future research directions for COVID-19 classification.

Growing Echo State Network With an Inverse-Free Weight Update Strategy

An echo state network (ESN) draws widespread attention and is applied in many scenarios. As the most typical approach for solving the ESN, the matrix inverse operation of high computational complexity is involved. However, in the modern big data era, addressing the heavy computational burden problem is necessary. In order to reduce the computational load, an inverse-free ESN (IFESN) is proposed for the first time in this article. Besides, an incremental IFESN is constructed to attain the network topology with theoretical proof on the training error's monotone decline property. Simulations and experiments are conducted on several numerical and real-world time-series benchmarks, and corresponding results indicate that the proposed model is superior to some existing models and possesses excellent practical application potential. The source code is publicly available at https://github.com/LongJin-lab/the-supplementary-file-for-CYB-E-2021-04-0944.

Deep reinforcement learning for automated search of model parameters: photo-fenton wastewater disinfection case study

Numerical optimization solves problems that are analytically intractable at the cost of arriving at a sufficiently good but rarely optimal solution. To maximize the result, optimization algorithms are run with the guidance and supervision of a human, usually an expert in the problem. Recent advances in deep reinforcement learning motivate interest in an artificial agent capable of learning to do the expert’s task. Specifically, we present a proximal policy optimization agent that learns to optimize in a real case study such as the modeling of the photo-fenton disinfection process, which involves a number of parameters that have to be adjusted to minimize the error of the model with respect to the experimental data collected in several trials. The expert spends an average of 4 h to find a suitable set of parameters. On the other hand, the agent we present does not require a human expert to guide or validate the optimization procedure and achieves similar results in $$2.5\times$$ 2.5 × less time.

A novel activation function based recurrent neural networks and their applications on sentiment classification and dynamic problems solving

In this paper, a nonlinear activation function (NAF) is proposed to constructed three recurrent neural network (RNN) models (Simple RNN (SRNN) model, Long Short-term Memory (LSTM) model and Gated Recurrent Unit (GRU) model) for sentiment classification. The Internet Movie Database (IMDB) sentiment classification experiment results demonstrate that the three RNN models using the NAF achieve better accuracy and lower loss values compared with other commonly used activation functions (AF), such as ReLU, SELU etc. Moreover, in terms of dynamic problems solving, a fixed-time convergent recurrent neural network (FTCRNN) model with the NAF is constructed. Additionally, the fixed-time convergence property of the FTCRNN model is strictly validated and the upper bound convergence time formula of the FTCRNN model is obtained. Furthermore, the numerical simulation results of dynamic Sylvester equation (DSE) solving using the FTCRNN model indicate that the neural state solutions of the FTCRNN model quickly converge to the theoretical solutions of DSE problems whether there are noises or not. Ultimately, the FTCRNN model is also utilized to realize trajectory tracking of robot manipulator and electric circuit currents computation for the further validation of its accurateness and robustness, and the corresponding results further validate its superior performance and widespread applicability.

Convolution neural network with batch normalization and inception-residual modules for Android malware classification

Deep learning technology is changing the landscape of cybersecurity research, especially the study of large amounts of data. With the rapid growth in the number of malware, developing of an efficient and reliable method for classifying malware has become one of the research priorities. In this paper, a new method, BIR-CNN, is proposed to classify of Android malware. It combines convolution neural network (CNN) with batch normalization and inception-residual (BIR) network modules by using 347-dim network traffic features. CNN combines inception-residual modules with a convolution layer that can enhance the learning ability of the model. Batch Normalization can speed up the training process and avoid over-fitting of the model. Finally, experiments are conducted on the publicly available network traffic dataset CICAndMal2017 and compared with three traditional machine learning algorithms and CNN. The accuracy of BIR-CNN is 99.73% in binary classification (2-classifier). Moreover, the BIR-CNN can classify malware by its category (4-classifier) and malicious family (35-classifier), with a classification accuracy of 99.53% and 94.38%, respectively. The experimental results show that the proposed model is an effective method for Android malware classification, especially in malware category and family classifier.

A Hybrid ARIMA-GABP Model for Predicting Sea Surface Temperature

Sea surface temperature (SST) is one of the most important parameters in air–sea interaction, and its accurate prediction is of great significance in the study of global climate change. However, SST is affected by heat flux, ocean dynamic processes, cloud coverage, and other factors, which means it contains linear and nonlinear components. Existing prediction models, especially single prediction models, cannot effectively handle these linear and nonlinear components in the meantime, degrading their accuracy concerning the prediction of SST. To remedy this weakness, this paper proposes a novel prediction model by the Lagrange multiplier method to combine the auto-regressive integrated moving average (ARIMA) model and the back propagation (BP) neural network model, where these two models have superior prediction performance for linear and nonlinear components, respectively. Moreover, the genetic algorithm is exploited to construct the genetic algorithm BP (GABP) neural network to further improve the performance of the proposed model. To verify the effectiveness of the proposed model, experiments predicting the SST based on historic time-series data are performed. The experiment results indicate that the mean absolute error (MAE) of the ARIMA-GABP model is only 0.3033 °C and the root mean square error (RMSE) is 0.3970 °C, which is better than the ARIMA model, BP neural network model, long short-term memory (LSTM) model, GABP neural network model, and ensemble empirical model decomposition BP model among various datasets. Therefore, the proposed model has superior and robust performance concerning predicting SST.

An Extra-Contrast Affinity Network for Facial Expression Recognition in the Wild

Learning discriminative features for facial expression recognition (FER) in the wild is a challenging task due to the significant intra-class variations, inter-class similarities, and extreme class imbalances. In order to solve these issues, a contrastive-learning-based extra-contrast affinity network (ECAN) method is proposed. The ECAN consists of a feature processing network and two proposed loss functions, namely extra negative supervised contrastive loss (ENSC loss) and multi-view affinity loss (MVA loss). The feature processing network provides current and historical deep features to satisfy the necessary conditions for these loss functions. Specifically, the ENSC loss function simultaneously considers many positive samples and extra negative samples from other minibatches to maximize intra-class similarity and the inter-class separation of deep features, while also automatically turning the attention of the model to majority and minority classes to alleviate the class imbalance issue. The MVA loss function improves upon the center loss function by leveraging additional deep feature groups from other minibatches to dynamically learn more accurate class centers and further enhance the intra-class compactness of deep features. The numerical results obtained using two public wild FER datasets (RAFDB and FER2013) indicate that the proposed method outperforms most state-of-the-art models in FER.

Automatic Detection of Liver Cancer Using Hybrid Pre-Trained Models

Liver cancer is a life-threatening illness and one of the fastest-growing cancer types in the world. Consequently, the early detection of liver cancer leads to lower mortality rates. This work aims to build a model that will help clinicians determine the type of tumor when it occurs within the liver region by analyzing images of tissue taken from a biopsy of this tumor. Working within this stage requires effort, time, and accumulated experience that must be possessed by a tissue expert to determine whether this tumor is malignant and needs treatment. Thus, a histology expert can make use of this model to obtain an initial diagnosis. This study aims to propose a deep learning model using convolutional neural networks (CNNs), which are able to transfer knowledge from pre-trained global models and decant this knowledge into a single model to help diagnose liver tumors from CT scans. Thus, we obtained a hybrid model capable of detecting CT images of a biopsy of a liver tumor. The best results that we obtained within this research reached an accuracy of 0.995, a precision value of 0.864, and a recall value of 0.979, which are higher than those obtained using other models. It is worth noting that this model was tested on a limited set of data and gave good detection results. This model can be used as an aid to support the decisions of specialists in this field and save their efforts. In addition, it saves the effort and time incurred by the treatment of this type of cancer by specialists, especially during periodic examination campaigns every year.

Prescribed-Time Convergent Adaptive ZNN for Time-Varying Matrix Inversion under Harmonic Noise

Harmonic noises widely exist in industrial fields and always affect the computational accuracy of neural network models. The existing original adaptive zeroing neural network (OAZNN) model can effectively suppress harmonic noises. Nevertheless, the OAZNN model’s convergence rate only stays at the exponential convergence, that is, its convergence speed is usually greatly affected by the initial state. Consequently, to tackle the above issue, this work combines the dynamic characteristics of harmonic signals with prescribed-time convergence activation function, and proposes a prescribed-time convergent adaptive ZNN (PTCAZNN) for solving time-varying matrix inverse problem (TVMIP) under harmonic noises. Owing to the nonlinear activation function used having the ability to reject noises itself and the adaptive term also being able to compensate the influence of noises, the PTCAZNN model can realize double noise suppression. More importantly, the theoretical analysis of PTCAZNN model with prescribed-time convergence and robustness performance is provided. Finally, by varying a series of conditions such as the frequency of single harmonic noise, the frequency of multi-harmonic noise, and the initial value and the dimension of the matrix, the comparative simulation results further confirm the effectiveness and superiority of the PTCAZNN model.

Serial and parallel convolutional neural network schemes for NFDM signals

Two conceptual convolutional neural network (CNN) schemes are proposed, developed and analysed for directly decoding nonlinear frequency division multiplexing (NFDM) signals with hardware implementation taken into consideration. A serial network scheme with a small network size is designed for small user applications, and a parallel network scheme with high speed is designed for places such as data centres. The work aimed at showing the potential of using CNN for practical NFDM-based fibre optic communication. In the numerical demonstrations, the serial network only occupies 0.5 MB of memory space while the parallel network occupies 128 MB of memory but allows parallel computing. Both network schemes were trained with simulated data and reached more than 99.9% accuracy.

Distributed and Time-Delayed k-Winner-Take-All Network for Competitive Coordination of Multiple Robots

In this article, a distributed and time-delayed k-winner-take-all (DT-kWTA) network is established and analyzed for competitively coordinated task assignment of a multirobot system. It is considered and designed from the following three aspects. First, a network is built based on a k-winner-take-all (kWTA) competitive algorithm that selects k maximum values from the inputs. Second, a distributed control strategy is used to improve the network in terms of communication load and computational burden. Third, the time-delayed problem prevalent in arbitrary causal systems (especially, in networks) is taken into account in the proposed network. This work combines distributed kWTA competition network with time delay for the first time, thus enabling it to better handle realistic applications than previous work. In addition, it theoretically derives the maximum delay allowed by the network and proves the convergence and robustness of the network. The results are applied to a multirobot system to conduct its robots' competitive coordination to complete the given task.

Alzheimer's Disease Diagnosis With Brain Structural MRI Using Multiview-Slice Attention and 3D Convolution Neural Network

Numerous artificial intelligence (AI) based approaches have been proposed for automatic Alzheimer's disease (AD) prediction with brain structural magnetic resonance imaging (sMRI). Previous studies extract features from the whole brain or individual slices separately, ignoring the properties of multi-view slices and feature complementarity. For this reason, we present a novel AD diagnosis model based on the multiview-slice attention and 3D convolution neural network (3D-CNN). Specifically, we begin by extracting the local slice-level characteristic in various dimensions using multiple sub-networks. Then we proposed a slice-level attention mechanism to emphasize specific 2D-slices to exclude the redundancy features. After that, a 3D-CNN was employed to capture the global subject-level structural changes. Finally, all these 2D and 3D features were fused to obtain more discriminative representations. We conduct the experiments on 1,451 subjects from ADNI-1 and ADNI-2 datasets. Experimental results showed the superiority of our model over the state-of-the-art approaches regarding dementia classification. Specifically, our model achieves accuracy values of 91.1 and 80.1% on ADNI-1 for AD diagnosis and mild cognitive impairment (MCI) convention prediction, respectively.

Generative Adversarial Training for Supervised and Semi-supervised Learning

Neural networks have played critical roles in many research fields. The recently proposed adversarial training (AT) can improve the generalization ability of neural networks by adding intentional perturbations in the training process, but sometimes still fail to generate worst-case perturbations, thus resulting in limited improvement. Instead of designing a specific smoothness function and seeking an approximate solution used in existing AT methods, we propose a new training methodology, named Generative AT (GAT) in this article, for supervised and semi-supervised learning. The key idea of GAT is to formulate the learning task as a minimax game, in which the perturbation generator aims to yield the worst-case perturbations that maximize the deviation of output distribution, while the target classifier is to minimize the impact of this perturbation and prediction error. To solve this minimax optimization problem, a new adversarial loss function is constructed based on the cross-entropy measure. As a result, the smoothness and confidence of the model are both greatly improved. Moreover, we develop a trajectory-preserving-based alternating update strategy to enable the stable training of GAT. Numerous experiments conducted on benchmark datasets clearly demonstrate that the proposed GAT significantly outperforms the state-of-the-art AT methods in terms of supervised and semi-supervised learning tasks, especially when the number of labeled examples is rather small in semi-supervised learning.

Design and Analysis of Anti-Noise Parameter-Variable Zeroing Neural Network for Dynamic Complex Matrix Inversion and Manipulator Trajectory Tracking

Dynamic complex matrix inversion (DCMI) problems frequently arise in the territories of mathematics and engineering, and various recurrent neural network (RNN) models have been reported to effectively find the solutions of the DCMI problems. However, most of the reported works concentrated on solving DCMI problems in ideal no noise environment, and the inevitable noises in reality are not considered. To enhance the robustness of the existing models, an anti-noise parameter-variable zeroing neural network (ANPVZNN) is proposed by introducing a novel activation function (NAF). Both of mathematical analysis and numerical simulation results demonstrate that the proposed ANPVZNN model possesses fixed-time convergence and robustness for solving DCMI problems. Besides, a successful ANPVZNN-based manipulator trajectory tracking example further verifies its robustness and effectiveness in practical applications.

Adaptive Label Allocation for Unsupervised Person Re-Identification

Most unsupervised methods of person re-identification (Re-ID) obtain pseudo-labels through clustering. However, in the process of clustering, the hard quantization loss caused by clustering errors will make the model produce false pseudo-labels. In order to solve this problem, an unsupervised model based on softened labels training method is proposed. The innovation of this method is that the correlation among image features is used to find the reliable positive samples and train them in a smooth manner. To further explore the correlation among image features, some modules are carefully designed in this article. The dynamic adaptive label allocation (DALA) method which generates pseudo-labels of adaptive size according to different metric relationships among features is proposed. The channel attention and transformer architecture (CATA) auxiliary module is designed, which, associated with convolutional neural network (CNN), functioned as the feature extractor of the model aimed to capture long range dependencies and acquire more distinguishable features. The proposed model is evaluated on the Market-1501 and the DukeMTMC-reID. The experimental results of the proposed method achieve 60.8 mAP on Market-1501 and 49.6 mAP on DukeMTMC-reID respectively, which outperform most state-of-the-art models in fully unsupervised Re-ID task.

SinLU: Sinu-Sigmoidal Linear Unit

Non-linear activation functions are integral parts of deep neural architectures. Given the large and complex dataset of a neural network, its computational complexity and approximation capability can differ significantly based on what activation function is used. Parameterizing an activation function with the introduction of learnable parameters generally improves the performance. Herein, a novel activation function called Sinu-sigmoidal Linear Unit (or SinLU) is proposed. SinLU is formulated as SinLU(x)=(x+asinbx)·σ(x), where σ(x) is the sigmoid function. The proposed function incorporates the sine wave, allowing new functionalities over traditional linear unit activations. Two trainable parameters of this function control the participation of the sinusoidal nature in the function, and help to achieve an easily trainable, and fast converging function. The performance of the proposed SinLU is compared against widely used activation functions, such as ReLU, GELU and SiLU. We showed the robustness of the proposed activation function by conducting experiments in a wide array of domains, using multiple types of neural network-based models on some standard datasets. The use of sine wave with trainable parameters results in a better performance of SinLU than commonly used activation functions.

Multimodal transistors as ReLU activation functions in physical neural network classifiers

Artificial neural networks (ANNs) providing sophisticated, power-efficient classification are finding their way into thin-film electronics. Thin-film technologies require robust, layout-efficient devices with facile manufacturability. Here, we show how the multimodal transistor’s (MMT’s) transfer characteristic, with linear dependence in saturation, replicates the rectified linear unit (ReLU) activation function of convolutional ANNs (CNNs). Using MATLAB, we evaluate CNN performance using systematically distorted ReLU functions, then substitute measured and simulated MMT transfer characteristics as proxies for ReLU. High classification accuracy is maintained, despite large variations in geometrical and electrical parameters, as CNNs use the same activation functions for training and classification.

AI-Crime Hunter: An AI Mixture of Experts for Crime Discovery on Twitter

Maintaining a healthy cyber society is a great challenge due to the users’ freedom of expression and behavior. This can be solved by monitoring and analyzing the users’ behavior and taking proper actions. This research aims to present a platform that monitors the public content on Twitter by extracting tweet data. After maintaining the data, the users’ interactions are analyzed using graph analysis methods. Then, the users’ behavioral patterns are analyzed by applying metadata analysis, in which the timeline of each profile is obtained; also, the time-series behavioral features of users are investigated. Then, in the abnormal behavior detection and filtering component, the interesting profiles are selected for further examinations. Finally, in the contextual analysis component, the contents are analyzed using natural language processing techniques; a binary text classification model (SVM (Support Vector Machine) + TF-IDF (Term Frequency—Inverse Document Frequency) with 88.89% accuracy) is used to detect if a tweet is related to crime or not. Then, a sentiment analysis method is applied to the crime-related tweets to perform aspect-based sentiment analysis (DistilBERT + FFNN (Feed-Forward Neural Network) with 80% accuracy), because sharing positive opinions about a crime-related topic can threaten society. This platform aims to provide the end-user (the police) with suggestions to control hate speech or terrorist propaganda.

Hybrid Machine Learning Model for Body Fat Percentage Prediction Based on Support Vector Regression and Emotional Artificial Neural Networks

Obesity or excessive body fat causes multiple health problems and diseases. However, obesity treatment and control need an accurate determination of body fat percentage (BFP). The existing methods for BFP estimation require several procedures, which reduces their cost-effectivity and generalization. Therefore, developing cost-effective models for BFP estimation is vital for obesity treatment. Machine learning models, particularly hybrid models, have a strong ability to analyze challenging data and perform predictions by combining different characteristics of the models. This study proposed a hybrid machine learning model based on support vector regression and emotional artificial neural networks (SVR-EANNs) for accurate recent BFP prediction using a primary BFP dataset. SVR was applied as a consistent attribute selection model on seven properties and measurements, using the left-out sensitivity analysis, and the regression ability of the EANN was considered in the prediction phase. The proposed model was compared to seven benchmark machine learning models. The obtained results show that the proposed hybrid model (SVR-EANN) outperformed other machine learning models by achieving superior results in the three considered evaluation metrics. Furthermore, the proposed model suggested that abdominal circumference is a significant factor in BFP prediction, while age has a minor effect.