A novel type of activation function in artificial neural networks: Trained activation function

Optimizing skin cancer screening with convolutional neural networks in smart healthcare systems

Skin cancer is among the most prevalent types of malignancy all over the global and is strongly associated with the patient's prognosis and the accuracy of the initial diagnosis. Clinical examination of skin lesions is a key aspect that is important in the assessment of skin disease but comes with some drawbacks mainly with interpretational aspects, time-consuming and healthare expenditure. Skin cancer if detected early and treated in time can be controlled and its deadly impacts arrested completely. Algorithms applied in convolutional neural network (CNN) could lead to an enhanced speed of identifying and distinguishing a disease, which in turn leads to early detection and treatment. So as to eliminate these challenges, optimized CNN prediction models for cancer skin classification is studied in this researche. The objectives of this study were to develop reliable optimized CNN prediction models for skin cancer classification, to handle the severe class imbalance problem where skin cancer class was found to be much smaller than the healthy class. To evaluate model interpretability and to develop an end-to-end smart healthcare system using explainable AI (XAI) such as Grad-CAM and Grad-CAM++. In this researche new activation function namely NGNDG-AF was offered specifically to enhance the capabilities of network fitting and generalization ability, convergence rate and reduction in mathematical computational cost. A research used an optimized CNN and ResNet152V2 with the HAM10000 dataset to differentiate between the seven forms of skin cancer. Model training involved the use of two optimization functions (RMSprop and Adam) and NGNDG-AF activation functions. Cross validation technique the holdout validation is used to estimate of the model's generalization performance for unseed data. Optimized CNN is performing well as compare to ResNet152V2 for unseen data. The efficacy of the optimized CNN method with NGNDG-AF was examined by a comparative study wirh popular CNN with various activation functions shows that better performance of NGNDG-AF, achieving the classification accuracy rates that are as high as 99% in training and 98% in the validation. The recommended system also involves the integration of the smart healthcare application as a central component to give the doctors as well as the healthcare providers diagnosing and tools that would assist in the early detection of skin cancer hence leading to better outcomes of the treatment.

Optimizing cryptographic protocols against side channel attacks using WGAN-GP and genetic algorithms

This research introduces a novel hybrid cryptographic framework that combines traditional cryptographic protocols with advanced methodologies, specifically Wasserstein Generative Adversarial Networks with Gradient Penalty (WGAN-GP) and Genetic Algorithms (GA). We evaluated several cryptographic protocols, including AES-ECB, AES-GCM, ChaCha20, RSA, and ECC, against critical metrics such as security level, efficiency, side-channel resistance, and cryptanalysis resistance. Our findings demonstrate that this integrated approach significantly enhances both security and efficiency across all evaluated protocols. Notably, the AES-GCM algorithm exhibited superior performance, achieving minimal computation time and robust side-channel resistance. This study underscores the potential of leveraging machine learning and evolutionary algorithms to advance cryptographic protocol security and efficiency, laying a robust foundation for future advancements in cybersecurity.

Dynamics of heterogeneous Hopfield neural network with adaptive activation function based on memristor

Memristor and activation function are two important nonlinear factors of the memristive Hopfield neural network. The effects of different memristors on the dynamics of Hopfield neural networks have been studied by many researchers. However, less attention has been paid to the activation function. In this paper, we present a heterogeneous memristive Hopfield neural network with neurons using different activation functions. The activation functions include fixed activation functions and an adaptive activation function, where the adaptive activation function is based on a memristor. The theoretical and experimental study of the neural network's dynamics has been conducted using phase portraits, bifurcation diagrams, and Lyapunov exponents spectras. Numerical results show that complex dynamical behaviors such as multi-scroll chaos, transient chaos, state jumps and multi-type coexisting attractors can be observed in the heterogeneous memristive Hopfield neural network. In addition, the hardware implementation of memristive Hopfield neural network with adaptive activation function is designed and verified. The experimental results are in good agreement with those obtained using numerical simulations.

Stability and Bifurcation Exploration of Delayed Neural Networks With Radial-Ring Configuration and Bidirectional Coupling

For decades, studying the dynamic performances of artificial neural networks (ANNs) is widely considered to be a good way to gain a deeper insight into actual neural networks. However, most models of ANNs are focused on a finite number of neurons and a single topology. These studies are inconsistent with actual neural networks composed of thousands of neurons and sophisticated topologies. There is still a discrepancy between theory and practice. In this article, not only a novel construction of a class of delayed neural networks with radial-ring configuration and bidirectional coupling is proposed, but also an effective analytical approach to dynamic performances of large-scale neural networks with a cluster of topologies is developed. First, Coates' flow diagram is applied to acquire the characteristic equation of the system, which contains multiple exponential terms. Second, by means of the idea of the holistic element, the sum of the neuron synapse transmission delays is regarded as the bifurcation argument to investigate the stability of the zero equilibrium point and the beingness of Hopf bifurcation. Finally, multiple sets of computerized simulations are utilized to confirm the conclusions. The simulation results expound that the increase in transmission delay may cause a leading impact on the generation of Hopf bifurcation. Meanwhile, the number and the self-feedback coefficient of neurons are also playing significant roles in the appearance of periodic oscillations.

nPCA: a linear dimensionality reduction method using a multilayer perceptron

Background: Linear dimensionality reduction techniques are widely used in many applications. The goal of dimensionality reduction is to eliminate the noise of data and extract the main features of data. Several dimension reduction methods have been developed, such as linear-based principal component analysis (PCA), nonlinear-based t-distributed stochastic neighbor embedding (t-SNE), and deep-learning-based autoencoder (AE). However, PCA only determines the projection direction with the highest variance, t-SNE is sometimes only suitable for visualization, and AE and nonlinear methods discard the linear projection. Results: To retain the linear projection of raw data and generate a better result of dimension reduction either for visualization or downstream analysis, we present neural principal component analysis (nPCA), an unsupervised deep learning approach capable of retaining richer information of raw data as a promising improvement to PCA. To evaluate the performance of the nPCA algorithm, we compare the performance of 10 public datasets and 6 single-cell RNA sequencing (scRNA-seq) datasets of the pancreas, benchmarking our method with other classic linear dimensionality reduction methods. Conclusion: We concluded that the nPCA method is a competitive alternative method for dimensionality reduction tasks.

Nondestructive Estimation of Hazelnut (Corylus avellana L.) Terminal Velocity and Drag Coefficient Based on Some Fruit Physical Properties Using Machine Learning Algorithms

Hazelnut culture originated in Turkey, which has the highest volume and area of hazelnut production in the world. For the design and sizing of equipment and structures in agricultural operations for the hazelnut industry, especially harvesting operations and post-harvest operations, it is essential that an understanding of hazelnuts' aerodynamic properties, i.e., terminal velocity and drag coefficient, is acquired. In this study, the moisture, mass, density, projected area, surface area, and geometric diameter were used as independent variables in the data set, and the dependent variables terminal velocity and drag coefficient estimation were determined. In this study, logistic regression (LR), support vector regression (SVR), and artificial neural networks (ANNs) were used based on machine learning methods. When the results were evaluated according to R2 (determination coefficient), MSE (mean squared error), and MAE (mean absolute error) metrics, it was seen that the most successful models were the ANN, SVR, and LR, respectively. According to the R2 metric, the ANN method achieved 91.5% for the terminal velocity of hazelnuts and 85.9% for the drag coefficient of hazelnuts. Using the independent variables in the study, it was seen that the terminal velocity and drag coefficient value of hazelnuts could be successfully estimated.

Digital Media Teaching and Effectiveness Evaluation Integrating Big Data and Artificial Intelligence

With the development of digital media technology, its application in teaching and learning is becoming more widespread. Digital media technology helps present information in transmitting knowledge or skills, reduces cognitive load, and promotes understanding of knowledge. Evaluation of the effectiveness of digital media teaching has also become important. A scientific and reasonable evaluation of digital media teaching effectiveness can help teachers select digital media technology and grasp the amount, degree, and timing of digital media use to change teaching effectiveness. This paper proposed using a combination of big data and artificial intelligence methods to evaluate the effectiveness of digital media teaching methods using the RBF neural network model. The digital media teaching effectiveness evaluation was used as the input variable of RBF, the degree of digital media effectiveness was the output variable and the neural network was trained through the collected sample data. The research results showed that the RBF neural network model proposed in this paper has a strong generalization and extension ability in evaluating digital media teaching effectiveness, providing a new way to evaluate digital media teaching effectiveness.

Spaceborne GNSS-R Wind Speed Retrieval Using Machine Learning Methods

This paper focuses on sea surface wind speed estimation using L1B level v3.1 data of reflected GNSS signals from the Cyclone GNSS (CYGNSS) mission and European Centre for Medium-range Weather Forecast Reanalysis (ECMWF) wind speed data. Seven machine learning methods are applied for wind speed retrieval, i.e., Regression trees (Binary Tree (BT), Ensembles of Trees (ET), XGBoost (XGB), LightGBM (LGBM)), ANN (Artificial neural network), Stepwise Linear Regression (SLR), and Gaussian Support Vector Machine (GSVM), and a comparison of their performance is made. The wind speed is divided into two different ranges to study the suitability of the different algorithms. A total of 10 observation variables are considered as input parameters to study the importance of individual variables or combinations thereof. The results show that the LGBM model performs the best with an RMSE of 1.419 and a correlation coefficient of 0.849 in the low wind speed interval (0–15 m/s), while the ET model performs the best with an RMSE of 1.100 and a correlation coefficient of 0.767 in the high wind speed interval (15–30 m/s). The effects of the variables used in wind speed retrieval models are investigated using the XGBoost importance metric, showing that a number of variables play a very significant role in wind speed retrieval. It is expected that these results will provide a useful reference for the development of advanced wind speed retrieval algorithms in the future.

Biophysical Model: A Promising Method in the Study of the Mechanism of Propofol: A Narrative Review

The physiological and neuroregulatory mechanism of propofol is largely based on very limited knowledge. It is one of the important puzzling issues in anesthesiology and is of great value in both scientific and clinical fields. It is acknowledged that neural networks which are comprised of a number of neural circuits might be involved in the anesthetic mechanism. However, the mechanism of this hypothesis needs to be further elucidated. With the progress of artificial intelligence, it is more likely to solve this problem through using artificial neural networks to perform temporal waveform data analysis and to construct biophysical computational models. This review focuses on current knowledge regarding the anesthetic mechanism of propofol, an intravenous general anesthetic, by constructing biophysical computational models.

Deep learning in electron microscopy

Deep learning is transforming most areas of science and technology, including electron microscopy. This review paper offers a practical perspective aimed at developers with limited familiarity. For context, we review popular applications of deep learning in electron microscopy. Following, we discuss hardware and software needed to get started with deep learning and interface with electron microscopes. We then review neural network components, popular architectures, and their optimization. Finally, we discuss future directions of deep learning in electron microscopy.

Comparison of Diagnosis Accuracy between a Backpropagation Artificial Neural Network Model and Linear Regression in Digestive Disease Patients: an Empirical Research

Introduction

A Noninvasive diagnosis model for digestive diseases is the vital issue for the current clinical research. Our systematic review is aimed at demonstrating diagnosis accuracy between the BP-ANN algorithm and linear regression in digestive disease patients, including their activation function and data structure.

Methods

We reported the systematic review according to the PRISMA guidelines. We searched related articles from seven electronic scholarly databases for comparison of the diagnosis accuracy focusing on BP-ANN and linear regression. The characteristics, patient number, input/output marker, diagnosis accuracy, and results/conclusions related to comparison were extracted independently based on inclusion criteria.

Results

Nine articles met all the criteria and were enrolled in our review. Of those enrolled articles, the publishing year ranged from 1991 to 2017. The sample size ranged from 42 to 3222 digestive disease patients, and all of the patients showed comparable biomarkers between the BP-ANN algorithm and linear regression. According to our study, 8 literature demonstrated that the BP-ANN model is superior to linear regression in predicting the disease outcome based on AUROC results. One literature reported linear regression to be superior to BP-ANN for the early diagnosis of colorectal cancer.

Conclusion

The BP-ANN algorithm and linear regression both had high capacity in fitting the diagnostic model and BP-ANN displayed more prediction accuracy for the noninvasive diagnosis model of digestive diseases. We compared the activation functions and data structure between BP-ANN and linear regression for fitting the diagnosis model, and the data suggested that BP-ANN was a comprehensive recommendation algorithm.

Stochastic Selection of Activation Layers for Convolutional Neural Networks

In recent years, the field of deep learning has achieved considerable success in pattern recognition, image segmentation, and many other classification fields. There are many studies and practical applications of deep learning on images, video, or text classification. Activation functions play a crucial role in discriminative capabilities of the deep neural networks and the design of new "static" or "dynamic" activation functions is an active area of research. The main difference between "static" and "dynamic" functions is that the first class of activations considers all the neurons and layers as identical, while the second class learns parameters of the activation function independently for each layer or even each neuron. Although the "dynamic" activation functions perform better in some applications, the increased number of trainable parameters requires more computational time and can lead to overfitting. In this work, we propose a mixture of "static" and "dynamic" activation functions, which are stochastically selected at each layer. Our idea for model design is based on a method for changing some layers along the lines of different functional blocks of the best performing CNN models, with the aim of designing new models to be used as stand-alone networks or as a component of an ensemble. We propose to replace each activation layer of a CNN (usually a ReLU layer) by a different activation function stochastically drawn from a set of activation functions: in this way, the resulting CNN has a different set of activation function layers.

Environmental odour management by artificial neural network - A review

Unwanted odour emissions are considered air pollutants that may cause detrimental impacts to the environment as well as an indicator of unhealthy air to the affected individuals resulting in annoyance and health related issues. These pollutants are challenging to handle due to their invisibility to the naked eye and can only be felt by the human olfactory stimuli. A strategy to address this issue is by introducing an intelligent processing system to odour monitoring instrument such as artificial neural network to achieve a robust result. In this paper, a review on the application of artificial neural network for the management of environmental odours is presented. The principal factors in developing an optimum artificial neural network were identified as elements, structure and learning algorithms. The management of environmental odour has been distinguished into four aspects such as measurement, characterization, control and treatment and continuous monitoring. For each aspect, the performance of the neural network is critically evaluated emphasizing the strengths and weaknesses. This work aims to address the scarcity of information by addressing the gaps from existing studies in terms of the selection of the most suitable configuration, the benefits and consequences. Adopting this technique could provide a new avenue in the management of environmental odours through the use of a powerful mathematical computing tool for a more efficient and reliable outcome.

An Intelligent and Smart Environment Monitoring System for Healthcare

Skin wound healing is influenced by two kinds of environment i.e., exterior environment that is nearby to wound surface and interior environment that is the environment of the adjacent part under wound surface. Both types of environment play a vital role in wound healing, which may contribute to continuous or impaired wound healing. Although, different previous studies provided wound care solutions, but they focused on single environmental factors either wound moisture level, pH value or healing enzymes. Practically, it is insignificant to consider environmental effect by determination of single factors or two, as both types of environment contain a lot of other factors which must be part of investigation e.g., smoke, air pollution, air humidity, temperature, hydrogen gases etc. Also, previous studies didn’t classify overall healing either as continuous or impaired based on exterior environment effect. In current research work, we proposed an effective wound care solution based on exterior environment monitoring system integrated with Neural Network Model to consider exterior environment effect on wound healing process, either as continuous or impaired. Current research facilitates patients by providing them intelligent wound care solution to monitor and control wound healing at their home.