Universal Approximation Abilities of a Modular Differentiable Neural Network

Approximation ability is one of the most important topics in the field of neural networks (NNs). Feedforward NNs, activated by rectified linear units and some of their specific smoothed versions, provide universal approximators to convex as well as continuous functions. However, most of these networks are investigated empirically, or their characteristics are analyzed based on specific operation rules. Moreover, an adequate level of interpretability of the networks is missing as well. In this work, we propose a class of new network architecture, built with reusable neural modules (functional blocks), to supply differentiable and interpretable approximators for convex and continuous target functions. Specifically, first, we introduce a concrete model construction mechanism with particular blocks based on differentiable programming and the composition essence of the max operator, extending the scope of existing activation functions. Moreover, explicit block diagrams are provided for a clear understanding of the external architecture and the internal processing mechanism. Subsequently, the approximation behavior of the proposed network to convex functions and continuous functions is rigorously proved as well, by virtue of mathematical induction. Finally, plenty of numerical experiments are conducted on a wide variety of problems, which exhibit the effectiveness and the superiority of the proposed model over some existing ones.

Interpretable optimisation-based approach for hyper-box classification

Data classification is considered a fundamental research subject within the machine learning community. Researchers seek the improvement of machine learning algorithms in not only accuracy, but also interpretability. Interpretable algorithms allow humans to easily understand the decisions that a machine learning model makes, which is challenging for black box models. Mathematical programming-based classification algorithms have attracted considerable attention due to their ability to effectively compete with leading-edge algorithms in terms of both accuracy and interpretability. Meanwhile, the training of a hyper-box classifier can be mathematically formulated as a Mixed Integer Linear Programming (MILP) model and the predictions combine accuracy and interpretability. In this work, an optimisation-based approach is proposed for multi-class data classification using a hyper-box representation, thus facilitating the extraction of compact IF-THEN rules. The key novelty of our approach lies in the minimisation of the number and length of the generated rules for enhanced interpretability. Through a number of real-world datasets, it is demonstrated that the algorithm exhibits favorable performance when compared to well-known alternatives in terms of prediction accuracy and rule set simplicity.

Machine Learning-Based Fault Detection and Exclusion for Global Navigation Satellite System Pseudorange in the Measurement Domain

Global Navigation Satellite Systems (GNSS) support numerous applications, including mission-critical ones that require a high level of integrity for safe operations, such as air, maritime, and land-based navigation. Fault Detection and Exclusion (FDE) is crucial for mission-critical applications, as faulty measurements significantly impact system integrity. FDE can be applied within the positioning algorithm in the measurement's domain and the integrity monitoring domain. Previous research has utilized a limited number of Machine Learning (ML) models and Quality Indicators (QIs) for the FDE process in the measurement domain. It has not evaluated the pseudorange measurement fault thresholds that need to be detected. In addition, ML models were mainly evaluated based on accuracy, which alone does not provide a comprehensive evaluation. This paper introduces a comprehensive framework for traditional ML-based FDE prediction models in the measurement domain for pseudorange in complex environments. For the first time, this study evaluates the fault detection thresholds across 40 values, ranging from 1 to 40 m, using six ML models for FDE. These models include Decision Tree, K-Nearest Neighbors (KNN), Discriminant, Logistic, Neural Network, and Trees (Boosted, Bagged, and Rusboosted). The models are comprehensively assessed based on four key aspects: accuracy, probability of misdetection, probability of fault detection, and the percentage of excluded data. The results show that ML models can provide a high level of performance in the FDE process, exceeding 95% accuracy when the fault threshold is equal to or greater than 4 m, with KNN providing the highest FDE performance.

Comparative analysis of modified Johnson-Cook model and artificial neural network for flow stress prediction in BN-reinforced AZ80 magnesium composite

Boron nitride (BN), renowned for its exceptional optoelectrical properties, mechanical robustness, and thermal stability, has emerged as a promising two-dimensional material. Reinforcing AZ80 magnesium alloy with BN can significantly enhance its mechanical properties. To investigate and predict this enhancement during hot deformation, we introduce two independent modeling approaches a modified Johnson-Cook constitutive model and an artificial neural network (ANN). These models aim to capture both linear and nonlinear deformation characteristics. Hot compression tests conducted across various temperatures and strain rates provided a comprehensive dataset for model validation. The MJCC model, accounting for strain rate and temperature effects, achieved a correlation coefficientRof 0.96 and an average absolute relative error (AARE) of 6.28%. In contrast, the ANN, trained on experimental data, improved the correlation coefficient toRof 0.99 and reduced the AARE to below 1.5%, significantly enhancing predictive accuracy. These results indicate that while the modified J-C model provides reliable predictions under moderate conditions, the ANN more effectively captures complex behaviors under extreme deformation conditions. By comparing these modeling approaches, our study offers valuable insights for accurately predicting the rheological behavior of BN-reinforced AZ80 magnesium composite, aiding process optimization in industrial applications.

Optimisation of Clutch Disc Friction Material Using a Multi-Layer Perceptron Artificial Neural Network

This paper presents an analysis of four clutch disc friction materials (from different manufacturers) used in manual transmissions. Scanning electron microscopy and energy-dispersive X-ray spectroscopy were employed for the microstructural and chemical characterisation of the friction materials. To reveal the tribological properties of the selected clutch discs, three measurements of the friction coefficient between the material and the cast iron disc were conducted. The findings were employed to construct an artificial neural network using Easy NN software (V 14), with the objective of optimising the friction material. The chemical composition of the friction materials was employed as the input data, whereas the minimum, maximum, and average values of the friction coefficient, as well as the temperature generated during friction, were utilised as the output data. To assess the efficacy of the neural network, the correlation between the importance of input data and their sensitivity to output data was examined. It was determined that the model with three hidden layers exhibited a notable correlation between the six most influential chemical elements and their sensitivity. Based on this neural model, the chemical composition of the friction disc materials was optimised using the "Query" mode, aiming to minimise discrepancies in friction coefficients and temperature development.

MOSDNET: A multi-omics classification framework using simplified multi-view deep discriminant representation learning and dynamic edge GCN with multi-task learning

The integration of multi-omics data offers a robust approach to understanding the complexity of diseases by combining information from various biological levels, such as genomics, transcriptomics, proteomics, and metabolomics. This integrated approach is essential for a comprehensive understanding of disease mechanisms and for developing more effective diagnostic and therapeutic strategies. Nevertheless, most current methodologies fail to effectively extract both shared and specific representations from omics data. This study introduces MOSDNET, a multi-omics classification framework that effectively extracts shared and specific representations. This framework leverages Simplified Multi-view Deep Discriminant Representation Learning (S-MDDR) and Dynamic Edge GCN (DEGCN) to enhance the accuracy and efficiency of disease classification. Initially, MOSDNET utilizes S-MDDR to establish similarity and orthogonal constraints for extracting these representations, which are then concatenated to integrate the multi-omics data. Subsequently, MOSDNET constructs a comprehensive view of the sample data by employing patient similarity networks. By incorporating similarity networks into DEGCN, MOSDNET learns intricate network structures and node representations, which enables superior classification outcomes. MOSDNET is trained through a multitask learning approach, effectively leveraging the complementary knowledge from both the data integration and classification components. After conducting extensive comparative experiments, we have conclusively demonstrated that MOSDNET outperforms leading state-of-the-art multi-omics classification models in terms of classification accuracy. Simultaneously, we employ MOSDNET to identify pivotal biomarkers within the multi-omics data, providing insights into disease etiology and progression.

Recurrence microstates for machine learning classification

Recurrence microstates are obtained from the cross recurrence of two sequences of values embedded in a time series, being the generalization of the concept of recurrence of a given state in phase space. The probability of occurrence of each microstate constitutes a recurrence quantifier. The set of probabilities of all microstates are capable of detecting even small changes in the data pattern. This creates an ideal tool for generating features in machine learning algorithms. Thanks to the sensitivity of the set of probabilities of occurrence of microstates, it can be used to feed a deep neural network, namely, a microstate multi-layer perceptron (MMLP) to classify parameters of chaotic systems. Additionally, we show that with more microstates, the accuracy of the MMLP increases, showing that the increasing size and number of microstates insert new and independent information into the analysis. We also explore potential applications of the proposed method when adapted to different contexts.

Optimizing Rare Disease Gait Classification through Data Balancing and Generative AI: Insights from Hereditary Cerebellar Ataxia

The interpretability of gait analysis studies in people with rare diseases, such as those with primary hereditary cerebellar ataxia (pwCA), is frequently limited by the small sample sizes and unbalanced datasets. The purpose of this study was to assess the effectiveness of data balancing and generative artificial intelligence (AI) algorithms in generating synthetic data reflecting the actual gait abnormalities of pwCA. Gait data of 30 pwCA (age: 51.6 ± 12.2 years; 13 females, 17 males) and 100 healthy subjects (age: 57.1 ± 10.4; 60 females, 40 males) were collected at the lumbar level with an inertial measurement unit. Subsampling, oversampling, synthetic minority oversampling, generative adversarial networks, and conditional tabular generative adversarial networks (ctGAN) were applied to generate datasets to be input to a random forest classifier. Consistency and explainability metrics were also calculated to assess the coherence of the generated dataset with known gait abnormalities of pwCA. ctGAN significantly improved the classification performance compared with the original dataset and traditional data augmentation methods. ctGAN are effective methods for balancing tabular datasets from populations with rare diseases, owing to their ability to improve diagnostic models with consistent explainability.

One-step Bayesian example-dependent cost classification: The OsC-MLP method

Example-dependent cost classification problems are those where the decision costs depend not only on the true and the attributed classes but also on the sample features. Discriminative algorithms that carry out such classification tasks must take this dependence into account. In some applications, the decision costs are known for the training set but not in production, which complicates the problem. In this paper, we introduce a new one-step Bayesian formulation to train Neural Networks and solve the above limitation for binary cases with one-step Learning Machines, avoiding the drawbacks that unknown analytical forms of the example-dependent costs create. The formulation is based on defining an artificial likelihood ratio by using the available training classification costs in its definition, and proposes a test that does not require the values of the costs for unseen samples. Furthermore, it also includes Bayesian rebalancing mechanisms to combat the negative effects of class imbalance. Experimental results support the consistency and effectiveness of the corresponding algorithms.

Using voice recognition to measure trust during interactions with automated vehicles

Trust in an automated vehicle system (AVs) can impact the experience and safety of drivers and passengers. This work investigates the effects of speech to measure drivers' trust in the AVs. Seventy-five participants were randomly assigned to high-trust (the AVs with 100% correctness, 0 crash, and 4 system messages with visual-auditory TORs) and low-trust group (the AVs with a correctness of 60%, a crash rate of 40%, 2 system messages with visual-only TORs). Voice interaction tasks were used to collect speech information during the driving process. The results revealed that our settings successfully induced trust and distrust states. The corresponding extracted speech feature data of the two trust groups were used for back-propagation neural network training and evaluated for its ability to accurately predict the trust classification. The highest classification accuracy of trust was 90.80%. This study proposes a method for accurately measuring trust in automated vehicles using voice recognition.

The application of machine learning on brain imaging features of different narcolepsy subtypes

STUDY OBJECTIVES

Narcolepsy is a central hypersomnia disorder, and differential diagnoses between its subtypes can be difficult. Hence, we applied machine learning to analyze the positron emission tomography (PET) data of patients with type 1 or type 2 narcolepsy, and patients with type 1 narcolepsy and comorbid schizophrenia, to construct predictive models to facilitate the diagnosis.

METHODS

This is a retrospective and prospective case-control study of adolescent and young adult patients with type 1 or type 2 narcolepsy, and type 1 narcolepsy and comorbid schizophrenia. All participants received 18-F-fluorodeoxy glucose PET, sleep studies, neurocognitive tests, sleep questionnaires, and human leukocyte antigen typing. The collected PET data were analyzed by feature selections and classification methods in machine learning to construct predictive models.

RESULTS

A total of 314 participants with narcolepsy were enrolled; 204 had type 1 narcolepsy, 90 had type 2 narcolepsy, and 20 had type 1 narcolepsy and comorbid schizophrenia. We used three filter methods for feature selection followed by a comparative analysis of classification methods. To apply a small number of regions of interest (ROI) and high classification accuracy, the Naïve Bayes classifier with the Term Variance as feature selection achieved the goal with only three ROIs (left basal ganglia, left Heschl, and left striatum) and produced an accuracy of higher than 99%.

CONCLUSIONS

The accuracy of our predictive model of PET data are promising and can aid clinicians in the diagnosis of narcolepsy subtypes. Future research with a larger sample size could further refine the predictive model of narcolepsy.

Scale-Fractal Detrended Fluctuation Analysis for Fault Diagnosis of a Centrifugal Pump and a Reciprocating Compressor

Reciprocating compressors and centrifugal pumps are rotating machines used in industry, where fault detection is crucial for avoiding unnecessary and costly downtime. A novel method for fault classification in reciprocating compressors and multi-stage centrifugal pumps is proposed. In the feature extraction stage, raw vibration signals are processed using multi-fractal detrended fluctuation analysis (MFDFA) to extract features indicative of different types of faults. Such MFDFA features enable the training of machine learning models for classifying faults. Several classical machine learning models and a deep learning model corresponding to the convolutional neural network (CNN) are compared with respect to their classification accuracy. The cross-validation results show that all models are highly accurate for classifying the 13 types of faults in the centrifugal pump, the 17 valve faults, and the 13 multi-faults in the reciprocating compressor. The random forest subspace discriminant (RFSD) and the CNN model achieved the best results using MFDFA features calculated with quadratic approximations. The proposed method is a promising approach for fault classification in reciprocating compressors and multi-stage centrifugal pumps.

Constitutively Synergistic Multiagent Drug Formulations Targeting MERTK, FLT3, and BCL-2 for Treatment of AML

PURPOSE

Although high-dose, multiagent chemotherapy has improved leukemia survival rates, treatment outcomes remain poor in high-risk subsets, including acute myeloid leukemia (AML) and acute lymphoblastic leukemia (ALL) in infants. The development of new, more effective therapies for these patients is therefore an urgent, unmet clinical need.

METHODS

The dual MERTK/FLT3 inhibitor MRX-2843 and BCL-2 family protein inhibitors were screened in high-throughput against a panel of AML and MLL-rearranged precursor B-cell ALL (infant ALL) cell lines. A neural network model was built to correlate ratiometric drug synergy and target gene expression. Drugs were loaded into liposomal nanocarriers to assess primary AML cell responses.

RESULTS

MRX-2843 synergized with venetoclax to reduce AML cell density in vitro. A neural network classifier based on drug exposure and target gene expression predicted drug synergy and growth inhibition in AML with high accuracy. Combination monovalent liposomal drug formulations delivered defined drug ratios intracellularly and recapitulated synergistic drug activity. The magnitude and frequency of synergistic responses were both maintained and improved following drug formulation in a genotypically diverse set of primary AML bone marrow specimens.

CONCLUSIONS

We developed a nanoscale combination drug formulation that exploits ectopic expression of MERTK tyrosine kinase and dependency on BCL-2 family proteins for leukemia cell survival in pediatric AML and infant ALL cells. We demonstrate ratiometric drug delivery and synergistic cell killing in AML, a result achieved by a systematic, generalizable approach of combination drug screening and nanoscale formulation that may be extended to other drug pairs or diseases in the future.

The Application of Digital Technologies and Artificial Intelligence in Healthcare: An Overview on Nutrition Assessment

In the last decade, artificial intelligence (AI) and AI-mediated technologies have undergone rapid evolution in healthcare and medicine, from apps to computer software able to analyze medical images, robotic surgery and advanced data storage system. The main aim of the present commentary is to briefly describe the evolution of AI and its applications in healthcare, particularly in nutrition and clinical biochemistry. Indeed, AI is revealing itself to be an important tool in clinical nutrition by using telematic means to self-monitor various health metrics, including blood glucose levels, body weight, heart rate, fat percentage, blood pressure, activity tracking and calorie intake trackers. In particular, the application of the most common digital technologies used in the field of nutrition as well as the employment of AI in the management of diabetes and obesity, two of the most common nutrition-related pathologies worldwide, will be presented.

Analysis of predictive performance and reliability of classifiers for quality assessment of medical evidence revealed important variation by medical area

OBJECTIVES

A major obstacle in deployment of models for automated quality assessment is their reliability. To analyze their calibration and selective classification performance.

STUDY DESIGN AND SETTING

We examine two systems for assessing the quality of medical evidence, EvidenceGRADEr and RobotReviewer, both developed from Cochrane Database of Systematic Reviews (CDSR) to measure strength of bodies of evidence and risk of bias (RoB) of individual studies, respectively. We report their calibration error and Brier scores, present their reliability diagrams, and analyze the risk-coverage trade-off in selective classification.

RESULTS

The models are reasonably well calibrated on most quality criteria (expected calibration error [ECE] 0.04-0.09 for EvidenceGRADEr, 0.03-0.10 for RobotReviewer). However, we discover that both calibration and predictive performance vary significantly by medical area. This has ramifications for the application of such models in practice, as average performance is a poor indicator of group-level performance (e.g., health and safety at work, allergy and intolerance, and public health see much worse performance than cancer, pain, and anesthesia, and Neurology). We explore the reasons behind this disparity.

CONCLUSION

Practitioners adopting automated quality assessment should expect large fluctuations in system reliability and predictive performance depending on the medical area. Prospective indicators of such behavior should be further researched.

Machine Learning Approach for Muscovy Duck (Cairina moschata) Semen Quality Assessment

This study aimed to develop a comprehensive approach for assessing fresh ejaculate from Muscovy duck (Cairina moschata) drakes to fulfil the requirements of artificial insemination in farm practices. The approach combines sperm kinetics (CASA) with non-kinetic parameters, such as vitality, enzyme activities (alkaline phosphatase (AP), creatine kinase (CK), lactate dehydrogenase (LDH), and γ-glutamyl-transferase (GGT)), and total DNA methylation as training features for a set of machine learning (ML) models designed to enhance the predictive capacity of sperm parameters. Samples were classified based on their progressive motility and DNA methylation features, exhibiting significant differences in total and progressive motility, curvilinear velocity (VCL), velocity of the average path (VAP), linear velocity (VSL), amplitude of lateral head displacement (ALH), beat-cross frequency (BCF), and live normal sperm cells in favour of fast motility ones. Additionally, there were significant differences in enzyme activities for AP and CK, with correlations to LDH and GGT levels. Although motility showed no correlation with total DNA methylation, ALH, wobble of the curvilinear trajectory (WOB), and VCL were significantly different in the newly introduced classification for "suggested good quality", where both motility and methylation were high. The performance differences observed while training various ML classifiers using different feature subsets highlight the importance of DNA methylation for achieving more accurate sample quality classification, even though there is no correlation between motility and DNA methylation. The parameters ALH, VCL, triton extracted LDH, and VAP were top-ranking for "suggested good quality" predictions by the neural network and gradient boosting models. In conclusion, integrating non-kinetic parameters into machine-learning-based sample classification offers a promising approach for selecting kinetically and morphologically superior duck sperm samples that might otherwise be hindered by a predominance of lowly methylated cells.

A Data-Driven Machine Learning Algorithm for Predicting the Outcomes of NBA Games

We propose a new, data-driven model for the prediction of the outcomes of NBA and possibly other basketball league games by using machine learning methods. The paper starts with a strict mathematical formulation of the basketball statistical quantities and the performance indicators derived from them. The backbone of our model is the extended team efficiency index, which consists of two asymmetric parts: (i) the team efficiency index, generally based on some individual efficiency index—in our case, the NBA player efficiency index, and (ii) the comparing part, in which the observed team is rewarded for every selected feature in which it outperforms its rival. Based on the average of the past extended indices, the predicted extended indices are calculated symmetrically for both teams competing in the observed future game. The relative value of those indices defines the win function, which predicts the game outcome. The prediction model includes the concept of the optimal time window (OTW) for the training data. The training datasets were extracted from maximally four and the testing datasets from maximally two of the five consecutive observed NBA seasons (2013/2014–2017/2018). The model uses basic, derived, advanced, and league-wise basketball game elements as its features, whose preparation and extraction were briefly discussed. The proposed model was tested for several choices of the training and testing sets’ seasons, without and with OTWs. The average obtained prediction accuracy is around 66%, and the maximal obtained accuracy is around 78%. This is satisfactory and in the range of better results in the works of other authors.

BatTS: a hybrid method for optimizing deep feedforward neural network

Deep feedforward neural networks (DFNNs) have attained remarkable success in almost every computational task. However, the selection of DFNN architecture is still based on handcraft or hit-and-trial methods. Therefore, an essential factor regarding DFNN is about designing its architecture. Unfortunately, creating architecture for DFNN is a very laborious and time-consuming task for performing state-of-art work. This article proposes a new hybrid methodology (BatTS) to optimize the DFNN architecture based on its performance. BatTS is a result of integrating the Bat algorithm, Tabu search (TS), and Gradient descent with a momentum backpropagation training algorithm (GDM). The main features of the BatTS are the following: a dynamic process of finding new architecture based on Bat, the skill to escape from local minima, and fast convergence in evaluating new architectures based on the Tabu search feature. The performance of BatTS is compared with the Tabu search based approach and random trials. The process goes through an empirical evaluation of four different benchmark datasets and shows that the proposed hybrid methodology has improved performance over existing techniques which are mainly random trials.

Quantitative Analysis of Visual Representation of Sign Elements in COVID-19 Context

Visual representation as a means of communication uses elements to build a narrative. We propose using computer analysis to perform a quantitative analysis of the elements used in the visual creations that have been produced in reference to the epidemic, using 927 images compiled from The Covid Art Museum's Instagram account. This process has been carried out with techniques based on deep learning to detect objects contained in each study image. The research reveals the elements that are repeated in images to create narratives and the relations of association that are established in the sample. The predominant discourses in the sample do not show concern for the effects of illness. On the contrary, the impact and effects of confinement, through the prominent presence of elements such as human figures, windows, and buildings, are the most expressed experiences in the creations analyzed.

Machine Learning in the Classification of Pediatric Posterior Fossa Tumors: A Systematic Review

Background: Posterior fossa tumors (PFTs) are a morbid group of central nervous system tumors that most often present in childhood. While early diagnosis is critical to drive appropriate treatment, definitive diagnosis is currently only achievable through invasive tissue collection and histopathological analyses. Machine learning has been investigated as an alternative means of diagnosis. In this systematic review and meta-analysis, we evaluated the primary literature to identify all machine learning algorithms developed to classify and diagnose pediatric PFTs using imaging or molecular data. Methods: Of the 433 primary papers identified in PubMed, EMBASE, and Web of Science, 25 ultimately met the inclusion criteria. The included papers were extracted for algorithm architecture, study parameters, performance, strengths, and limitations. Results: The algorithms exhibited variable performance based on sample size, classifier(s) used, and individual tumor types being investigated. Ependymoma, medulloblastoma, and pilocytic astrocytoma were the most studied tumors with algorithm accuracies ranging from 37.5% to 94.5%. A minority of studies compared the developed algorithm to a trained neuroradiologist, with three imaging-based algorithms yielding superior performance. Common algorithm and study limitations included small sample sizes, uneven representation of individual tumor types, inconsistent performance reporting, and a lack of application in the clinical environment. Conclusions: Artificial intelligence has the potential to improve the speed and accuracy of diagnosis in this field if the right algorithm is applied to the right scenario. Work is needed to standardize outcome reporting and facilitate additional trials to allow for clinical uptake.

Application of Meta-Heuristic Algorithms for Training Neural Networks and Deep Learning Architectures: A Comprehensive Review

The learning process and hyper-parameter optimization of artificial neural networks (ANNs) and deep learning (DL) architectures is considered one of the most challenging machine learning problems. Several past studies have used gradient-based back propagation methods to train DL architectures. However, gradient-based methods have major drawbacks such as stucking at local minimums in multi-objective cost functions, expensive execution time due to calculating gradient information with thousands of iterations and needing the cost functions to be continuous. Since training the ANNs and DLs is an NP-hard optimization problem, their structure and parameters optimization using the meta-heuristic (MH) algorithms has been considerably raised. MH algorithms can accurately formulate the optimal estimation of DL components (such as hyper-parameter, weights, number of layers, number of neurons, learning rate, etc.). This paper provides a comprehensive review of the optimization of ANNs and DLs using MH algorithms. In this paper, we have reviewed the latest developments in the use of MH algorithms in the DL and ANN methods, presented their disadvantages and advantages, and pointed out some research directions to fill the gaps between MHs and DL methods. Moreover, it has been explained that the evolutionary hybrid architecture still has limited applicability in the literature. Also, this paper classifies the latest MH algorithms in the literature to demonstrate their effectiveness in DL and ANN training for various applications. Most researchers tend to extend novel hybrid algorithms by combining MHs to optimize the hyper-parameters of DLs and ANNs. The development of hybrid MHs helps improving algorithms performance and capable of solving complex optimization problems. In general, the optimal performance of the MHs should be able to achieve a suitable trade-off between exploration and exploitation features. Hence, this paper tries to summarize various MH algorithms in terms of the convergence trend, exploration, exploitation, and the ability to avoid local minima. The integration of MH with DLs is expected to accelerate the training process in the coming few years. However, relevant publications in this way are still rare.

Artificial Intelligence-Based Diabetes Diagnosis with Belief Functions Theory

We compared various machine learning (ML) methods, such as the K-nearest neighbor (KNN), support vector machine (SVM), and decision tree and deep learning (DL) methods, like the recurrent neural network, convolutional neural network, long short-term memory (LSTM), and gated recurrent unit (GRU), to determine the ones with the highest precision. These algorithms learn from data and are subject to different imprecisions and uncertainties. The uncertainty arises from the bad reading of data and/or inaccurate sensor acquisition. We studied how these methods may be combined in a fusion classifier to improve their performance. The Dempster–Shafer method, which uses the formalism of belief functions characterized by asymmetry to model nonprecise and uncertain data, is used for classifier fusion. Diagnosis in the medical field is an important step for the early detection of diseases. In this study, the fusion classifiers were used to diagnose diabetes with the required accuracy. The results demonstrated that the fusion classifiers outperformed the individual classifiers as well as those obtained in the literature. The combined LSTM and GRU fusion classifiers achieved the highest accuracy rate of 98%.

Qualitative and Quantitative Detection of Food Adulteration Using a Smart E-Nose

Food adulteration is the most serious problem found in the food industry as it harms people's healths and undermines their beliefs. The present study is focused on designing and developing a smart electronic nose (SE-Nose) for the qualitative and quantitative fast-track detection of food adulteration. The SE-Nose methodology is comprised of a dataset, sample slicing window protocol, normalization, pattern recognition, and output blocks. The dataset pork adulteration in beef is used to validate the SE-Nose methodology. The sample slicing window protocol extracts the early part of the signal. The sample slicing window protocol and pattern recognition models (classification and regression models) together achieved the high-performance and fast-track detection of pork adulteration in beef. With classification models, the qualitative analysis of adulteration is measured, and with regression models, the quantitative analysis of adulteration is measured. An accuracy of 99.996% and an RMSE of 0.02864 were achieved with the SVM classification and regression model. The recognition time in detecting pork adulteration in beef with SVM models is 40 s. With the proposed SE-Nose methodology, the recognition time is reduced by one-third. To validate the classification and regression models, a 10-fold cross-validation method was used.

Review of Artificial Intelligence and Machine Learning Technologies: Classification, Restrictions, Opportunities and Challenges

Artificial intelligence (AI) is an evolving set of technologies used for solving a wide range of applied issues. The core of AI is machine learning (ML)—a complex of algorithms and methods that address the problems of classification, clustering, and forecasting. The practical application of AI&ML holds promising prospects. Therefore, the researches in this area are intensive. However, the industrial applications of AI and its more intensive use in society are not widespread at the present time. The challenges of widespread AI applications need to be considered from both the AI (internal problems) and the societal (external problems) perspective. This consideration will identify the priority steps for more intensive practical application of AI technologies, their introduction, and involvement in industry and society. The article presents the identification and discussion of the challenges of the employment of AI technologies in the economy and society of resource-based countries. The systematization of AI&ML technologies is implemented based on publications in these areas. This systematization allows for the specification of the organizational, personnel, social and technological limitations. This paper outlines the directions of studies in AI and ML, which will allow us to overcome some of the limitations and achieve expansion of the scope of AI&ML applications.

Machine Learning Approach to Support the Detection of Parkinson's Disease in IMU-Based Gait Analysis

The aim of this study was to determine which supervised machine learning (ML) algorithm can most accurately classify people with Parkinson’s disease (pwPD) from speed-matched healthy subjects (HS) based on a selected minimum set of IMU-derived gait features. Twenty-two gait features were extrapolated from the trunk acceleration patterns of 81 pwPD and 80 HS, including spatiotemporal, pelvic kinematics, and acceleration-derived gait stability indexes. After a three-level feature selection procedure, seven gait features were considered for implementing five ML algorithms: support vector machine (SVM), artificial neural network, decision trees (DT), random forest (RF), and K-nearest neighbors. Accuracy, precision, recall, and F1 score were calculated. SVM, DT, and RF showed the best classification performances, with prediction accuracy higher than 80% on the test set. The conceptual model of approaching ML that we proposed could reduce the risk of overrepresenting multicollinear gait features in the model, reducing the risk of overfitting in the test performances while fostering the explainability of the results.

Multi-Temporal Landsat-8 Images for Retrieval and Broad Scale Mapping of Soil Copper Concentration Using Empirical Models

Mapping soil heavy metal concentration using machine learning models based on readily available satellite remote sensing images is highly desirable. Accurate mapping relies on appropriate data, feature extraction, and model selection. To this end, a data processing pipeline for soil copper (Cu) concentration estimation has been designed. First, instead of using single Landsat scenes, the utilization of multiple Landsat scenes of the same location over time is considered. Second, to generate a preferred feature set as input to a regression model, a number of feature extraction methods are motivated and compared. Third, to find a preferred regression model, a variety of approaches are implemented and compared for accuracy. In this research, 11 Landsat-8 images from 2013 to 2017 of Gulin County, Sichuan China, and 138 soil samples with lab-measured Cu concentrations collected from the area in 2015 are used. A variety a metrics under cross-validation are used for comparison. The results indicate that multi-temporal images increase accuracy compared to single Landsat images. The preferred feature extraction varies based on the regression model used; however, the best results are obtained using support vector regression and the original data. The final soil Cu map generated using the recommended data processing pipeline shows a consistent spatial pattern with a ground-truth land cover classification map. These results indicate that machine learning has the ability to perform large-scale soil heavy metal mapping from widely available satellite remote sensing images.

On Adaptive Learning Framework for Deep Weighted Sparse Autoencoder: A Multiobjective Evolutionary Algorithm

In this article, an adaptive learning framework is established for a deep weighted sparse autoencoder (AE) by resorting to the multiobjective evolutionary algorithm (MOEA). The weighted sparsity is introduced to facilitate the design of the varying degrees of the sparsity constraints imposed on the hidden units of the AE. The MOEA is exploited to adaptively seek appropriate hyperparameters, where the divide-and-conquer strategy is implemented to enhance the MOEA's performance in the context of deep neural networks. Moreover, a sharing scheme is proposed to further reduce the time complexity of the learning process at the slight expense of the learning precision. It is shown via extensive experiments that the established adaptive learning framework is effective, where different sparse models are utilized to demonstrate the generality of the proposed results. Then, the generality of the proposed framework is examined on the convolutional AE and VGG-16 network. Finally, the developed framework is applied to the blind image quantity assessment that illustrates the applicability of the established algorithms.

Fast and automated biomarker detection in breath samples with machine learning

Volatile organic compounds (VOCs) in human breath can reveal a large spectrum of health conditions and can be used for fast, accurate and non-invasive diagnostics. Gas chromatography-mass spectrometry (GC-MS) is used to measure VOCs, but its application is limited by expert-driven data analysis that is time-consuming, subjective and may introduce errors. We propose a machine learning-based system to perform GC-MS data analysis that exploits deep learning pattern recognition ability to learn and automatically detect VOCs directly from raw data, thus bypassing expert-led processing. We evaluate this new approach on clinical samples and with four types of convolutional neural networks (CNNs): VGG16, VGG-like, densely connected and residual CNNs. The proposed machine learning methods showed to outperform the expert-led analysis by detecting a significantly higher number of VOCs in just a fraction of time while maintaining high specificity. These results suggest that the proposed novel approach can help the large-scale deployment of breath-based diagnosis by reducing time and cost, and increasing accuracy and consistency.

A Supervised ML Applied Classification Model for Brain Tumors MRI

Brain Tumor originates from abnormal cells, which is developed uncontrollably. Magnetic resonance imaging (MRI) is developed to generate high-quality images and provide extensive medical research information. The machine learning algorithms can improve the diagnostic value of MRI to obtain automation and accurate classification of MRI. In this research, we propose a supervised machine learning applied training and testing model to classify and analyze the features of brain tumors MRI in the performance of accuracy, precision, sensitivity and F1 score. The result presents that more than 95% accuracy is obtained in this model. It can be used to classify features more accurate than other existing methods.

Hybrid Decision Based on DNN and DTC for Model Predictive Torque Control of PMSM

To address the issue of poor real-time performance caused by the heavy computational burden of the finite control set model predictive torque control (MPTC) of a permanent magnet synchronous motor (PMSM), a data-driven control method using a deep neural network (DNN) is proposed in this paper. The DNN can learn the MPTC’s selective laws from its operation data by training offline and then substitute them for voltage vector selection online. Aiming to address the data-driven runaway problems caused by the asymmetry between the dynamic and static training data, a hybrid decision control strategy based on DNN and DTC (direct torque control) is further proposed, which can realize four-quadrant operation with a control effect basically equivalent to MPTC. The proposed strategy has great application potential for use in multi-level inverter and matrix converter driving with multiple candidate voltage vectors or multi-step prediction.

Combining Unsupervised Approaches for Near Real-Time Network Traffic Anomaly Detection

The 0-day attack is a cyber-attack based on vulnerabilities that have not yet been published. The detection of anomalous traffic generated by such attacks is vital, as it can represent a critical problem, both in a technical and economic sense, for a smart enterprise as for any system largely dependent on technology. To predict this kind of attack, one solution can be to use unsupervised machine learning approaches, as they guarantee the detection of anomalies regardless of their prior knowledge. It is also essential to identify the anomalous and unknown behaviors that occur within a network in near real-time. Three different approaches have been proposed and benchmarked in exactly the same condition: Deep Autoencoding with GMM and Isolation Forest, Deep Autoencoder with Isolation Forest, and Memory Augmented Deep Autoencoder with Isolation Forest. These approaches are thus the result of combining different unsupervised algorithms. The results show that the addition of the Isolation Forest improves the accuracy values and increases the inference time, although this increase does not represent a relevant problematic factor. This paper also explains the features that the various models consider most important for classifying an event as an attack using the explainable artificial intelligence methodology called Shapley Additive Explanations (SHAP). Experiments were conducted on KDD99, NSL-KDD, and CIC-IDS2017 datasets.

Intelligent Traffic Management in Next-Generation Networks

The recent development of smart devices has lead to an explosion in data generation and heterogeneity. Hence, current networks should evolve to become more intelligent, efficient, and most importantly, scalable in order to deal with the evolution of network traffic. In recent years, network softwarization has drawn significant attention from both industry and academia, as it is essential for the flexible control of networks. At the same time, machine learning (ML) and especially deep learning (DL) methods have also been deployed to solve complex problems without explicit programming. These methods can model and learn network traffic behavior using training data/environments. The research community has advocated the application of ML/DL in softwarized environments for network traffic management, including traffic classification, prediction, and anomaly detection. In this paper, we survey the state of the art on these topics. We start by presenting a comprehensive background beginning from conventional ML algorithms and DL and follow this with a focus on different dimensionality reduction techniques. Afterward, we present the study of ML/DL applications in sofwarized environments. Finally, we highlight the issues and challenges that should be considered.

LC-MS peak assignment based on unanimous selection by six machine learning algorithms

Recent mass spectrometry (MS)-based techniques enable deep proteome coverage with relative quantitative analysis, resulting in increased identification of very weak signals accompanied by increased data size of liquid chromatography (LC)–MS/MS spectra. However, the identification of weak signals using an assignment strategy with poorer performance results in imperfect quantification with misidentification of peaks and ratio distortions. Manually annotating a large number of signals within a very large dataset is not a realistic approach. In this study, therefore, we utilized machine learning algorithms to successfully extract a higher number of peptide peaks with high accuracy and precision. Our strategy evaluated each peak identified using six different algorithms; peptide peaks identified by all six algorithms (i.e., unanimously selected) were subsequently assigned as true peaks, which resulted in a reduction in the false-positive rate. Hence, exact and highly quantitative peptide peaks were obtained, providing better performance than obtained applying the conventional criteria or using a single machine learning algorithm.

Machine Learning Techniques for Differential Diagnosis of Vertigo and Dizziness: A Review

Vertigo is a sensation of movement that results from disorders of the inner ear balance organs and their central connections, with aetiologies that are often benign and sometimes serious. An individual who develops vertigo can be effectively treated only after a correct diagnosis of the underlying vestibular disorder is reached. Recent advances in artificial intelligence promise novel strategies for the diagnosis and treatment of patients with this common symptom. Human analysts may experience difficulties manually extracting patterns from large clinical datasets. Machine learning techniques can be used to visualize, understand, and classify clinical data to create a computerized, faster, and more accurate evaluation of vertiginous disorders. Practitioners can also use them as a teaching tool to gain knowledge and valuable insights from medical data. This paper provides a review of the literatures from 1999 to 2021 using various feature extraction and machine learning techniques to diagnose vertigo disorders. This paper aims to provide a better understanding of the work done thus far and to provide future directions for research into the use of machine learning in vertigo diagnosis.

BH-index: A predictive system based on serum biomarkers and ensemble learning for early colorectal cancer diagnosis in mass screening

BACKGROUND AND OBJECTIVE

Colorectal cancer is one of the most common malignancies among the general population. Artificial Intelligence methodologies based on serum parameters are in continuous development to obtain less expensive tools for highly sensitive diagnoses. This study proposes a predictive system based on serum biomarkers and ensemble learning to predict colorectal cancer presence and the related TNM stage in patients.

METHODS

We have selected 17 significant plasmatic proteins, i.e., Carcinoembryonic Antigen, CA 19-9, CA 125, CA 50, CA 72-4, Tissue Polypeptide Antigen, C-Reactive Protein, Ceruloplasmin, Haptoglobin, Transferrin, Ferritin, α-1-Antitrypsin, α-2-Macroglobulin, α-1 Acid Glycoprotein, Complement C4, Complement C3, and Retinol Binding Protein, regarding 345 patients (248 affected by the neoplastic disease). The proposed system consists of two predictors, i.e., binary and staging; the former predicts the presence/absence of cancer, while the latter identifies the related TNM stage (I, II, III, or IV). The experiments were conducted by deploying and comparing Random Forest, XGBoost, Support Vector Machine, and Multilayer Perceptron with feature selection based on Gini Importance and with dimensionality reduction via PCA.

RESULTS

The results show that the system composed of XGBoost as binary and staging predictor reaches 91.30% accuracy, 90% sensitivity, and 93.33% specificity for the absence/presence outcome, while 66.66% accuracy for the staging response. With the expansion of the training set in favor of positive patients and majority voting, the system composed of the combination of Support Vector Machine, XGBoost, and Multilayer Perceptron as the binary predictor reaches 98.03% accuracy, 100% sensitivity, and 92.30% specificity, while the combination of Random Forest, XGBoost, and Multilayer Perceptron as staging predictor achieves 60% accuracy. The final system reaches, in terms of accuracy, 98.03%, and 66.66% for the binary and staging predictors, respectively. It was also found that the biomarkers which contribute most to the binary decision are Ceruloplasmin and α-2-Macroglobulin, while the least significant dimensions are CA 50 and α-1-Antitrypsin; instead, Carcinoembryonic Antigen and α-1 Acid Glycoprotein are the most significant to the staging decision.

CONCLUSIONS

The present study proves the effectiveness of deploying serum biomarkers as feature dimensions for early colorectal cancer diagnosis and of using majority voting for noise reduction in the prediction.

Cortical and phase rim lesions on 7 T MRI as markers of multiple sclerosis disease progression

In multiple sclerosis, individual lesion-type patterns on magnetic resonance imaging might be valuable for predicting clinical outcome and monitoring treatment effects. Neuropathological and imaging studies consistently show that cortical lesions contribute to disease progression. The presence of chronic active white matter lesions harbouring a paramagnetic rim on susceptibility-weighted magnetic resonance imaging has also been associated with an aggressive form of multiple sclerosis. It is, however, still uncertain how these two types of lesions relate to each other, or which one plays a greater role in disability progression. In this prospective, longitudinal study in 100 multiple sclerosis patients (74 relapsing-remitting, 26 secondary progressive), we used ultra-high field 7-T susceptibility imaging to characterize cortical and rim lesion presence and evolution. Clinical evaluations were obtained over a mean period of 3.2 years in 71 patients, 46 of which had a follow-up magnetic resonance imaging. At baseline, cortical and rim lesions were identified in 96% and 63% of patients, respectively. Rim lesion prevalence was similar across disease stages. Patients with rim lesions had higher cortical and overall white matter lesion load than subjects without rim lesions (P = 0.018-0.05). Altogether, cortical lesions increased by both count and volume (P = 0.004) over time, while rim lesions expanded their volume (P = 0.023) whilst lacking new rim lesions; rimless white matter lesions increased their count but decreased their volume (P = 0.016). We used a modern machine learning algorithm based on extreme gradient boosting techniques to assess the cumulative power as well as the individual importance of cortical and rim lesion types in predicting disease stage and disability progression, alongside with more traditional imaging markers. The most influential imaging features that discriminated between multiple sclerosis stages (area under the curve±standard deviation = 0.82 ± 0.08) included, as expected, the normalized white matter and thalamic volume, white matter lesion volume, but also leukocortical lesion volume. Subarachnoid cerebrospinal fluid and leukocortical lesion volumes, along with rim lesion volume were the most important predictors of Expanded Disability Status Scale progression (area under the curve±standard deviation = 0.69 ± 0.12). Taken together, these results indicate that while cortical lesions are extremely frequent in multiple sclerosis, rim lesion development occurs only in a subset of patients. Both, however, persist over time and relate to disease progression. Their combined assessment is needed to improve the ability of identifying multiple sclerosis patients at risk of progressing disease.

Artificial intelligence explainability: the technical and ethical dimensions

In recent years, several new technical methods have been developed to make AI-models more transparent and interpretable. These techniques are often referred to collectively as 'AI explainability' or 'XAI' methods. This paper presents an overview of XAI methods, and links them to stakeholder purposes for seeking an explanation. Because the underlying stakeholder purposes are broadly ethical in nature, we see this analysis as a contribution towards bringing together the technical and ethical dimensions of XAI. We emphasize that use of XAI methods must be linked to explanations of human decisions made during the development life cycle. Situated within that wider accountability framework, our analysis may offer a helpful starting point for designers, safety engineers, service providers and regulators who need to make practical judgements about which XAI methods to employ or to require. This article is part of the theme issue 'Towards symbiotic autonomous systems'.

Deep learning approaches to pattern extraction and recognition in paintings and drawings: an overview

This paper provides an overview of some of the most relevant deep learning approaches to pattern extraction and recognition in visual arts, particularly painting and drawing. Recent advances in deep learning and computer vision, coupled with the growing availability of large digitized visual art collections, have opened new opportunities for computer science researchers to assist the art community with automatic tools to analyse and further understand visual arts. Among other benefits, a deeper understanding of visual arts has the potential to make them more accessible to a wider population, ultimately supporting the spread of culture.

A new approach to monitor water quality in the Menor sea (Spain) using satellite data and machine learning methods

The Menor sea is a coastal lagoon declared by the European Union as a sensitive area to eutrophication due to human activities. To control the deterioration of its water quality, it is necessary to monitor some parameters such as chlorophyll-a (chl-a), which indicates phytoplankton biomass in the water. In the study area, current efforts focus on in-situ measurements to estimate chl-a by means of a few permanent stations and seasonal oceanographic campaigns, however they are expensive and time consuming. In this work, we proposed a machine learning approach based on Sentinel-2 data to estimate chl-a content on the upper part of the water column. Random forest (rf), support vector machine (svmRadial), Artificial Neural Network (ANN) and Deep Neural Network (DNN) algorithms were utilized under three feature selection scenarios, and several spectral indices were used in combination with Sentinel 2 bands. Rf, svmRadial and DNN performed better when all the available predictors were included in the models (RMSE = 0.82, 0.82 and 1.76 mg/m³ respectively), whereas ANN achieved better results under scenario c (principal components). Our results demonstrate the possibility to estimate chl-a concentration in a cost-effective manner and thereby provide near-real time information to monitor the water quality of the Menor sea, what can be of great interest for local authorities, tourism and fishing industry.

A permutation test for assessing the presence of individual differences in treatment effects

An important goal of personalized medicine is to identify heterogeneity in treatment effects and then use that heterogeneity to target the intervention to those most likely to benefit. Heterogeneity is assessed using the predicted individual treatment effects framework, and a permutation test is proposed to establish if significant heterogeneity is present given the covariates and predictive model or algorithm used for predicted individual treatment effects. We first show evidence for heterogeneity in the effects of treatment across an illustrative example data set. We then use simulations with two different predictive methods (linear regression model and Random Forests) to show that the permutation test has adequate type-I error control. Next, we use an example dataset as the basis for simulations to demonstrate the ability of the permutation test to find heterogeneity in treatment effects for a predicted individual treatment effects estimate as a function of both effect size and sample size. We find that the proposed test has good power for detecting heterogeneity in treatment effects when the heterogeneity was due primarily to a single predictor, or when it was spread across the predictors. Power was found to be greater for predictions from a linear model than from random forests. This non-parametric permutation test can be used to test for significant differences across individuals in predicted individual treatment effects obtained with a given set of covariates using any predictive method with no additional assumptions.

A Survey of Stochastic Computing Neural Networks for Machine Learning Applications

Neural networks (NNs) are effective machine learning models that require significant hardware and energy consumption in their computing process. To implement NNs, stochastic computing (SC) has been proposed to achieve a tradeoff between hardware efficiency and computing performance. In an SC NN, hardware requirements and power consumption are significantly reduced by moderately sacrificing the inference accuracy and computation speed. With recent developments in SC techniques, however, the performance of SC NNs has substantially been improved, making it comparable with conventional binary designs yet by utilizing less hardware. In this article, we begin with the design of a basic SC neuron and then survey different types of SC NNs, including multilayer perceptrons, deep belief networks, convolutional NNs, and recurrent NNs. Recent progress in SC designs that further improve the hardware efficiency and performance of NNs is subsequently discussed. The generality and versatility of SC NNs are illustrated for both the training and inference processes. Finally, the advantages and challenges of SC NNs are discussed with respect to binary counterparts.