Learning active subspaces and discovering important features with Gaussian radial basis functions neural networks

Providing a model that achieves a strong predictive performance and is simultaneously interpretable by humans is one of the most difficult challenges in machine learning research due to the conflicting nature of these two objectives. To address this challenge, we propose a modification of the radial basis function neural network model by equipping its Gaussian kernel with a learnable precision matrix. We show that precious information is contained in the spectrum of the precision matrix that can be extracted once the training of the model is completed. In particular, the eigenvectors explain the directions of maximum sensitivity of the model revealing the active subspace and suggesting potential applications for supervised dimensionality reduction. At the same time, the eigenvectors highlight the relationship in terms of absolute variation between the input and the latent variables, thereby allowing us to extract a ranking of the input variables based on their importance to the prediction task enhancing the model interpretability. We conducted numerical experiments for regression, classification, and feature selection tasks, comparing our model against popular machine learning models, the state-of-the-art deep learning-based embedding feature selection techniques, and a transformer model for tabular data. Our results demonstrate that the proposed model does not only yield an attractive prediction performance compared to the competitors but also provides meaningful and interpretable results that potentially could assist the decision-making process in real-world applications. A PyTorch implementation of the model is available on GitHub at the following link.1.

Predicting Visual Acuity Responses to Anti-VEGF Treatment in the Comparison of Age-related Macular Degeneration Treatments Trials Using Machine Learning

PURPOSE

To evaluate multiple machine learning (ML) models for predicting 2-year visual acuity (VA) responses to anti-vascular endothelial growth factor (anti-VEGF) treatment in the Comparison of Age-related Macular Degeneration (AMD) Treatments Trials (CATT) for patients with neovascular AMD (nAMD).

DESIGN

Secondary analysis of public data from a randomized clinical trial.

PARTICIPANTS

A total of 1029 CATT participants who completed 2 years of follow-up with untreated active nAMD and baseline VA between 20/25 and 20/320 in the study eye.

METHODS

Five ML models (support vector machine, random forest, extreme gradient boosting, multilayer perceptron neural network, and lasso) were applied to clinical and image data from baseline and weeks 4, 8, and 12 for predicting 4 VA outcomes (≥ 15-letter VA gain, ≥ 15-letter VA loss, VA change from baseline, and actual VA) at 2 years. The CATT data from 1029 participants were randomly split for training (n = 717), from which the models were trained using 10-fold cross-validation, and for final validation on a test data set (n = 312).

MAIN OUTCOME MEASURES

Performances of ML models were assessed by R2 and mean absolute error (MAE) for predicting VA change from baseline and actual VA at 2 years, by the area under the receiver operating characteristic curve (AUC) for predicting ≥ 15-letter VA gain and loss from baseline.

RESULTS

Using training data up to week 12, the ML models from cross-validation achieved mean R2 of 0.24 to 0.29 (MAE = 9.1-9.8 letters) for predicting VA change and 0.37 to 0.41 (MAE = 9.3-10.2 letters) for predicting actual VA at 2 years. The mean AUCs for predicting ≥ 15-letter VA gain and loss at 2 years was 0.84 to 0.85 and 0.58 to 0.73, respectively. In final validation on the test data set up to week 12, the models had an R2 of 0.33 to 0.38 (MAE = 8.9-9.9 letters) for predicting VA change, an R2 of 0.37 to 0.45 (MAE = 8.8-10.2 letters) for predicting actual VA at 2 years, and AUCs of 0.85 to 0.87 and 0.67 to 0.79 for predicting ≥ 15-letter VA gain and loss, respectively.

CONCLUSIONS

Machine learning models have the potential to predict 2-year VA response to anti-VEGF treatment using clinical and imaging features from the loading dose phase, which can aid in decision-making around treatment protocols for patients with nAMD.

FINANCIAL DISCLOSURE(S)

The author(s) have no proprietary or commercial interest in any materials discussed in this article.

Machine Learning Aided Design and Optimization of Thermal Metamaterials

Artificial Intelligence (AI) has advanced material research that were previously intractable, for example, the machine learning (ML) has been able to predict some unprecedented thermal properties. In this review, we first elucidate the methodologies underpinning discriminative and generative models, as well as the paradigm of optimization approaches. Then, we present a series of case studies showcasing the application of machine learning in thermal metamaterial design. Finally, we give a brief discussion on the challenges and opportunities in this fast developing field. In particular, this review provides: (1) Optimization of thermal metamaterials using optimization algorithms to achieve specific target properties. (2) Integration of discriminative models with optimization algorithms to enhance computational efficiency. (3) Generative models for the structural design and optimization of thermal metamaterials.

The DIRAC framework: Geometric structure underlies roles of diversity and accuracy in combining classifiers

Combining classification systems potentially improves predictive accuracy, but outcomes have proven impossible to predict. Similar to improving binary classification with fusion, fusing ranking systems most commonly increases Pearson or Spearman correlations with a target when the input classifiers are "sufficiently good" (generalized as "accuracy") and "sufficiently different" (generalized as "diversity"), but the individual and joint quantitative influence of these factors on the final outcome remains unknown. We resolve these issues. Building on our previous empirical work establishing the DIRAC (DIversity of Ranks and ACcuracy) framework, which accurately predicts the outcome of fusing binary classifiers, we demonstrate that the DIRAC framework similarly explains the outcome of fusing ranking systems. Specifically, precise geometric representation of diversity and accuracy as angle-based distances within rank-based combinatorial structures (permutahedra) fully captures their synergistic roles in rank approximation, uncouples them from the specific metrics of a given problem, and represents them as generally as possible.

Effects of geographical origin, vintage, and soil on stable isotopes and mineral elements in Ecolly grape berries for traceability

Stable isotopes and multi-element profiles of grapes and corresponding soils from different origins and vintages were determined by IRMS and ICP-MS, respectively. Stable isotope ratios and multi-element contents show significant differences among distinct regions and vintages. Grapes and soils were separated using δ2H and δ18O according to regions and vintages. PCA and CA results further verified that multi-element profiles were influenced by origins and vintages. In particular, δ2H, δ18O, and 21 elements in grapes were correlated with those in soil. Redundancy and Spearman analyses revealed that the BCF values were related to the longitude, latitude, altitude, precipitation, and average temperature. RF shows better performance than PLS-DA for discriminating grape origins and vintages. K, Tb, Cs, δ2H, and Co were important variables in discriminating grape origins. These findings confirm that isotopic and elemental profiles depend on the origin, vintage, and soil, establishing a promising method to discriminate grape origins and vintages.

Topological learning in multiclass data sets

We specialize techniques from topological data analysis to the problem of characterizing the topological complexity (as defined in the body of the paper) of a multiclass data set. As a by-product, a topological classifier is defined that uses an open subcovering of the data set. This subcovering can be used to construct a simplicial complex whose topological features (e.g., Betti numbers) provide information about the classification problem. We use these topological constructs to study the impact of topological complexity on learning in feedforward deep neural networks (DNNs). We hypothesize that topological complexity is negatively correlated with the ability of a fully connected feedforward deep neural network to learn to classify data correctly. We evaluate our topological classification algorithm on multiple constructed and open-source data sets. We also validate our hypothesis regarding the relationship between topological complexity and learning in DNN's on multiple data sets.

Enhancing predictive performance for spectroscopic studies in wildlife science through a multi-model approach: A case study for species classification of live amphibians

Near infrared spectroscopy coupled with predictive modeling is a growing field of study for addressing questions in wildlife science aimed at improving management strategies and conservation outcomes for managed and threatened fauna. To date, the majority of spectroscopic studies in wildlife and fisheries applied chemometrics and predictive modeling with a single-algorithm approach. By contrast, multi-model approaches are used routinely for analyzing spectroscopic datasets across many major industries (e.g., medicine, agriculture) to maximize predictive outcomes for real-world applications. In this study, we conducted a benchmark modeling exercise to compare the performance of several machine learning algorithms in a multi-class problem utilizing a multivariate spectroscopic dataset obtained from live animals. Spectra obtained from live individuals representing eleven amphibian species were classified according to taxonomic designation. Seven modeling techniques were applied to generate prediction models, which varied significantly (p < 0.05) with regard to mean classification accuracy (e.g., support vector machine: 95.8 ± 0.8% vs. K-nearest neighbors: 89.3 ± 1.0%). Through the use of a multi-algorithm approach, candidate algorithms can be identified and applied to more effectively model complex spectroscopic data collected for wildlife sciences. Other key considerations in the predictive modeling workflow that serve to optimize spectroscopic model performance (e.g., variable selection and cross-validation procedures) are also discussed.

Research on eight machine learning algorithms applicability on different characteristics data sets in medical classification tasks

With the vigorous development of data mining field, more and more algorithms have been proposed or improved. How to quickly select a data mining algorithm that is suitable for data sets in medical field is a challenge for some medical workers. The purpose of this paper is to study the comparative characteristics of the general medical data set and the general data sets in other fields, and find the applicability rules of the data mining algorithm suitable for the characteristics of the current research data set. The study quantified characteristics of the research data set with 26 indicators, including simple indicators, statistical indicators and information theory indicators. Eight machine learning algorithms with high maturity, low user involvement and strong family representation were selected as the base algorithms. The algorithm performances were evaluated by three aspects: prediction accuracy, running speed and memory consumption. By constructing decision tree and stepwise regression model to learn the above metadata, the algorithm applicability knowledge of medical data set is obtained. Through cross-verification, the accuracy of all the algorithm applicability prediction models is above 75%, which proves the validity and feasibility of the applicability knowledge.

Designing optimal behavioral experiments using machine learning

Computational models are powerful tools for understanding human cognition and behavior. They let us express our theories clearly and precisely and offer predictions that can be subtle and often counter-intuitive. However, this same richness and ability to surprise means our scientific intuitions and traditional tools are ill-suited to designing experiments to test and compare these models. To avoid these pitfalls and realize the full potential of computational modeling, we require tools to design experiments that provide clear answers about what models explain human behavior and the auxiliary assumptions those models must make. Bayesian optimal experimental design (BOED) formalizes the search for optimal experimental designs by identifying experiments that are expected to yield informative data. In this work, we provide a tutorial on leveraging recent advances in BOED and machine learning to find optimal experiments for any kind of model that we can simulate data from, and show how by-products of this procedure allow for quick and straightforward evaluation of models and their parameters against real experimental data. As a case study, we consider theories of how people balance exploration and exploitation in multi-armed bandit decision-making tasks. We validate the presented approach using simulations and a real-world experiment. As compared to experimental designs commonly used in the literature, we show that our optimal designs more efficiently determine which of a set of models best account for individual human behavior, and more efficiently characterize behavior given a preferred model. At the same time, formalizing a scientific question such that it can be adequately addressed with BOED can be challenging and we discuss several potential caveats and pitfalls that practitioners should be aware of. We provide code to replicate all analyses as well as tutorial notebooks and pointers to adapt the methodology to different experimental settings.

Machine-learning-guided Directed Evolution for AAV Capsid Engineering

Target gene delivery is crucial to gene therapy. Adeno-associated virus (AAV) has emerged as a primary gene therapy vector due to its broad host range, long-term expression, and low pathogenicity. However, AAV vectors have some limitations, such as immunogenicity and insufficient targeting. Designing or modifying capsids is a potential method of improving the efficacy of gene delivery, but hindered by weak biological basis of AAV, complexity of the capsids, and limitations of current screening methods. Artificial intelligence (AI), especially machine learning (ML), has great potential to accelerate and improve the optimization of capsid properties as well as decrease their development time and manufacturing costs. This review introduces the traditional methods of designing AAV capsids and the general steps of building a sequence-function ML model, highlights the applications of ML in the development workflow, and summarizes its advantages and challenges.

A simulation-driven supervised learning framework to estimate brain microstructure using diffusion MRI

We propose a framework to train supervised learning models on synthetic data to estimate brain microstructure parameters using diffusion magnetic resonance imaging (dMRI). Although further validation is necessary, the proposed framework aims to seamlessly incorporate realistic simulations into dMRI microstructure estimation. Synthetic data were generated from over 1,000 neuron meshes converted from digital neuronal reconstructions and linked to their neuroanatomical parameters (such as soma volume and neurite length) using an optimized diffusion MRI simulator that produces intracellular dMRI signals from the solution of the Bloch-Torrey partial differential equation. By combining random subsets of simulated neuron signals with a free diffusion compartment signal, we constructed a synthetic dataset containing dMRI signals and 40 tissue microstructure parameters of 1.45 million artificial brain voxels. To implement supervised learning models we chose multilayer perceptrons (MLPs) and trained them on a subset of the synthetic dataset to estimate some microstructure parameters, namely, the volume fractions of soma, neurites, and the free diffusion compartment, as well as the area fractions of soma and neurites. The trained MLPs perform satisfactorily on the synthetic test sets and give promising in-vivo parameter maps on the MGH Connectome Diffusion Microstructure Dataset (CDMD). Most importantly, the estimated volume fractions showed low dependence on the diffusion time, the diffusion time independence of the estimated parameters being a desired property of quantitative microstructure imaging. The synthetic dataset we generated will be valuable for the validation of models that map between the dMRI signals and microstructure parameters. The surface meshes and microstructures parameters of the aforementioned neurons have been made publicly available.

Deep Features from Pretrained Networks Do Not Outperform Hand-Crafted Features in Radiomics

In radiomics, utilizing features extracted from pretrained deep networks could result in models with a higher predictive performance than those relying on hand-crafted features. This study compared the predictive performance of models trained with either deep features, hand-crafted features, or a combination of these features in terms of the area under the receiver-operating characteristic curve (AUC) and other metrics. We trained models on ten radiological datasets using five feature selection methods and three classifiers. Our results indicate that models based on deep features did not show an improved AUC compared to those utilizing hand-crafted features (deep: AUC 0.775, hand-crafted: AUC 0.789; p = 0.28). Including morphological features alongside deep features led to overall improvements in prediction performance for all models (+0.02 gain in AUC; p < 0.001); however, the best model did not benefit from this (+0.003 gain in AUC; p = 0.57). Using all hand-crafted features in addition to the deep features resulted in a further overall improvement (+0.034 in AUC; p < 0.001), but only a minor improvement could be observed for the best model (deep: AUC 0.798, hand-crafted: AUC 0.789; p = 0.92). Furthermore, our results show that models based on deep features extracted from networks pretrained on medical data have no advantage in predictive performance over models relying on features extracted from networks pretrained on ImageNet data. Our study contributes a benchmarking analysis of models trained on hand-crafted and deep features from pretrained networks across multiple datasets. It also provides a comprehensive understanding of their applicability and limitations in radiomics. Our study shows, in conclusion, that models based on features extracted from pretrained deep networks do not outperform models trained on hand-crafted ones.

Current state and prospects of artificial intelligence in allergy

The field of medicine is witnessing an exponential growth of interest in artificial intelligence (AI), which enables new research questions and the analysis of larger and new types of data. Nevertheless, applications that go beyond proof of concepts and deliver clinical value remain rare, especially in the field of allergy. This narrative review provides a fundamental understanding of the core concepts of AI and critically discusses its limitations and open challenges, such as data availability and bias, along with potential directions to surmount them. We provide a conceptual framework to structure AI applications within this field and discuss forefront case examples. Most of these applications of AI and machine learning in allergy concern supervised learning and unsupervised clustering, with a strong emphasis on diagnosis and subtyping. A perspective is shared on guidelines for good AI practice to guide readers in applying it effectively and safely, along with prospects of field advancement and initiatives to increase clinical impact. We anticipate that AI can further deepen our knowledge of disease mechanisms and contribute to precision medicine in allergy.

Survival analysis for pediatric heart transplant patients using a novel machine learning algorithm: A UNOS analysis

BACKGROUND

Impact of pretransplantation risk factors on mortality in the first year after heart transplantation remains largely unknown. Using machine learning algorithms, we selected clinically relevant identifiers that could predict 1-year mortality after pediatric heart transplantation.

METHODS

Data were obtained from the United Network for Organ Sharing Database for years 2010-2020 for patients 0-17 years receiving their first heart transplant (N = 4150). Features were selected using subject experts and literature review. Scikit-Learn, Scikit-Survival, and Tensorflow were used. A train:test split of 70:30 was used. N-repeated k-fold validation was performed (N = 5, k = 5). Seven models were tested, Hyperparameter tuning performed using Bayesian optimization and the concordance index (C-index) was used for model assessment.

RESULTS

A C-index above 0.6 for test data was considered acceptable for survival analysis models. C-indices obtained were 0.60 (Cox proportional hazards), 0.61 (Cox with elastic net), 0.64 (gradient boosting), 0.64 (support vector machine), 0.68 (random forest), 0.66 (component gradient boosting), and 0.54 (survival trees). Machine learning models show an improvement over the traditional Cox proportional hazards model, with random forest performing the best on the test set. Analysis of the feature importance for the gradient boosted model found that the top 5 features were the most recent serum total bilirubin, the travel distance from the transplant center, the patient body mass index, the deceased donor terminal Serum glutamic pyruvic transaminase/Alanine transaminase (SGPT/ALT), and the donor PCO2.

CONCLUSIONS

Combination of machine learning and expert-based methodology of selecting predictors of survival for pediatric heart transplantation provides a reasonable prediction of 1- and 3-year survival outcomes. SHapley Additive exPlanations can be an effective tool for modeling and visualizing nonlinear interactions.

Machine‐learning radiomics to predict bone marrow metastasis of neuroblastoma using magnetic resonance imaging

Background

Neuroblastoma is one common pediatric malignancy notorious for high temporal and spatial heterogeneities. More than half of its patients develop distant metastases involving vascularized organs, especially the bone marrow. It is thus necessary to have an economical, noninvasive method without much radiation for follow‐ups. Radiomics has been used in many cancers to assist accurate diagnosis but not yet in bone marrow metastasis in neuroblastoma.

Methods

A total of 182 patients with neuroblastoma were retrospectively collected and randomly divided into the training and validation sets. Five‐hundred and seventy‐two radiomics features were extracted from magnetic resonance imaging, among which 41 significant ones were selected via T‐test for model development. We attempted 13 machine‐learning algorithms and eventually chose three best‐performed models. The integrative performance evaluations are based on the area under the curves (AUCs), calibration curves, risk deciles plots, and other indexes.

Results

Extreme gradient boosting, random forest (RF), and adaptive boosting were the top three to predict bone marrow metastases in neuroblastoma while RF was the most accurate one. Its AUC was 0.90 (0.86–0.93), F1 score was 0.82, sensitivity was 0.76, and negative predictive value was 0.79 in the training set. The values were 0.82 (0.71–0.93), 0.80, 0.75, and 0.92 in the validation set, respectively.

Conclusions

Radiomics models are likely to contribute more to metastatic diagnoses and the formulation of personalized healthcare strategies in clinics. It has great potential of being a revolutionary method to replace traditional interventions in the future.

Predicting Sociodemographic Attributes from Mobile Usage Patterns: Applications and Privacy Implications

When users interact with their mobile devices, they leave behind unique digital footprints that can be viewed as predictive proxies that reveal an array of users' characteristics, including their demographics. Predicting users' demographics based on mobile usage can provide significant benefits for service providers and users, including improving customer targeting, service personalization, and market research efforts. This study uses machine learning algorithms and mobile usage data from 235 demographically diverse users to examine the accuracy of predicting their sociodemographic attributes (age, gender, income, and education) from mobile usage metadata, filling the gap in the current literature by quantifying the predictive power of each attribute and discussing the practical applications and privacy implications. According to the results, gender can be most accurately predicted (balanced accuracy = 0.862) from mobile usage footprints, whereas predicting users' education level is more challenging (balanced accuracy = 0.719). Moreover, the classification models were able to classify users based on whether their age or income was above or below a certain threshold with acceptable accuracy. The study also presents the practical applications of inferring demographic attributes from mobile usage data and discusses the implications of the findings, such as privacy and discrimination risks, from the perspectives of different stakeholders.

Diagnosis of Obstructive Sleep Apnea Using Feature Selection, Classification Methods, and Data Grouping Based Age, Sex, and Race

Obstructive sleep apnea (OSA) is a prevalent sleep disorder that affects approximately 3-7% of males and 2-5% of females. In the United States alone, 50-70 million adults suffer from various sleep disorders. OSA is characterized by recurrent episodes of breathing cessation during sleep, thereby leading to adverse effects such as daytime sleepiness, cognitive impairment, and reduced concentration. It also contributes to an increased risk of cardiovascular conditions and adversely impacts patient overall quality of life. As a result, numerous researchers have focused on developing automated detection models to identify OSA and address these limitations effectively and accurately. This study explored the potential benefits of utilizing machine learning methods based on demographic information for diagnosing the OSA syndrome. We gathered a comprehensive dataset from the Torr Sleep Center in Corpus Christi, Texas, USA. The dataset comprises 31 features, including demographic characteristics such as race, age, sex, BMI, Epworth score, M. Friedman tongue position, snoring, and more. We devised a novel process encompassing pre-processing, data grouping, feature selection, and machine learning classification methods to achieve the research objectives. The classification methods employed in this study encompass decision tree (DT), naive Bayes (NB), k-nearest neighbor (kNN), support vector machine (SVM), linear discriminant analysis (LDA), logistic regression (LR), and subspace discriminant (Ensemble) classifiers. Through rigorous experimentation, the results indicated the superior performance of the optimized kNN and SVM classifiers for accurately classifying sleep apnea. Moreover, significant enhancements in model accuracy were observed when utilizing the selected demographic variables and employing data grouping techniques. For instance, the accuracy percentage demonstrated an approximate improvement of 4.5%, 5%, and 10% with the feature selection approach when applied to the grouped data of Caucasians, females, and individuals aged 50 or below, respectively. Furthermore, a comparison with prior studies confirmed that effective data grouping and proper feature selection yielded superior performance in OSA detection when combined with an appropriate classification method. Overall, the findings of this research highlight the importance of leveraging demographic information, employing proper feature selection techniques, and utilizing optimized classification models for accurate and efficient OSA diagnosis.

Lightweight and effective tensor sensitivity for atomistic neural networks

Atomistic machine learning focuses on the creation of models that obey fundamental symmetries of atomistic configurations, such as permutation, translation, and rotation invariances. In many of these schemes, translation and rotation invariance are achieved by building on scalar invariants, e.g., distances between atom pairs. There is growing interest in molecular representations that work internally with higher rank rotational tensors, e.g., vector displacements between atoms, and tensor products thereof. Here, we present a framework for extending the Hierarchically Interacting Particle Neural Network (HIP-NN) with Tensor Sensitivity information (HIP-NN-TS) from each local atomic environment. Crucially, the method employs a weight tying strategy that allows direct incorporation of many-body information while adding very few model parameters. We show that HIP-NN-TS is more accurate than HIP-NN, with negligible increase in parameter count, for several datasets and network sizes. As the dataset becomes more complex, tensor sensitivities provide greater improvements to model accuracy. In particular, HIP-NN-TS achieves a record mean absolute error of 0.927 kcalmol for conformational energy variation on the challenging COMP6 benchmark, which includes a broad set of organic molecules. We also compare the computational performance of HIP-NN-TS to HIP-NN and other models in the literature.

Bayesian learning for neural networks: an algorithmic survey

The last decade witnessed a growing interest in Bayesian learning. Yet, the technicality of the topic and the multitude of ingredients involved therein, besides the complexity of turning theory into practical implementations, limit the use of the Bayesian learning paradigm, preventing its widespread adoption across different fields and applications. This self-contained survey engages and introduces readers to the principles and algorithms of Bayesian Learning for Neural Networks. It provides an introduction to the topic from an accessible, practical-algorithmic perspective. Upon providing a general introduction to Bayesian Neural Networks, we discuss and present both standard and recent approaches for Bayesian inference, with an emphasis on solutions relying on Variational Inference and the use of Natural gradients. We also discuss the use of manifold optimization as a state-of-the-art approach to Bayesian learning. We examine the characteristic properties of all the discussed methods, and provide pseudo-codes for their implementation, paying attention to practical aspects, such as the computation of the gradients.

SchizoNET: a robust and accurate Margenau-Hill time-frequency distribution based deep neural network model for schizophrenia detection using EEG signals

Objective.Schizophrenia (SZ) is a severe chronic illness characterized by delusions, cognitive dysfunctions, and hallucinations that impact feelings, behaviour, and thinking. Timely detection and treatment of SZ are necessary to avoid long-term consequences. Electroencephalogram (EEG) signals are one form of a biomarker that can reveal hidden changes in the brain during SZ. However, the EEG signals are non-stationary in nature with low amplitude. Therefore, extracting the hidden information from the EEG signals is challenging.Approach.The time-frequency domain is crucial for the automatic detection of SZ. Therefore, this paper presents the SchizoNET model combining the Margenau-Hill time-frequency distribution (MH-TFD) and convolutional neural network (CNN). The instantaneous information of EEG signals is captured in the time-frequency domain using MH-TFD. The time-frequency amplitude is converted to two-dimensional plots and fed to the developed CNN model.Results.The SchizoNET model is developed using three different validation techniques, including holdout, five-fold cross-validation, and ten-fold cross-validation techniques using three separate public SZ datasets (Dataset 1, 2, and 3). The proposed model achieved an accuracy of 97.4%, 99.74%, and 96.35% on Dataset 1 (adolescents: 45 SZ and 39 HC subjects), Dataset 2 (adults: 14 SZ and 14 HC subjects), and Dataset 3 (adults: 49 SZ and 32 HC subjects), respectively. We have also evaluated six performance parameters and the area under the curve to evaluate the performance of our developed model.Significance.The SchizoNET is robust, effective, and accurate, as it performed better than the state-of-the-art techniques. To the best of our knowledge, this is the first work to explore three publicly available EEG datasets for the automated detection of SZ. Our SchizoNET model can help neurologists detect the SZ in various scenarios.

PREDICT-GTN 1: Can we improve the FIGO scoring system in gestational trophoblastic neoplasia?

Gestational trophoblastic neoplasia (GTN) patients are treated according to the eight-variable International Federation of Gynaecology and Obstetrics (FIGO) scoring system, that aims to predict first-line single-agent chemotherapy resistance. FIGO is imperfect with one-third of low-risk patients developing disease resistance to first-line single-agent chemotherapy. We aimed to generate simplified models that improve upon FIGO. Logistic regression (LR) and multilayer perceptron (MLP) modelling (n = 4191) generated six models (M1-6). M1, all eight FIGO variables (scored data); M2, all eight FIGO variables (scored and raw data); M3, nonimaging variables (scored data); M4, nonimaging variables (scored and raw data); M5, imaging variables (scored data); and M6, pretreatment hCG (raw data) + imaging variables (scored data). Performance was compared to FIGO using true and false positive rates, positive and negative predictive values, diagnostic odds ratio, receiver operating characteristic (ROC) curves, Bland-Altman calibration plots, decision curve analysis and contingency tables. M1-6 were calibrated and outperformed FIGO on true positive rate and positive predictive value. Using LR and MLP, M1, M2 and M4 generated small improvements to the ROC curve and decision curve analysis. M3, M5 and M6 matched FIGO or performed less well. Compared to FIGO, most (excluding LR M4 and MLP M5) had significant discordance in patient classification (McNemar's test P < .05); 55-112 undertreated, 46-206 overtreated. Statistical modelling yielded only small gains over FIGO performance, arising through recategorisation of treatment-resistant patients, with a significant proportion of under/overtreatment as the available data have been used a priori to allocate primary chemotherapy. Streamlining FIGO should now be the focus.

Ambient Electromagnetic Radiation as a Predictor of Honey Bee (Apis mellifera) Traffic in Linear and Non-Linear Regression: Numerical Stability, Physical Time and Energy Efficiency

Since bee traffic is a contributing factor to hive health and electromagnetic radiation has a growing presence in the urban milieu, we investigate ambient electromagnetic radiation as a predictor of bee traffic in the hive's vicinity in an urban environment. To that end, we built two multi-sensor stations and deployed them for four and a half months at a private apiary in Logan, UT, USA. to record ambient weather and electromagnetic radiation. We placed two non-invasive video loggers on two hives at the apiary to extract omnidirectional bee motion counts from videos. The time-aligned datasets were used to evaluate 200 linear and 3,703,200 non-linear (random forest and support vector machine) regressors to predict bee motion counts from time, weather, and electromagnetic radiation. In all regressors, electromagnetic radiation was as good a predictor of traffic as weather. Both weather and electromagnetic radiation were better predictors than time. On the 13,412 time-aligned weather, electromagnetic radiation, and bee traffic records, random forest regressors had higher maximum R2 scores and resulted in more energy efficient parameterized grid searches. Both types of regressors were numerically stable.

A Multi-Stage Deep-Learning-Based Vehicle and License Plate Recognition System with Real-Time Edge Inference

Video streaming-based real-time vehicle identification and license plate recognition systems are challenging to design and deploy in terms of real-time processing on edge, dealing with low image resolution, high noise, and identification. This paper addresses these issues by introducing a novel multi-stage, real-time, deep learning-based vehicle identification and license plate recognition system. The system is based on a set of algorithms that efficiently integrate two object detectors, an image classifier, and a multi-object tracker to recognize car models and license plates. The information redundancy of Saudi license plates' Arabic and English characters is leveraged to boost the license plate recognition accuracy while satisfying real-time inference performance. The system optimally achieves real-time performance on edge GPU devices and maximizes models' accuracy by taking advantage of the temporally redundant information of the video stream's frames. The edge device sends a notification of the detected vehicle and its license plate only once to the cloud after completing the processing. The system was experimentally evaluated on vehicles and license plates in real-world unconstrained environments at several parking entrance gates. It achieves 17.1 FPS on a Jetson Xavier AGX edge device with no delay. The comparison between the accuracy on the videos and on static images extracted from them shows that the processing of video streams using this proposed system enhances the relative accuracy of the car model and license plate recognition by 13% and 40%, respectively. This research work has won two awards in 2021 and 2022.

Developing a machine learning algorithm to predict probability of retear and functional outcomes in patients undergoing rotator cuff repair surgery: protocol for a retrospective, multicentre study

INTRODUCTION

The effectiveness of rotator cuff tear repair surgery is influenced by multiple patient-related, pathology-centred and technical factors, which is thought to contribute to the reported retear rates between 17% and 94%. Adequate patient selection is thought to be essential in reaching satisfactory results. However, no clear consensus has been reached on which factors are most predictive of successful surgery. A clinical decision tool that encompassed all aspects is still to be made. Artificial intelligence (AI) and machine learning algorithms use complex self-learning models that can be used to make patient-specific decision-making tools. The aim of this study is to develop and train an algorithm that can be used as an online available clinical prediction tool, to predict the risk of retear in patients undergoing rotator cuff repair.

METHODS AND ANALYSIS

This is a retrospective, multicentre, cohort study using pooled individual patient data from multiple studies of patients who have undergone rotator cuff repair and were evaluated by advanced imaging for healing at a minimum of 6 months after surgery. This study consists of two parts. Part one: collecting all potential factors that might influence retear risks from retrospective multicentre data, aiming to include more than 1000 patients worldwide. Part two: combining all influencing factors into a model that can clinically be used as a prediction tool using machine learning.

ETHICS AND DISSEMINATION

For safe multicentre data exchange and analysis, our Machine Learning Consortium adheres to the WHO regulation 'Policy on Use and Sharing of Data Collected by WHO in Member States Outside the Context of Public Health Emergencies'. The study results will be disseminated through publication in a peer-reviewed journal. Institutional Review Board approval does not apply to the current study protocol.

Machine learning identifies pathophysiologically and prognostically informative phenotypes among patients with mitral regurgitation undergoing transcatheter edge-to-edge repair

AIMS

Patients with mitral regurgitation (MR) present with considerable heterogeneity in cardiac damage depending on underlying aetiology, disease progression, and comorbidities. This study aims to capture their cardiopulmonary complexity by employing a machine-learning (ML)-based phenotyping approach.

METHODS AND RESULTS

Data were obtained from 1426 patients undergoing mitral valve transcatheter edge-to-edge repair (MV TEER) for MR. The ML model was developed using 609 patients (derivation cohort) and validated on 817 patients from two external institutions. Phenotyping was based on echocardiographic data, and ML-derived phenotypes were correlated with 5-year outcomes. Unsupervised agglomerative clustering revealed four phenotypes among the derivation cohort: Cluster 1 showed preserved left ventricular ejection fraction (LVEF; 56.5 ± 7.79%) and regular left ventricular end-systolic diameter (LVESD; 35.2 ± 7.52 mm); 5-year survival in Cluster 1, hereinafter serving as a reference, was 60.9%. Cluster 2 presented with preserved LVEF (55.7 ± 7.82%) but showed the largest mitral valve effective regurgitant orifice area (0.623 ± 0.360 cm2) and highest systolic pulmonary artery pressures (68.4 ± 16.2 mmHg); 5-year survival ranged at 43.7% (P-value: 0.032). Cluster 3 was characterized by impaired LVEF (31.0 ± 10.4%) and enlarged LVESD (53.2 ± 10.9 mm); 5-year survival was reduced to 38.3% (P-value: <0.001). The poorest 5-year survival (23.8%; P-value: <0.001) was observed in Cluster 4 with biatrial dilatation (left atrial volume: 312 ± 113 mL; right atrial area: 46.0 ± 8.83 cm2) although LVEF was only slightly reduced (51.5 ± 11.0%). Importantly, the prognostic significance of ML-derived phenotypes was externally confirmed.

CONCLUSION

ML-enabled phenotyping captures the complexity of extra-mitral valve cardiac damage, which does not necessarily occur in a sequential fashion. This novel phenotyping approach can refine risk stratification in patients undergoing MV TEER in the future.

The prediction of in-hospital mortality in chronic kidney disease patients with coronary artery disease using machine learning models

OBJECTIVE

Chronic kidney disease (CKD) patients with coronary artery disease (CAD) in the intensive care unit (ICU) have higher in-hospital mortality and poorer prognosis than patients with either single condition. The objective of this study is to develop a novel model that can predict the in-hospital mortality of that kind of patient in the ICU using machine learning methods.

METHODS

Data of CKD patients with CAD were extracted from the Medical Information Mart for Intensive Care IV (MIMIC-IV) database. Boruta algorithm was conducted for the feature selection process. Eight machine learning algorithms, such as logistic regression (LR), random forest (RF), Decision Tree, K-nearest neighbors (KNN), Gradient Boosting Decision Tree Machine (GBDT), Support Vector Machine (SVM), Neural Network (NN), and Extreme Gradient Boosting (XGBoost), were conducted to construct the predictive model for in-hospital mortality and performance was evaluated by average precision (AP) and area under the receiver operating characteristic curve (AUC). Shapley Additive Explanations (SHAP) algorithm was applied to explain the model visually. Moreover, data from the Telehealth Intensive Care Unit Collaborative Research Database (eICU-CRD) were acquired as an external validation set.

RESULTS

3590 and 1657 CKD patients with CAD were acquired from MIMIC-IV and eICU-CRD databases, respectively. A total of 78 variables were selected for the machine learning model development process. Comparatively, GBDT had the highest predictive performance according to the results of AUC (0.946) and AP (0.778). The SHAP method reveals the top 20 factors based on the importance ranking. In addition, GBDT had good predictive value and a certain degree of clinical value in the external validation according to the AUC (0.865), AP (0.672), decision curve analysis, and calibration curve.

CONCLUSION

Machine learning algorithms, especially GBDT, can be reliable tools for accurately predicting the in-hospital mortality risk for CKD patients with CAD in the ICU. This contributed to providing optimal resource allocation and reducing in-hospital mortality by tailoring precise management and implementation of early interventions.

Persistent homology classification algorithm

Data classification is an important aspect of machine learning, as it is utilized to solve issues in a wide variety of contexts. There are numerous classifiers, but there is no single best-performing classifier for all types of data, as the no free lunch theorem implies. Topological data analysis is an emerging topic concerned with the shape of data. One of the key tools in this field for analyzing the shape or topological properties of a dataset is persistent homology, an algebraic topology-based method for estimating the topological features of a space of points that persists across several resolutions. This study proposes a supervised learning classification algorithm that makes use of persistent homology between training data classes in the form of persistence diagrams to predict the output category of new observations. Validation of the developed algorithm was performed on real-world and synthetic datasets. The performance of the proposed classification algorithm on these datasets was compared to that of the most widely used classifiers. Validation runs demonstrated that the proposed persistent homology classification algorithm performed at par if not better than the majority of classifiers considered.

Ensemble Methods for APS In-Flight Particle Temperature and Velocity Prediction Considering Torch Electrodes Ageing

The nonlinear relationship between the input process parameters and in-flight particle characteristics of the atmospheric plasma spray (APS) is of paramount importance for coating properties design and quality. It is also known that the ageing of torch electrodes affects this relationship. In recent years, machine learning algorithms have proven to be able to take into account such complex nonlinear interactions. This work illustrates the application of ensemble methods to predict the in-flight particle temperature and velocity during an APS process considering torch electrodes ageing. Experiments were performed to record simultaneously the input process parameters, the in-flight powder particle characteristics and the electrodes usage time. Random Forest (RF) and Gradient Boosting (GB) were used to rank and select the features for the APS process data recorded as the electrodes aged and the corresponding predictive models were compared. The time series aspect of the multivariate APS in-flight particle characteristics data is explored. Two strategies of time series embedding are considered. The first one simply embeds the attributes and the targets from the previous n time segments considered without any modification; whereas the second strategy first performs differencing to make the time series stationary before embedding. For the present application, RF is found to be more suitable than GB since RF can predict both the in-flight particle velocity and temperature simultaneously, properly considering the interactions between the two targets. On the other hand, GB can only predict these two targets one at a time. The superior performance of both embedded predictive models and the feature rankings of them suggest that it is better to consider the APS data as time series for the in-flight particle characteristic prediction. In particular, it is demonstrated that it is advantageous to first make the time series stationary using the traditional differencing technique, even when modeling using RF.

Human health risk assessment for contaminated sites: A retrospective review

Soil contamination is a serious global hazard as contaminants can migrate to the human body through the soil, water, air, and food, threatening human health. Human Health Risk Assessment (HHRA) is a commonly used method for estimating the magnitude and probability of adverse health effects in humans that may be exposed to contaminants in contaminated environmental media in the present or future. Such estimations have improved for decades with various risk assessment frameworks and well-established models. However, the existing literature does not provide a comprehensive overview of the methods and models of HHRA that are needed to grasp the current status of HHRA and future research directions. Thus, this paper aims to systematically review the HHRA approaches and models, particularly those related to contaminated sites from peer-reviewed literature and guidelines. The approaches and models focus on methods used in hazard identification, toxicity databases in dose-response assessment, approaches and fate and transport models in exposure assessment, risk characterization, and uncertainty characterization. The features and applicability of the most commonly used HHRA tools are also described. The future research trend for HHRA for contaminated sites is also forecasted. The transition from animal experiments to new methods in risk identification, the integration and update and sharing of existing toxicity databases, the integration of human biomonitoring into the risk assessment process, and the integration of migration and transformation models and risk assessment are the way forward for risk assessment in the future. This review provides readers with an overall understanding of HHRA and a grasp of its developmental direction.

Predicting Unreported Micronutrients from Food Labels: A Machine Learning Approach (Preprint)

BACKGROUND

Micronutrient deficiencies represent a major global health issue, with over 2 billion individuals experiencing deficiencies in essential vitamins and minerals. Food labels provide consumers with information regarding the nutritional content of food items and have been identified as a potential tool for improving diets. However, due to governmental regulations and the physical limitations of the labels, food labels often lack comprehensive information about the vitamins and minerals present in foods. As a result, information about most of the micronutrients is absent from existing food labels.

OBJECTIVE

This paper aims to examine the possibility of using machine learning algorithms to predict unreported micronutrients such as vitamin A (retinol), vitamin C, vitamin B1 (thiamin), vitamin B2 (riboflavin), vitamin B3 (niacin), vitamin B6, vitamin B12, vitamin E (alpha-tocopherol), vitamin K, and minerals such as magnesium, zinc, phosphorus, selenium, manganese, and copper from nutrition information provided on existing food labels. If unreported micronutrients can be predicted with acceptable accuracies from existing food labels using machine learning predictive models, such models can be integrated into mobile apps to provide consumers with additional micronutrient information about foods and help them make more informed diet decisions.

METHODS

Data from the Food and Nutrient Database for Dietary Studies (FNDDS) data set, representing a total of 5624 foods, were used to train a diverse set of machine learning classification and regression algorithms to predict unreported vitamins and minerals from existing food label data. For each model, hyperparameters were adjusted, and the models were evaluated using repeated cross-validation to ensure that the reported results were not subject to overfitting.

RESULTS

According to the results, while predicting the exact quantity of vitamins and minerals is shown to be challenging, with regression R² varying in a wide range from 0.28 (for magnesium) to 0.92 (for manganese), the classification models can accurately predict the category ("low," "medium," or "high") level of all minerals and vitamins with accuracies exceeding 0.80. The highest classification accuracies for specific micronutrients are achieved for vitamin B12 (0.94) and phosphorus (0.94), while the lowest are for vitamin E (0.81) and selenium (0.83).

CONCLUSIONS

This study demonstrates the feasibility of predicting unreported micronutrients from existing food labels using machine learning algorithms. The results show that the approach has the potential to significantly improve consumer knowledge about the micronutrient content of the foods they consume. Integrating these predictive models into mobile apps can enhance their accessibility and engagement with consumers. The implications of this research for public health are noteworthy, underscoring the potential of technology to augment consumers' understanding of the micronutrient content of their diets while also facilitating the tracking of food intake and providing personalized recommendations based on the micronutrient content and individual preferences.

Machine learning models exploring characteristic single-nucleotide signatures in yellow fever virus

Yellow fever virus (YFV) is the agent of the most severe mosquito-borne disease in the tropics. Recently, Brazil suffered major YFV outbreaks with a high fatality rate affecting areas where the virus has not been reported for decades, consisting of urban areas where a large number of unvaccinated people live. We developed a machine learning framework combining three different algorithms (XGBoost, random forest and regularized logistic regression) to analyze YFV genomic sequences. This method was applied to 56 YFV sequences from human infections and 27 from non-human primate (NHPs) infections to investigate the presence of genetic signatures possibly related to disease severity (in human related sequences) and differences in PCR cycle threshold (Ct) values (in NHP related sequences). Our analyses reveal four non-synonymous single nucleotide variations (SNVs) on sequences from human infections, in proteins NS3 (E614D), NS4a (I69V), NS5 (R727G, V643A) and six non-synonymous SNVs on NHP sequences, in proteins E (L385F), NS1 (A171V), NS3 (I184V) and NS5 (N11S, I374V, E641D). We performed comparative protein structural analysis on these SNVs, describing possible impacts on protein function. Despite the fact that the dataset is limited in size and that this study does not consider virus-host interactions, our work highlights the use of machine learning as a versatile and fast initial approach to genomic data exploration.

Reuse, Reduce, Support: Design Principles for Green Data Mining

This paper reports on a design science research (DSR) study that develops design principles for “green” – more environmentally sustainable – data mining processes. Grounded in the Cross Industry Standard Process for Data Mining (CRISP-DM) and on a review of relevant literature on data mining methods, Green IT, and Green IS, the study identifies eight design principles that fall into the three categories of reuse, reduce, and support. The paper develops an evaluation strategy and provides empirical evidence for the principles’ utility. It suggests that the results can inform the development of a more general approach towards Green Data Science and provide a suitable lens to study sustainable computing.

A Unified Neural Network Framework for Extended Redundancy Analysis

Component-based approaches have been regarded as a tool for dimension reduction to predict outcomes from observed variables in regression applications. Extended redundancy analysis (ERA) is one such component-based approach which reduces predictors to components explaining maximum variance in the outcome variables. In many instances, ERA can be extended to capture nonlinearity and interactions between observed and components, but only by specifying a priori functional form. Meanwhile, machine learning methods like neural networks are typically used in a data-driven manner to capture nonlinearity without specifying the exact functional form. In this paper, we introduce a new method that integrates neural networks algorithms into the framework of ERA, called NN-ERA, to capture any non-specified nonlinear relationships among multiple sets of observed variables for constructing components. Simulations and empirical datasets are used to demonstrate the usefulness of NN-ERA. The conclusion is that in social science datasets with unstructured data, where we expect nonlinear relationships that cannot be specified a priori, NN-ERA with its neural network algorithmic structure can serve as a useful tool to specify and test models otherwise not captured by the conventional component-based models.

Application of Machine Learning in Epileptic Seizure Detection

Epileptic seizure is a neurological condition caused by short and unexpectedly occurring electrical disruptions in the brain. It is estimated that roughly 60 million individuals worldwide have had an epileptic seizure. Experiencing an epileptic seizure can have serious consequences for the patient. Automatic seizure detection on electroencephalogram (EEG) recordings is essential due to the irregular and unpredictable nature of seizures. By thoroughly analyzing EEG records, neurophysiologists can discover important information and patterns, and proper and timely treatments can be provided for the patients. This research presents a novel machine learning-based approach for detecting epileptic seizures in EEG signals. A public EEG dataset from the University of Bonn was used to validate the approach. Meaningful statistical features were extracted from the original data using discrete wavelet transform analysis, then the relevant features were selected using feature selection based on the binary particle swarm optimizer. This facilitated the reduction of 75% data dimensionality and 47% computational time, which eventually sped up the classification process. After having been selected, relevant features were used to train different machine learning models, then hyperparameter optimization was utilized to further enhance the models' performance. The results achieved up to 98.4% accuracy and showed that the proposed method was very effective and practical in detecting seizure presence in EEG signals. In clinical applications, this method could help relieve the suffering of epilepsy patients and alleviate the workload of neurologists.

Non-Targeted Analytical Technology in Herbal Medicines: Applications, Challenges, and Perspectives

Herbal medicines (HMs) have been utilized to prevent and treat human ailments for thousands of years. Especially, HMs have recently played a crucial role in the treatment of COVID-19 in China. However, HMs are susceptible to various factors during harvesting, processing, and marketing, affecting their clinical efficacy. Therefore, it is necessary to conclude a rapid and effective method to study HMs so that they can be used in the clinical setting with maximum medicinal value. Non-targeted analytical technology is a reliable analytical method for studying HMs because of its unique advantages in analyzing unknown components. Based on the extensive literature, the paper summarizes the benefits, limitations, and applicability of non-targeted analytical technology. Moreover, the article describes the application of non-targeted analytical technology in HMs from four aspects: structure analysis, authentication, real-time monitoring, and quality assessment. Finally, the review has prospected the development trend and challenges of non-targeted analytical technology. It can assist HMs industry researchers and engineers select non-targeted analytical technology to analyze HMs' quality and authenticity.

Multiple instance neural networks based on sparse attention for cancer detection using T-cell receptor sequences

Early detection of cancers has been much explored due to its paramount importance in biomedical fields. Among different types of data used to answer this biological question, studies based on T cell receptors (TCRs) are under recent spotlight due to the growing appreciation of the roles of the host immunity system in tumor biology. However, the one-to-many correspondence between a patient and multiple TCR sequences hinders researchers from simply adopting classical statistical/machine learning methods. There were recent attempts to model this type of data in the context of multiple instance learning (MIL). Despite the novel application of MIL to cancer detection using TCR sequences and the demonstrated adequate performance in several tumor types, there is still room for improvement, especially for certain cancer types. Furthermore, explainable neural network models are not fully investigated for this application. In this article, we propose multiple instance neural networks based on sparse attention (MINN-SA) to enhance the performance in cancer detection and explainability. The sparse attention structure drops out uninformative instances in each bag, achieving both interpretability and better predictive performance in combination with the skip connection. Our experiments show that MINN-SA yields the highest area under the ROC curve scores on average measured across 10 different types of cancers, compared to existing MIL approaches. Moreover, we observe from the estimated attentions that MINN-SA can identify the TCRs that are specific for tumor antigens in the same T cell repertoire.

Smart breeding driven by big data, artificial intelligence, and integrated genomic-enviromic prediction

The first paradigm of plant breeding involves direct selection-based phenotypic observation, followed by predictive breeding using statistical models for quantitative traits constructed based on genetic experimental design and, more recently, by incorporation of molecular marker genotypes. However, plant performance or phenotype (P) is determined by the combined effects of genotype (G), envirotype (E), and genotype by environment interaction (GEI). Phenotypes can be predicted more precisely by training a model using data collected from multiple sources, including spatiotemporal omics (genomics, phenomics, and enviromics across time and space). Integration of 3D information profiles (G-P-E), each with multidimensionality, provides predictive breeding with both tremendous opportunities and great challenges. Here, we first review innovative technologies for predictive breeding. We then evaluate multidimensional information profiles that can be integrated with a predictive breeding strategy, particularly envirotypic data, which have largely been neglected in data collection and are nearly untouched in model construction. We propose a smart breeding scheme, integrated genomic-enviromic prediction (iGEP), as an extension of genomic prediction, using integrated multiomics information, big data technology, and artificial intelligence (mainly focused on machine and deep learning). We discuss how to implement iGEP, including spatiotemporal models, environmental indices, factorial and spatiotemporal structure of plant breeding data, and cross-species prediction. A strategy is then proposed for prediction-based crop redesign at both the macro (individual, population, and species) and micro (gene, metabolism, and network) scales. Finally, we provide perspectives on translating smart breeding into genetic gain through integrative breeding platforms and open-source breeding initiatives. We call for coordinated efforts in smart breeding through iGEP, institutional partnerships, and innovative technological support.

Application of Meta-Heuristic Algorithms for Training Neural Networks and Deep Learning Architectures: A Comprehensive Review

The learning process and hyper-parameter optimization of artificial neural networks (ANNs) and deep learning (DL) architectures is considered one of the most challenging machine learning problems. Several past studies have used gradient-based back propagation methods to train DL architectures. However, gradient-based methods have major drawbacks such as stucking at local minimums in multi-objective cost functions, expensive execution time due to calculating gradient information with thousands of iterations and needing the cost functions to be continuous. Since training the ANNs and DLs is an NP-hard optimization problem, their structure and parameters optimization using the meta-heuristic (MH) algorithms has been considerably raised. MH algorithms can accurately formulate the optimal estimation of DL components (such as hyper-parameter, weights, number of layers, number of neurons, learning rate, etc.). This paper provides a comprehensive review of the optimization of ANNs and DLs using MH algorithms. In this paper, we have reviewed the latest developments in the use of MH algorithms in the DL and ANN methods, presented their disadvantages and advantages, and pointed out some research directions to fill the gaps between MHs and DL methods. Moreover, it has been explained that the evolutionary hybrid architecture still has limited applicability in the literature. Also, this paper classifies the latest MH algorithms in the literature to demonstrate their effectiveness in DL and ANN training for various applications. Most researchers tend to extend novel hybrid algorithms by combining MHs to optimize the hyper-parameters of DLs and ANNs. The development of hybrid MHs helps improving algorithms performance and capable of solving complex optimization problems. In general, the optimal performance of the MHs should be able to achieve a suitable trade-off between exploration and exploitation features. Hence, this paper tries to summarize various MH algorithms in terms of the convergence trend, exploration, exploitation, and the ability to avoid local minima. The integration of MH with DLs is expected to accelerate the training process in the coming few years. However, relevant publications in this way are still rare.

Finding a Suitable Class Distribution for Building Histological Images Datasets Used in Deep Model Training-The Case of Cancer Detection

The class distribution of a training dataset is an important factor which influences the performance of a deep learning-based system. Understanding the optimal class distribution is therefore crucial when building a new training set which may be costly to annotate. This is the case for histological images used in cancer diagnosis where image annotation requires domain experts. In this paper, we tackle the problem of finding the optimal class distribution of a training set to be able to train an optimal model that detects cancer in histological images. We formulate several hypotheses which are then tested in scores of experiments with hundreds of trials. The experiments have been designed to account for both segmentation and classification frameworks with various class distributions in the training set, such as natural, balanced, over-represented cancer, and over-represented non-cancer. In the case of cancer detection, the experiments show several important results: (a) the natural class distribution produces more accurate results than the artificially generated balanced distribution; (b) the over-representation of non-cancer/negative classes (healthy tissue and/or background classes) compared to cancer/positive classes reduces the number of samples which are falsely predicted as cancer (false positive); (c) the least expensive to annotate non-ROI (non-region-of-interest) data can be useful in compensating for the performance loss in the system due to a shortage of expensive to annotate ROI data; (d) the multi-label examples are more useful than the single-label ones to train a segmentation model; and (e) when the classification model is tuned with a balanced validation set, it is less affected than the segmentation model by the class distribution of the training set.

Polygenic risk modeling of tumor stage and survival in bladder cancer

Introduction

Bladder cancer assessment with non-invasive gene expression signatures facilitates the detection of patients at risk and surveillance of their status, bypassing the discomforts given by cystoscopy. To achieve accurate cancer estimation, analysis pipelines for gene expression data (GED) may integrate a sequence of several machine learning and bio-statistical techniques to model complex characteristics of pathological patterns.

Methods

Numerical experiments tested the combination of GED preprocessing by discretization with tree ensemble embeddings and nonlinear dimensionality reductions to categorize oncological patients comprehensively. Modeling aimed to identify tumor stage and distinguish survival outcomes in two situations: complete and partial data embedding. This latter experimental condition simulates the addition of new patients to an existing model for rapid monitoring of disease progression. Machine learning procedures were employed to identify the most relevant genes involved in patient prognosis and test the performance of preprocessed GED compared to untransformed data in predicting patient conditions.

Results

Data embedding paired with dimensionality reduction produced prognostic maps with well-defined clusters of patients, suitable for medical decision support. A second experiment simulated the addition of new patients to an existing model (partial data embedding): Uniform Manifold Approximation and Projection (UMAP) methodology with uniform data discretization led to better outcomes than other analyzed pipelines. Further exploration of parameter space for UMAP and t-distributed stochastic neighbor embedding (t-SNE) underlined the importance of tuning a higher number of parameters for UMAP rather than t-SNE. Moreover, two different machine learning experiments identified a group of genes valuable for partitioning patients (gene relevance analysis) and showed the higher precision obtained by preprocessed data in predicting tumor outcomes for cancer stage and survival rate (six classes prediction).

Conclusions

The present investigation proposed new analysis pipelines for disease outcome modeling from bladder cancer-related biomarkers. Complete and partial data embedding experiments suggested that pipelines employing UMAP had a more accurate predictive ability, supporting the recent literature trends on this methodology. However, it was also found that several UMAP parameters influence experimental results, therefore deriving a recommendation for researchers to pay attention to this aspect of the UMAP technique. Machine learning procedures further demonstrated the effectiveness of the proposed preprocessing in predicting patients’ conditions and determined a sub-group of biomarkers significant for forecasting bladder cancer prognosis.

Inductive biases in theory-based reinforcement learning

Understanding the inductive biases that allow humans to learn in complex environments has been an important goal of cognitive science. Yet, while we have discovered much about human biases in specific learning domains, much of this research has focused on simple tasks that lack the complexity of the real world. In contrast, video games involving agents and objects embedded in richly structured systems provide an experimentally tractable proxy for real-world complexity. Recent work has suggested that key aspects of human learning in domains like video games can be captured by model-based reinforcement learning (RL) with object-oriented relational models-what we term theory-based RL. Restricting the model class in this way provides an inductive bias that dramatically increases learning efficiency, but in this paper we show that humans employ a stronger set of biases in addition to syntactic constraints on the structure of theories. In particular, we catalog a set of semantic biases that constrain the content of theories. Building these semantic biases into a theory-based RL system produces more human-like learning in video game environments.

Development and training of a machine learning algorithm to identify patients at risk for recurrence following an arthroscopic Bankart repair (CLEARER): protocol for a retrospective, multicentre, cohort study

INTRODUCTION

Shoulder instability is a common injury, with a reported incidence of 23.9 per 100 000 person-years. There is still an ongoing debate on the most effective treatment strategy. Non-operative treatment has recurrence rates of up to 60%, whereas operative treatments such as the Bankart repair and bone block procedures show lower recurrence rates (16% and 2%, respectively) but higher complication rates (<2% and up to 30%, respectively). Methods to determine risk of recurrence have been developed; however, patient-specific decision-making tools are still lacking. Artificial intelligence and machine learning algorithms use self-learning complex models that can be used to make patient-specific decision-making tools. The aim of the current study is to develop and train a machine learning algorithm to create a prediction model to be used in clinical practice-as an online prediction tool-to estimate recurrence rates following a Bankart repair.

METHODS AND ANALYSIS

This is a multicentre retrospective cohort study. Patients with traumatic anterior shoulder dislocations that were treated with an arthroscopic Bankart repair without remplissage will be included. This study includes two parts. Part 1, collecting all potential factors influencing the recurrence rate following an arthroscopic Bankart repair in patients using multicentre data, aiming to include data from >1000 patients worldwide. Part 2, the multicentre data will be re-evaluated (and where applicable complemented) using machine learning algorithms to predict outcomes. Recurrence will be the primary outcome measure.

ETHICS AND DISSEMINATION

For safe multicentre data exchange and analysis, our Machine Learning Consortium adhered to the WHO regulation 'Policy on Use and Sharing of Data Collected by WHO in Member States Outside the Context of Public Health Emergencies'. The study results will be disseminated through publication in a peer-reviewed journal. No Institutional Review Board is required for this study.

Comparing gradient boosting machine and Bayesian threshold BLUP for genome-based prediction of categorical traits in wheat breeding

Genomic selection (GS) is a predictive methodology that is changing plant breeding. Genomic selection trains a statistical machine-learning model using available phenotypic and genotypic data with which predictions are performed for individuals that were only genotyped. For this reason, some statistical machine-learning methods are being implemented in GS, but in order to improve the selection of new genotypes early in the prediction process, the exploration of new statistical machine-learning algorithms must continue. In this paper, we performed a benchmarking study between the Bayesian threshold genomic best linear unbiased predictor model (TGBLUP; popular in GS) and the gradient boosting machine (GBM). This comparison was done using four real wheat (Triticum aestivum L.) data sets with categorical traits measured in terms of two metrics: the proportion of cases correctly classified (PCCC) and the Kappa coefficient in the testing set. Under 10 random partitions with four different sizes of testing proportions (20, 40, 60, and 80%), we compared the two algorithms and found that in three of the four data sets, the GBM outperformed the TGBLUP model in terms of both metrics (PCCC and Kappa coefficient). In the larger data sets (Data Sets 3 and 4), the gain in terms of prediction accuracy of the GBM was considerably significant. For this reason, we encourage more research using the GBM in GS to evaluate its virtues in terms of prediction performance in the context of GS.