A Limitation of Gradient Descent Learning

Regularization Effect of Random Node Fault/Noise on Gradient Descent Learning Algorithm

For decades, adding fault/noise during training by gradient descent has been a technique for getting a neural network (NN) tolerant to persistent fault/noise or getting an NN with better generalization. In recent years, this technique has been readvocated in deep learning to avoid overfitting. Yet, the objective function of such fault/noise injection learning has been misinterpreted as the desired measure (i.e., the expected mean squared error (mse) of the training samples) of the NN with the same fault/noise. The aims of this article are: 1) to clarify the above misconception and 2) investigate the actual regularization effect of adding node fault/noise when training by gradient descent. Based on the previous works on adding fault/noise during training, we speculate the reason why the misconception appears. In the sequel, it is shown that the learning objective of adding random node fault during gradient descent learning (GDL) for a multilayer perceptron (MLP) is identical to the desired measure of the MLP with the same fault. If additive (resp. multiplicative) node noise is added during GDL for an MLP, the learning objective is not identical to the desired measure of the MLP with such noise. For radial basis function (RBF) networks, it is shown that the learning objective is identical to the corresponding desired measure for all three fault/noise conditions. Empirical evidence is presented to support the theoretical results and, hence, clarify the misconception that the objective function of a fault/noise injection learning might not be interpreted as the desired measure of the NN with the same fault/noise. Afterward, the regularization effect of adding node fault/noise during training is revealed for the case of RBF networks. Notably, it is shown that the regularization effect of adding additive or multiplicative node noise (MNN) during training an RBF is reducing network complexity. Applying dropout regularization in RBF networks, its effect is the same as adding MNN during training.

Machine Learning Can Predict the Probability of Biologic Therapy in Patients with Inflammatory Bowel Disease

Background: Inflammatory bowel disease (IBD) is of high medical and socioeconomic relevance. Moderate and severe disease courses often require treatment with biologics. The aim of this study was to evaluate machine learning (ML)-based methods for the prediction of biologic therapy in IBD patients using a large prescription database. Methods: The present retrospective cohort study utilized a longitudinal prescription database (LRx). Patients with at least one prescription for an intestinal anti-inflammatory agent from a gastroenterologist between January 2015 and July 2021 were included. Patients who had received an initial biologic therapy prescription (infliximab, adalimumab, golimumab, vedolizumab, or ustekinumab) were categorized as the “biologic group”. The potential predictors included in the machine learning-based models were age, sex, and the 100 most frequently prescribed drugs within 12 months prior to the index date. Six machine learning-based methods were used for the prediction of biologic therapy. Results: A total of 122,089 patients were included in this study. Of these, 15,824 (13.0%) received at least one prescription for a biologic drug. The Light Gradient Boosting Machine had the best performance (accuracy = 74%) and was able to correctly identify 78.5% of the biologics patients and 72.6% of the non-biologics patients in the testing dataset. The most important variable was prednisolone, followed by lower age, mesalazine, budesonide, and ferric iron. Conclusions: In summary, this study reveals the advantages of ML-based models in predicting biologic therapy in IBD patients based on pre-treatment and demographic variables. There is a need for further studies in this regard that take into account individual patient characteristics, i.e., genetics and gut microbiota, to adequately address the challenges of finding optimal treatment strategies for patients with IBD.

Deep learning in electron microscopy

Deep learning is transforming most areas of science and technology, including electron microscopy. This review paper offers a practical perspective aimed at developers with limited familiarity. For context, we review popular applications of deep learning in electron microscopy. Following, we discuss hardware and software needed to get started with deep learning and interface with electron microscopes. We then review neural network components, popular architectures, and their optimization. Finally, we discuss future directions of deep learning in electron microscopy.