G -Softmax: Improving Intraclass Compactness and Interclass Separability of Features

Breast Cancer Subtype Prediction Model Employing Artificial Neural Network and 18F-Fluorodeoxyglucose Positron Emission Tomography/ Computed Tomography

Introduction

Although positron emission tomography/computed tomography (PET/CT) is a common tool for measuring breast cancer (BC), subtypes are not automatically classified by it. Therefore, the purpose of this research is to use an artificial neural network (ANN) to evaluate the clinical subtypes of BC based on the value of the tumor marker.

Materials and Methods

In our nuclear medical facility, 122 BC patients (training and testing) had 18F-fluoro-D-glucose (18F-FDG) PET/CT to identify the various subtypes of the disease. 18F-FDG-18 injections were administered to the patients before the scanning process. We carried out the scan according to protocol. Based on the tumor marker value, the ANN's output layer uses the Softmax function with cross-entropy loss to detect different subtypes of BC.

Results

With an accuracy of 95.77%, the result illustrates the ANN model for K-fold cross-validation. The mean values of specificity and sensitivity were 0.955 and 0.958, respectively. The area under the curve on average was 0.985.

Conclusion

Subtypes of BC may be categorized using the suggested approach. The PET/CT may be updated to diagnose BC subtypes using the appropriate tumor maker value when the suggested model is clinically implemented.

Gaussian class-conditional simplex loss for accurate, adversarially robust deep classifier training

In this work, we present the Gaussian Class-Conditional Simplex (GCCS) loss: a novel approach for training deep robust multiclass classifiers that improves over the state-of-the-art in terms of classification accuracy and adversarial robustness, with little extra cost for network training. The proposed method learns a mapping of the input classes onto Gaussian target distributions in a latent space such that a hyperplane can be used as the optimal decision surface. Instead of maximizing the likelihood of target labels for individual samples, our loss function pushes the network to produce feature distributions yielding high inter-class separation and low intra-class separation. The mean values of the learned distributions are centered on the vertices of a simplex such that each class is at the same distance from every other class. We show that the regularization of the latent space based on our approach yields excellent classification accuracy. Moreover, GCCS provides improved robustness against adversarial perturbations, outperforming models trained with conventional adversarial training (AT). In particular, our model learns a decision space that minimizes the presence of short paths toward neighboring decision regions. We provide a comprehensive empirical evaluation that shows how GCCS outperforms state-of-the-art approaches over challenging datasets for targeted and untargeted gradient-based, as well as gradient-free adversarial attacks, both in terms of classification accuracy and adversarial robustness.

Constrained Center Loss for Convolutional Neural Networks

From the feature representation's point of view, the feature learning module of a convolutional neural network (CNN) is to transform an input pattern into a feature vector. This feature vector is then multiplied with a number of output weight vectors to produce softmax scores. The common training objective in CNNs is based on the softmax loss, which ignores the intra-class compactness. This brief proposes a constrained center loss (CCL)-based algorithm to extract robust features. The training objective of a CNN consists of two terms, softmax loss and CCL. The aim of the softmax loss is to push the feature vectors from different classes apart. Meanwhile, the CCL aims at clustering the feature vectors such that the feature vectors from the same classes are close together. Instead of using stochastic gradient descent (SGD) algorithms to learn all the connection weights and the cluster centers at the same time. Our CCL-based algorithm is based on the alternative learning strategy. We first fix the connection weights of the CNN and update the cluster centers based on an analytical formula, which can be implemented based on the minibatch concept. We then fix the cluster centers and update the connection weights for a number of SGD minibatch iterations. We also propose a simplified CCL (SCCL) algorithm. Experiments are performed on six commonly used benchmark datasets. The results demonstrate that the two proposed algorithms outperform several state-of-the-art approaches.

Mental Health Management of English Teachers in English Teaching Under the COVID-19 Era

Background

The COVID-19 pandemic has brought new challenges and attention to the mental health of all social groups, making mental health increasingly necessary and important. However, people only focus on the mental health of undergraduates, and the mental health of teachers has not received much attention from society. College teachers are the backbone of the teachers' group, and their mental health not only affects the teaching quality and research level but also plays an important role in the mental health and personality development of undergraduates.

Method

During the COVID-19 pandemic, online teaching is a major challenge for college teachers, especially English teachers. To this end, this article proposes a bipartite graph convolutional network (BGCN) model based on the psychological test questionnaire and its structural characteristics for the recognition of the mental health crisis.

Results

Experimental results show that the proposed BGCN model is superior to neural network algorithms and other machine learning algorithms in accuracy, precision, F1, and recall and can be well used for the mental health management of English teachers in the era of COVID-19.

Feature Transformation Framework for Enhancing Compactness and Separability of Data Points in Feature Space for Small Datasets

Compactness and separability of data points are two important properties that contribute to the accuracy of machine learning tasks such as classification and clustering. We propose a framework that enhances the goodness criteria of the two properties by transforming the data points to a subspace in the same feature space, where data points of the same class are most similar to each other. Most related research about feature engineering in the input data points space relies on manually specified transformation functions. In contrast, our work utilizes a fully automated pipeline, in which the transformation function is learnt via an autoencoder for extraction of latent representation and multi-layer perceptron (MLP) regressors for the feature mapping. We tested our framework on both standard small datasets and benchmark-simulated small datasets by taking small fractions of their samples for training. Our framework consistently produced the best results in all semi-supervised clustering experiments based on K-means and different seeding techniques, with regards to clustering metrics and execution time. In addition, it enhances the performance of linear support vector machine (LSVM) and artificial neural network (ANN) classifier, when embedded as a preprocessing step before applying the classifiers.

Artificial neural network for the prediction model of glomerular filtration rate to estimate the normal or abnormal stages of kidney using gamma camera

OBJECTIVE

Chronic kidney disease (CKD) is evaluated based on glomerular filtration rate (GFR) using a gamma camera in the nuclear medicine center or hospital in a routine procedure, but the gamma camera does not provide the accurate stages of the diseases. Therefore, this research aimed to find out the normal or abnormal stages of CKD based on the value of GFR using an artificial neural network (ANN).

METHODS

Two hundred fifty (Training 188, Testing 62) kidney patients who underwent the ultrasonography test to diagnose the renal test in our nuclear medical centre were scanned using gamma camera. The patients were injected with 99mTc-DTPA before the scanning procedure. After pushing the syringe into the patient's vein, the pre-syringe and post syringe radioactive counts were calculated using the gamma camera. The artificial neural network uses the softmax function with cross-entropy loss to diagnose CKD normal or abnormal labels depending on the value of GFR in the output layer.

RESULTS

The results showed that the accuracy of the proposed ANN model was 99.20% for K-fold cross-validation. The sensitivity and specificity were 99.10% and 99.20%, respectively. The Area under the curve (AUC) was 0.9994.

CONCLUSION

The proposed model using an artificial neural network can classify the normal or abnormal stages of CKD. After implementing the proposed model clinically, it may upgrade the gamma camera to diagnose the normal or abnormal stages of the CKD with an appropriate GFR value.

Enhanced Routing Algorithm Based on Reinforcement Machine Learning-A Case of VoIP Service

The routing algorithm is one of the main factors that directly impact on network performance. However, conventional routing algorithms do not consider the network data history, for instances, overloaded paths or equipment faults. It is expected that routing algorithms based on machine learning present advantages using that network data. Nevertheless, in a routing algorithm based on reinforcement learning (RL) technique, additional control message headers could be required. In this context, this research presents an enhanced routing protocol based on RL, named e-RLRP, in which the overhead is reduced. Specifically, a dynamic adjustment in the Hello message interval is implemented to compensate the overhead generated by the use of RL. Different network scenarios with variable number of nodes, routes, traffic flows and degree of mobility are implemented, in which network parameters, such as packet loss, delay, throughput and overhead are obtained. Additionally, a Voice-over-IP (VoIP) communication scenario is implemented, in which the E-model algorithm is used to predict the communication quality. For performance comparison, the OLSR, BATMAN and RLRP protocols are used. Experimental results show that the e-RLRP reduces network overhead compared to RLRP, and overcomes in most cases all of these protocols, considering both network parameters and VoIP quality.

Person Re-Identification across Data Distributions Based on General Purpose DNN Object Detector

Solving the person re-identification problem involves making associations between the same person’s appearances across disjoint camera views. Further, those associations have to be made on multiple surveillance cameras in order to obtain a more efficient and powerful re-identification system. The re-identification problem becomes particularly challenging in very crowded areas. This mainly happens for two reasons. First, the visibility is reduced and occlusions of people can occur. Further, due to congestion, as the number of possible matches increases, the re-identification is becoming challenging to achieve. Additional challenges consist of variations of lightning, poses, or viewpoints, and the existence of noise and blurring effects. In this paper, we aim to generalize person re-identification by implementing a first attempt of a general system, which is robust in terms of distribution variations. Our method is based on the YOLO (You Only Look Once) model, which represents a general object detection system. The novelty of the proposed re-identification method consists of using a simple detection model, with minimal additional costs, but with results that are comparable with those of the other existing dedicated methods.

Optimal Transport with Dimensionality Reduction for Domain Adaptation

Domain adaptation manages to learn a robust classifier for target domain, using the source domain, but they often follow different distributions. To bridge distribution shift between the two domains, most of previous works aim to align their feature distributions through feature transformation, of which optimal transport for domain adaptation has attract researchers’ interest, as it can exploit the local information of the two domains in the process of mapping the source instances to the target ones by minimizing Wasserstein distance between their feature distributions. However, it may weaken the feature discriminability of source domain, thus degrade domain adaptation performance. To address this problem, this paper proposes a two-stage feature-based adaptation approach, referred to as optimal transport with dimensionality reduction (OTDR). In the first stage, we apply the dimensionality reduction with intradomain variant maximization but source intraclass compactness minimization, to separate data samples as much as possible and enhance the feature discriminability of the source domain. In the second stage, we leverage optimal transport-based technique to preserve the local information of the two domains. Notably, the desirable properties in the first stage can mitigate the degradation of feature discriminability of the source domain in the second stage. Extensive experiments on several cross-domain image datasets validate that OTDR is superior to its competitors in classification accuracy.

Associated Learning: Decomposing End-to-End Backpropagation Based on Autoencoders and Target Propagation

Backpropagation (BP) is the cornerstone of today's deep learning algorithms, but it is inefficient partially because of backward locking, which means updating the weights of one layer locks the weight updates in the other layers. Consequently, it is challenging to apply parallel computing or a pipeline structure to update the weights in different layers simultaneously. In this letter, we introduce a novel learning structure, associated learning (AL), that modularizes the network into smaller components, each of which has a local objective. Because the objectives are mutually independent, AL can learn the parameters in different layers independently and simultaneously, so it is feasible to apply a pipeline structure to improve the training throughput. Specifically, this pipeline structure improves the complexity of the training time from O(nℓ), which is the time complexity when using BP and stochastic gradient descent (SGD) for training, to O(n+ℓ), where n is the number of training instances and ℓ is the number of hidden layers. Surprisingly, even though most of the parameters in AL do not directly interact with the target variable, training deep models by this method yields accuracies comparable to those from models trained using typical BP methods, in which all parameters are used to predict the target variable. Consequently, because of the scalability and the predictive power demonstrated in the experiments, AL deserves further study to determine the better hyperparameter settings, such as activation function selection, learning rate scheduling, and weight initialization, to accumulate experience, as we have done over the years with the typical BP method. In addition, perhaps our design can also inspire new network designs for deep learning. Our implementation is available at https://github.com/SamYWK/Associated_Learning.

High-Rankness Regularized Semi-Supervised Deep Metric Learning for Remote Sensing Imagery

Deep metric learning has recently received special attention in the field of remote sensing (RS) scene characterization, owing to its prominent capabilities for modeling distances among RS images based on their semantic information. Most of the existing deep metric learning methods exploit pairwise and triplet losses to learn the feature embeddings with the preservation of semantic-similarity, which requires the construction of image pairs and triplets based on the supervised information (e.g., class labels). However, generating such semantic annotations becomes a completely unaffordable task in large-scale RS archives, which may eventually constrain the availability of sufficient training data for this kind of models. To address this issue, we reformulate the deep metric learning scheme in a semi-supervised manner to effectively characterize RS scenes. Specifically, we aim at learning metric spaces by utilizing the supervised information from a small number of labeled RS images and exploring the potential decision boundaries for massive sets of unlabeled aerial scenes. In order to reach this goal, a joint loss function, composed of a normalized softmax loss with margin and a high-rankness regularization term, is proposed, as well as its corresponding optimization algorithm. The conducted experiments (including different state-of-the-art methods and two benchmark RS archives) validate the effectiveness of the proposed approach for RS image classification, clustering and retrieval tasks. The codes of this paper are publicly available.