Convolutional Ordinal Regression Forest for Image Ordinal Estimation

Learning Ordinal-Hierarchical Constraints for Deep Learning Classifiers

Real-world classification problems may disclose different hierarchical levels where the categories are displayed in an ordinal structure. However, no specific deep learning (DL) models simultaneously learn hierarchical and ordinal constraints while improving generalization performance. To fill this gap, we propose the introduction of two novel ordinal-hierarchical DL methodologies, namely, the hierarchical cumulative link model (HCLM) and hierarchical-ordinal binary decomposition (HOBD), which are able to model the ordinal structure within different hierarchical levels of the labels. In particular, we decompose the hierarchical-ordinal problem into local and global graph paths that may encode an ordinal constraint for each hierarchical level. Thus, we frame this problem as simultaneously minimizing global and local losses. Furthermore, the ordinal constraints are set by two approaches [ordinal binary decomposition (OBD) and cumulative link model (CLM)] within each global and local function. The effectiveness of the proposed approach is measured on four real-use case datasets concerning industrial, biomedical, computer vision, and financial domains. The extracted results demonstrate a statistically significant improvement to state-of-the-art nominal, ordinal, and hierarchical approaches.

HOPE: Hybrid-Granularity Ordinal Prototype Learning for Progression Prediction of Mild Cognitive Impairment

Mild cognitive impairment (MCI) is often at high risk of progression to Alzheimer's disease (AD). Existing works to identify the progressive MCI (pMCI) typically require MCI subtype labels, pMCI vs. stable MCI (sMCI), determined by whether or not an MCI patient will progress to AD after a long follow-up. However, prospectively acquiring MCI subtype data is time-consuming and resource-intensive; the resultant small datasets could lead to severe overfitting and difficulty in extracting discriminative information. Inspired by that various longitudinal biomarkers and cognitive measurements present an ordinal pathway on AD progression, we propose a novel Hybrid-granularity Ordinal PrototypE learning (HOPE) method to characterize AD ordinal progression for MCI progression prediction. First, HOPE learns an ordinal metric space that enables progression prediction by prototype comparison. Second, HOPE leverages a novel hybrid-granularity ordinal loss to learn the ordinal nature of AD via effectively integrating instance-to-instance ordinality, instance-to-class compactness, and class-to-class separation. Third, to make the prototype learning more stable, HOPE employs an exponential moving average strategy to learn the global prototypes of NC and AD dynamically. Experimental results on the internal ADNI and the external NACC datasets demonstrate the superiority of the proposed HOPE over existing state-of-the-art methods as well as its interpretability.

Ordinal Regression With Pinball Loss

Ordinal regression (OR) aims to solve multiclass classification problems with ordinal classes. Support vector OR (SVOR) is a typical OR algorithm and has been extensively used in OR problems. In this article, based on the characteristics of OR problems, we propose a novel pinball loss function and present an SVOR method with pinball loss (pin-SVOR). Pin-SVOR is fundamentally different from traditional SVOR with hinge loss. Traditional SVOR employs the hinge loss function, and the classifier is determined by only a few data points near the class boundary, called support vectors, which may lead to a noise sensitive and re-sampling unstable classifier. Distinctively, pin-SVOR employs the pinball loss function. It attaches an extra penalty to correctly classified data that lies inside the class, such that all the training data is involved in deciding the classifier. The data near the middle of each class has a small penalty, and that near the class boundary has a large penalty. Thus, the training data tend to lie near the middle of each class instead of on the class boundary, which leads to scatter minimization in the middle of each class and noise insensitivity. The experimental results show that pin-SVOR has better classification performance than state-of-the-art OR methods.

CA-XTree: Age Estimation of Grouped Gradient Regression Tree with Local Channel Attention

Face age estimation has been widely used in video surveillance, human-computer interaction, market analysis, image processing analysis, and many fields. There are several problems that need to be solved in image-based face age estimation: (1) redundant information of age characteristics; (2) limitations of age estimation methods in solving age estimation problems; (3) the performance of age estimation models being also affected by gender factors. This paper proposes CA-XTree network. Firstly, features are extracted through the convolution layer and then combined with the local channel attention module to strengthen the ability of age feature information interaction between different channels. Secondly, extracted features are inputted into the recommendation score function to obtain the recommendation score, by combining the recommendation score with the gradient ascending regression tree. The lifting tree processed loss function is the mean square loss function, and the final age value is obtained by the leaf node. This paper improves state of the art for image classification on MORPH and CACD datasets. The advantage of our model is that it is easy to implement and has no excess memory overhead. In the age dataset CACD, the mean absolute error (MAE) has reached 4.535 and cumulative score (CS) has reached 63.53%, respectively.

A novel deep ordinal classification approach for aesthetic quality control classification

Nowadays, decision support systems (DSSs) are widely used in several application domains, from industrial to healthcare and medicine fields. Concerning the industrial scenario, we propose a DSS oriented to the aesthetic quality control (AQC) task, which has quickly established itself as one of the most crucial challenges of Industry 4.0. Taking into account the increasing amount of data in this domain, the application of machine learning (ML) and deep learning (DL) techniques offers great opportunities to automatize the overall AQC process. State-of-the-art is mainly oriented to approach this problem with a nominal DL classification method which does not exploit the ordinal structure of the AQC task, thus not penalizing the error among distant AQC classes (which is a relevant aspect for the real use case). The paper introduces a DL ordinal methodology for the AQC classification. Differently from other deep ordinal methods, we combined the standard categorical cross-entropy with the cumulative link model and we imposed the ordinal constraint via the thresholds and slope parameters. Experimental results were performed for solving an AQC task on a novel image dataset originated from a specific company’s demand (i.e., aesthetic assessment of wooden stocks). We demonstrated how the proposed methodology is able to reduce misclassification errors (up to 0.937 quadratic weight kappa loss) among distant classes while overcoming other state-of-the-art deep ordinal models and reducing the bias factor related to the item geometry. The proposed DL approach was integrated as the main core of a DSS supported by Internet of Things (IoT) architecture that can support the human operator by reducing up to 90% the time needed for the qualitative analysis carried out manually in this specific domain.

An ensemble learning method based on ordinal regression for COVID-19 diagnosis from chest CT

Coronavirus disease 2019 (COVID-19) has brought huge losses to the world, and it remains a great threat to public health. X-ray computed tomography (CT) plays a central role in the management of COVID-19. Traditional diagnosis with pulmonary CT images is time-consuming and error-prone, which could not meet the need for precise and rapid COVID-19 screening. Nowadays, deep learning (DL) has been successfully applied to CT image analysis, which assists radiologists in workflow scheduling and treatment planning for patients with COVID-19. Traditional methods use cross-entropy as the loss function with a Softmax classifier following a fully-connected layer. Most DL-based classification methods target intraclass relationships in a certain class (early, progressive, severe, or dissipative phases), ignoring the natural order of different phases of the disease progression,i.e.,from an early stage and progress to a late stage. To learn both intraclass and interclass relationships among different stages and improve the accuracy of classification, this paper proposes an ensemble learning method based on ordinal regression, which leverages the ordinal information on COVID-19 phases. The proposed method uses multi-binary, neuron stick-breaking (NSB), and soft labels (SL) techniques, and ensembles the ordinal outputs through a median selection. To evaluate our method, we collected 172 confirmed cases. In a 2-fold cross-validation experiment, the accuracy is increased by 22% compared with traditional methods when we use modified ResNet-18 as the backbone. And precision, recall, andF1-score are also improved. The experimental results show that our proposed method achieves a better classification performance than the traditional methods, which helps establish guidelines for the classification of COVID-19 chest CT images.