Nearest Neighbor-Based Strategy to Optimize Multi-View Triplet Network for Classification of Small-Sample Medical Imaging Data

Local-Global Geometric Information and View Complementarity Introduced Multiview Metric Learning

Geometry studies the spatial structure and location information of objects, providing a priori knowledge and intuitive explanation for classification methods. Considering samples from a geometric perspective offers a novel approach to understanding their information. In this article, we propose a method called local-global geometric information and view complementarity introduced multiview metric learning (GIVCMML). Our method effectively exploits the geometric information of multiview samples. The learned metric space retains the geometric relations of samples and makes them more separable. First, we propose the global geometrical constraint in the maximum margin criterion framework. By maximizing the distance between class centers in the metric space, we ensure that samples from different classes are well separated. Second, to maintain the manifold structure of the original space, we build an adjacency matrix that contains the sample label information. This helps explore the local geometric information of sample pairs. Finally, to better mine the complementary information of multiview samples, GIVCMML maximizes the correlation between each view in the metric space. This enables each view to adaptively learn from the others and explore the complementary information between views. We extensively evaluate the effectiveness of our method on real-world datasets. The experimental results demonstrate that GIVCMML achieves competitive performance compared with multiview metric learning (MvML) methods.

Deep Multiview Adaptive Clustering With Semantic Invariance

Multiview clustering has attracted significant attention in various fields, due to the superiority in mining patterns of multiview data. However, previous methods are still confronted with two challenges. First, they do not fully consider the semantic invariance of multiview data in aggregating complementary information, degrading semantic robustness of fusion representations. Second, they rely on predefined clustering strategies to mine patterns, lacking adequate explorations of data structures. To address the challenges, deep multiview adaptive clustering via semantic invariance (DMAC-SI) is proposed, which learns an adaptive clustering strategy on semantics-robust fusion representations to fully explore structures in mining patterns. Specifically, a mirror fusion architecture is devised to explore interview invariance and intrainstance invariance hidden in multiview data, which captures invariant semantics of complementary information to learn semantics-robust fusion representations. Then, a Markov decision process of multiview data partitions is proposed within the reinforcement learning framework, which learns an adaptive clustering strategy on semantics-robust fusion representations to guarantee the structure explorations in mining patterns. The two components seamlessly collaborate in an end-to-end manner to accurately partition multiview data. Finally, extensive experiment results on five benchmark datasets demonstrate that DMAC-SI outperforms the state-of-the-art methods.

Anti-HER2 therapy response assessment for guiding treatment (de-)escalation in early HER2-positive breast cancer using a novel deep learning radiomics model

OBJECTIVES

Anti-HER2 targeted therapy significantly reduces risk of relapse in HER2 + breast cancer. New measures are needed for a precise risk stratification to guide (de-)escalation of anti-HER2 strategy.

METHODS

A total of 726 HER2 + cases who received no/single/dual anti-HER2 targeted therapies were split into three respective cohorts. A deep learning model (DeepTEPP) based on preoperative breast magnetic resonance (MR) was developed. Patients were scored and categorized into low-, moderate-, and high-risk groups. Recurrence-free survival (RFS) was compared in patients with different risk groups according to the anti-HER2 treatment they received, to validate the value of DeepTEPP in predicting treatment efficacy and guiding anti-HER2 strategy.

RESULTS

DeepTEPP was capable of risk stratification and guiding anti-HER2 treatment strategy: DeepTEPP-Low patients (60.5%) did not derive significant RFS benefit from trastuzumab (p = 0.144), proposing an anti-HER2 de-escalation. DeepTEPP-Moderate patients (19.8%) significantly benefited from trastuzumab (p = 0.048), but did not obtain additional improvements from pertuzumab (p = 0.125). DeepTEPP-High patients (19.7%) significantly benefited from dual HER2 blockade (p = 0.045), suggesting an anti-HER2 escalation.

CONCLUSIONS

DeepTEPP represents a pioneering MR-based deep learning model that enables the non-invasive prediction of adjuvant anti-HER2 effectiveness, thereby providing valuable guidance for anti-HER2 (de-)escalation strategies. DeepTEPP provides an important reference for choosing the appropriate individualized treatment in HER2 + breast cancer patients, warranting prospective validation.

CLINICAL RELEVANCE STATEMENT

We built an MR-based deep learning model DeepTEPP, which enables the non-invasive prediction of adjuvant anti-HER2 effectiveness, thus guiding anti-HER2 (de-)escalation strategies in early HER2-positive breast cancer patients.

KEY POINTS

• DeepTEPP is able to predict anti-HER2 effectiveness and to guide treatment (de-)escalation. • DeepTEPP demonstrated an impressive prognostic efficacy for recurrence-free survival and overall survival. • To our knowledge, this is one of the very few, also the largest study to test the efficacy of a deep learning model extracted from breast MR images on HER2-positive breast cancer survival and anti-HER2 therapy effectiveness prediction.

Incidence rate of occult lymph node metastasis in clinical T1-2N0M0 small cell lung cancer patients and radiomic prediction based on contrast-enhanced CT imaging: a multicenter study : Original research

BACKGROUND

This study aimed to explore the incidence of occult lymph node metastasis (OLM) in clinical T1 - 2N0M0 (cT1 - 2N0M0) small cell lung cancer (SCLC) patients and develop machine learning prediction models using preoperative intratumoral and peritumoral contrast-enhanced CT-based radiomic data.

METHODS

By conducting a retrospective analysis involving 242 eligible patients from 4 centeres, we determined the incidence of OLM in cT1 - 2N0M0 SCLC patients. For each lesion, two ROIs were defined using the gross tumour volume (GTV) and peritumoral volume 15 mm around the tumour (PTV). By extracting a comprehensive set of 1595 enhanced CT-based radiomic features individually from the GTV and PTV, five models were constucted and we rigorously evaluated the model performance using various metrics, including the area under the curve (AUC), accuracy, sensitivity, specificity, calibration curve, and decision curve analysis (DCA). For enhanced clinical applicability, we formulated a nomogram that integrates clinical parameters and the rad_score (GTV and PTV).

RESULTS

The initial investigation revealed a 33.9% OLM positivity rate in cT1 - 2N0M0 SCLC patients. Our combined model, which incorporates three radiomic features from the GTV and PTV, along with two clinical parameters (smoking status and shape), exhibited robust predictive capabilities. With a peak AUC value of 0.772 in the external validation cohort, the model outperformed the alternative models. The nomogram significantly enhanced diagnostic precision for radiologists and added substantial value to the clinical decision-making process for cT1 - 2N0M0 SCLC patients.

CONCLUSIONS

The incidence of OLM in SCLC patients surpassed that in non-small cell lung cancer patients. The combined model demonstrated a notable generalization effect, effectively distinguishing between positive and negative OLMs in a noninvasive manner, thereby guiding individualized clinical decisions for patients with cT1 - 2N0M0 SCLC.

Transfer learning-based PET/CT three-dimensional convolutional neural network fusion of image and clinical information for prediction of EGFR mutation in lung adenocarcinoma

BACKGROUND

To introduce a three-dimensional convolutional neural network (3D CNN) leveraging transfer learning for fusing PET/CT images and clinical data to predict EGFR mutation status in lung adenocarcinoma (LADC).

METHODS

Retrospective data from 516 LADC patients, encompassing preoperative PET/CT images, clinical information, and EGFR mutation status, were divided into training (n = 404) and test sets (n = 112). Several deep learning models were developed utilizing transfer learning, involving CT-only and PET-only models. A dual-stream model fusing PET and CT and a three-stream transfer learning model (TS_TL) integrating clinical data were also developed. Image preprocessing includes semi-automatic segmentation, resampling, and image cropping. Considering the impact of class imbalance, the performance of the model was evaluated using ROC curves and AUC values.

RESULTS

TS_TL model demonstrated promising performance in predicting the EGFR mutation status, with an AUC of 0.883 (95%CI = 0.849-0.917) in the training set and 0.730 (95%CI = 0.629-0.830) in the independent test set. Particularly in advanced LADC, the model achieved an AUC of 0.871 (95%CI = 0.823-0.919) in the training set and 0.760 (95%CI = 0.638-0.881) in the test set. The model identified distinct activation areas in solid or subsolid lesions associated with wild and mutant types. Additionally, the patterns captured by the model were significantly altered by effective tyrosine kinase inhibitors treatment, leading to notable changes in predicted mutation probabilities.

CONCLUSION

PET/CT deep learning model can act as a tool for predicting EGFR mutation in LADC. Additionally, it offers clinicians insights for treatment decisions through evaluations both before and after treatment.

Delicately Reinforced k-Nearest Neighbor Classifier Combined With Expert Knowledge Applied to Abnormity Forecast in Electrolytic Cell

As the profit and safety requirements become higher and higher, it is more and more necessary to realize an advanced intelligent analysis for abnormity forecast of the synthetical balance of material and energy (AF-SBME) on aluminum reduction cells (ARCs). Without loss of generality, AF-SBME belongs to classification problems. Its advanced intelligent analysis can be realized by high-performance data-driven classifiers. However, AF-SBME has some difficulties, including a high requirement for interpretability of data-driven classifiers, a small number, and decreasing-over-time correctness of training samples. In this article, based on a preferable data-driven classifier, which is called a reinforced k -nearest neighbor (R-KNN) classifier, a delicately R-KNN combined with expert knowledge (DR-KNN/CE) is proposed. It improves R-KNN in two ways, including using expert knowledge as external assistance and enhancing self-ability to mine and synthesize data knowledge. The related experiments on AF-SBME, where the relevant data are directly sampled from practical production, have demonstrated that the proposed DR-KNN/CE not only makes an effective improvement for R-KNN, but also has a more advanced performance compared with other existing high-performance data-driven classifiers.

MRI-based radiomic prognostic signature for locally advanced oral cavity squamous cell carcinoma: development, testing and comparison with genomic prognostic signatures

BACKGROUND

. At present, the prognostic prediction in advanced oral cavity squamous cell carcinoma (OCSCC) is based on the tumor-node-metastasis (TNM) staging system, and the most used imaging modality in these patients is magnetic resonance image (MRI). With the aim to improve the prediction, we developed an MRI-based radiomic signature as a prognostic marker for overall survival (OS) in OCSCC patients and compared it with published gene expression signatures for prognosis of OS in head and neck cancer patients, replicated herein on our OCSCC dataset.

METHODS

For each patient, 1072 radiomic features were extracted from T1 and T2-weighted MRI (T1w and T2w). Features selection was performed, and an optimal set of five of them was used to fit a Cox proportional hazard regression model for OS. The radiomic signature was developed on a multi-centric locally advanced OCSCC retrospective dataset (n = 123) and validated on a prospective cohort (n = 108).

RESULTS

The performance of the signature was evaluated in terms of C-index (0.68 (IQR 0.66-0.70)), hazard ratio (HR 2.64 (95% CI 1.62-4.31)), and high/low risk group stratification (log-rank p < 0.001, Kaplan-Meier curves). When tested on a multi-centric prospective cohort (n = 108), the signature had a C-index of 0.62 (IQR 0.58-0.64) and outperformed the clinical and pathologic TNM stage and six out of seven gene expression prognostic signatures. In addition, the significant difference of the radiomic signature between stages III and IVa/b in patients receiving surgery suggests a potential association of MRI features with the pathologic stage.

CONCLUSIONS

Overall, the present study suggests that MRI signatures, containing non-invasive and cost-effective remarkable information, could be exploited as prognostic tools.

Multi-view learning for lymph node metastasis prediction using tumor and nodal radiomics in gastric cancer

Purpose. This study aims to develop and validate a multi-view learning method by the combination of primary tumor radiomics and lymph node (LN) radiomics for the preoperative prediction of LN status in gastric cancer (GC). Methods. A total of 170 contrast-enhanced abdominal CT images from GC patients were enrolled in this retrospective study. After data preprocessing, two-step feature selection approach including Pearson correlation analysis and supervised feature selection method based on test-time budget (FSBudget) was performed to remove redundance of tumor and LN radiomics features respectively. Two types of discriminative features were then learned by an unsupervised multi-view partial least squares (UMvPLS) for a latent common space on which a logistic regression classifier is trained. Five repeated random hold-out experiments were employed. Results. On 20-dimensional latent common space, area under receiver operating characteristic curve (AUC), precision, accuracy, recall and F1-score are 0.9531 ± 0.0183, 0.9260 ± 0.0184, 0.9136 ± 0.0174, 0.9468 ± 0.0106 and 0.9362 ± 0.0125 for the training cohort respectively, and 0.8984 ± 0.0536, 0.8671 ± 0.0489, 0.8500 ± 0.0599, 0.9118 ± 0.0550 and 0.8882 ± 0.0440 for the validation cohort respectively (reported as mean ± standard deviation). It shows a better discrimination capability than single-view methods, our previous method, and eight baseline methods. When the dimension was reduced to 2, the model not only has effective prediction performance, but also is convenient for data visualization. Conclusions. Our proposed method by integrating radiomics features of primary tumor and LN can be helpful in predicting lymph node metastasis in patients of GC. It shows multi-view learning has great potential for guiding the prognosis and treatment decision-making in GC.

Development and External Validation of Deep-Learning-Based Tumor Grading Models in Soft-Tissue Sarcoma Patients Using MR Imaging

BACKGROUND

In patients with soft-tissue sarcomas, tumor grading constitutes a decisive factor to determine the best treatment decision. Tumor grading is obtained by pathological work-up after focal biopsies. Deep learning (DL)-based imaging analysis may pose an alternative way to characterize STS tissue. In this work, we sought to non-invasively differentiate tumor grading into low-grade (G1) and high-grade (G2/G3) STS using DL techniques based on MR-imaging.

METHODS

Contrast-enhanced T1-weighted fat-saturated (T1FSGd) MRI sequences and fat-saturated T2-weighted (T2FS) sequences were collected from two independent retrospective cohorts (training: 148 patients, testing: 158 patients). Tumor grading was determined following the French Federation of Cancer Centers Sarcoma Group in pre-therapeutic biopsies. DL models were developed using transfer learning based on the DenseNet 161 architecture.

RESULTS

The T1FSGd and T2FS-based DL models achieved area under the receiver operator characteristic curve (AUC) values of 0.75 and 0.76 on the test cohort, respectively. T1FSGd achieved the best F1-score of all models (0.90). The T2FS-based DL model was able to significantly risk-stratify for overall survival. Attention maps revealed relevant features within the tumor volume and in border regions.

CONCLUSIONS

MRI-based DL models are capable of predicting tumor grading with good reproducibility in external validation.