Prognostic value of automated assessment of interstitial lung disease on CT in systemic sclerosis

OBJECTIVE

Stratifying the risk of death in SSc-related interstitial lung disease (SSc-ILD) is a challenging issue. The extent of lung fibrosis on high-resolution CT (HRCT) is often assessed by a visual semiquantitative method that lacks reliability. We aimed to assess the potential prognostic value of a deep-learning-based algorithm enabling automated quantification of ILD on HRCT in patients with SSc.

METHODS

We correlated the extent of ILD with the occurrence of death during follow-up, and evaluated the additional value of ILD extent in predicting death based on a prognostic model including well-known risk factors in SSc.

RESULTS

We included 318 patients with SSc, among whom 196 had ILD; the median follow-up was 94 months (interquartile range 73-111). The mortality rate was 1.6% at 2 years and 26.3% at 10 years. For each 1% increase in the baseline ILD extent (up to 30% of the lung), the risk of death at 10 years was increased by 4% (hazard ratio 1.04, 95% CI 1.01, 1.07, P = 0.004). We constructed a risk prediction model that showed good discrimination for 10-year mortality (c index 0.789). Adding the automated quantification of ILD significantly improved the model for 10-year survival prediction (P = 0.007). Its discrimination was only marginally improved, but it improved prediction of 2-year mortality (difference in time-dependent area under the curve 0.043, 95% CI 0.002, 0.084, P = 0.040).

CONCLUSION

The deep-learning-based, computer-aided quantification of ILD extent on HRCT provides an effective tool for risk stratification in SSc. It might help identify patients at short-term risk of death.

Trastuzumab deruxtecan in metastatic breast cancer with variable HER2 expression: the phase 2 DAISY trial

The mechanisms of action of and resistance to trastuzumab deruxtecan (T-DXd), an anti-HER2-drug conjugate for breast cancer treatment, remain unclear. The phase 2 DAISY trial evaluated the efficacy of T-DXd in patients with HER2-overexpressing (n = 72, cohort 1), HER2-low (n = 74, cohort 2) and HER2 non-expressing (n = 40, cohort 3) metastatic breast cancer. In the full analysis set population (n = 177), the confirmed objective response rate (primary endpoint) was 70.6% (95% confidence interval (CI) 58.3-81) in cohort 1, 37.5% (95% CI 26.4-49.7) in cohort 2 and 29.7% (95% CI 15.9-47) in cohort 3. The primary endpoint was met in cohorts 1 and 2. Secondary endpoints included safety. No new safety signals were observed. During treatment, HER2-expressing tumors (n = 4) presented strong T-DXd staining. Conversely, HER2 immunohistochemistry 0 samples (n = 3) presented no or very few T-DXd staining (Pearson correlation coefficient r = 0.75, P = 0.053). Among patients with HER2 immunohistochemistry 0 metastatic breast cancer, 5 of 14 (35.7%, 95% CI 12.8-64.9) with ERBB2 expression below the median presented a confirmed objective response as compared to 3 of 10 (30%, 95% CI 6.7-65.2) with ERBB2 expression above the median. Although HER2 expression is a determinant of T-DXd efficacy, our study suggests that additional mechanisms may also be involved. (ClinicalTrials.gov identifier NCT04132960 .).

Dosiomics-Based Prediction of Radiation-Induced Valvulopathy after Childhood Cancer

Valvular Heart Disease (VHD) is a known late complication of radiotherapy for childhood cancer (CC), and identifying high-risk survivors correctly remains a challenge. This paper focuses on the distribution of the radiation dose absorbed by heart tissues. We propose that a dosiomics signature could provide insight into the spatial characteristics of the heart dose associated with a VHD, beyond the already-established risk induced by high doses. We analyzed data from the 7670 survivors of the French Childhood Cancer Survivors' Study (FCCSS), 3902 of whom were treated with radiotherapy. In all, 63 (1.6%) survivors that had been treated with radiotherapy experienced a VHD, and 57 of them had heterogeneous heart doses. From the heart-dose distribution of each survivor, we extracted 93 first-order and spatial dosiomics features. We trained random forest algorithms adapted for imbalanced classification and evaluated their predictive performance compared to the performance of standard mean heart dose (MHD)-based models. Sensitivity analyses were also conducted for sub-populations of survivors with spatially heterogeneous heart doses. Our results suggest that MHD and dosiomics-based models performed equally well globally in our cohort and that, when considering the sub-population having received a spatially heterogeneous dose distribution, the predictive capability of the models is significantly improved by the use of the dosiomics features. If these findings are further validated, the dosiomics signature may be incorporated into machine learning algorithms for radiation-induced VHD risk assessment and, in turn, into the personalized refinement of follow-up guidelines.

Learn2Reg: Comprehensive Multi-Task Medical Image Registration Challenge, Dataset and Evaluation in the Era of Deep Learning

Image registration is a fundamental medical image analysis task, and a wide variety of approaches have been proposed. However, only a few studies have comprehensively compared medical image registration approaches on a wide range of clinically relevant tasks. This limits the development of registration methods, the adoption of research advances into practice, and a fair benchmark across competing approaches. The Learn2Reg challenge addresses these limitations by providing a multi-task medical image registration data set for comprehensive characterisation of deformable registration algorithms. A continuous evaluation will be possible at https://learn2reg.grand-challenge.org. Learn2Reg covers a wide range of anatomies (brain, abdomen, and thorax), modalities (ultrasound, CT, MR), availability of annotations, as well as intra- and inter-patient registration evaluation. We established an easily accessible framework for training and validation of 3D registration methods, which enabled the compilation of results of over 65 individual method submissions from more than 20 unique teams. We used a complementary set of metrics, including robustness, accuracy, plausibility, and runtime, enabling unique insight into the current state-of-the-art of medical image registration. This paper describes datasets, tasks, evaluation methods and results of the challenge, as well as results of further analysis of transferability to new datasets, the importance of label supervision, and resulting bias. While no single approach worked best across all tasks, many methodological aspects could be identified that push the performance of medical image registration to new state-of-the-art performance. Furthermore, we demystified the common belief that conventional registration methods have to be much slower than deep-learning-based methods.

COMBING: Clustering in Oncology for Mathematical and Biological Identification of Novel Gene Signatures

Precision medicine is a paradigm shift in healthcare relying heavily on genomics data. However, the complexity of biological interactions, the large number of genes as well as the lack of comparisons on the analysis of data, remain a tremendous bottleneck regarding clinical adoption. In this paper, we introduce a novel, automatic and unsupervised framework to discover low-dimensional gene biomarkers. Our method is based on the LP-Stability algorithm, a high dimensional center-based unsupervised clustering algorithm. It offers modularity as concerns metric functions and scalability, while being able to automatically determine the best number of clusters. Our evaluation includes both mathematical and biological criteria to define a quantitative metric. The recovered signature is applied to a variety of biological tasks, including screening of biological pathways and functions, and characterization relevance on tumor types and subtypes. Quantitative comparisons among different distance metrics, commonly used clustering methods and a referential gene signature used in the literature, confirm state of the art performance of our approach. In particular, our signature, based on 27 genes, reports at least 30 times better mathematical significance (average Dunn's Index) and 25% better biological significance (average Enrichment in Protein-Protein Interaction) than those produced by other referential clustering methods. Finally, our signature reports promising results on distinguishing immune inflammatory and immune desert tumors, while reporting a high balanced accuracy of 92% on tumor types classification and averaged balanced accuracy of 68% on tumor subtypes classification, which represents, respectively 7% and 9% higher performance compared to the referential signature.

Using chest CT scan and unsupervised machine learning for predicting and evaluating response to lumacaftor-ivacaftor in people with cystic fibrosis

OBJECTIVES

Lumacaftor-ivacaftor is a cystic fibrosis transmembrane conductance regulator (CFTR) modulator known to improve clinical status in people with cystic fibrosis (CF). This study aimed to assess lung structural changes after one year of lumacaftor-ivacaftor treatment, and to use unsupervised machine learning to identify morphological phenotypes of lung disease that are associated with response to lumacaftor-ivacaftor.

METHODS

Adolescents and adults with CF from the French multicenter real-world prospective observational study evaluating the first year of treatment with lumacaftor-ivacaftor were included if they had pretherapeutic and follow-up chest computed tomography (CT)-scans available. CT scans were visually scored using a modified Bhalla score. A k-mean clustering method was performed based on 120 radiomics features extracted from unenhanced pretherapeutic chest CT scans.

RESULTS

A total of 283 patients were included. The Bhalla score significantly decreased after 1 year of lumacaftor-ivacaftor (-1.40±1.53 points compared with pretherapeutic CT; p<0.001). This finding was related to a significant decrease in mucus plugging (-0.35±0.62 points; p<0.001), bronchial wall thickening (-0.24±0.52 points; p<0.001) and parenchymal consolidations (-0.23±0.51 points; p<0.001). Cluster analysis identified 3 morphological clusters. Patients from cluster C were more likely to experience an increase in percent predicted forced expiratory volume in 1 sec (ppFEV1) ≥5 under lumacaftor-ivacaftor than those in the other clusters (54% of responders versus 32% and 33%; p=0.01).

CONCLUSION

One year treatment with lumacaftor-ivacaftor was associated with a significant visual improvement of bronchial disease on chest CT. Radiomics features on pretherapeutic CT scan may help in predicting lung function response under lumacaftor-ivacaftor.

Deep learning for lung disease segmentation on CT: Which reconstruction kernel should be used?

PURPOSE

The purpose of this study was to determine whether a single reconstruction kernel or both high and low frequency kernels should be used for training deep learning models for the segmentation of diffuse lung disease on chest computed tomography (CT).

MATERIALS AND METHODS

Two annotated datasets of COVID-19 pneumonia (323,960 slices) and interstitial lung disease (ILD) (4,284 slices) were used. Annotated CT images were used to train a U-Net architecture to segment disease. All CT slices were reconstructed using both a lung kernel (LK) and a mediastinal kernel (MK). Three different trainings, resulting in three different models were compared for each disease: training on LK only, MK only or LK+MK images. Dice similarity scores (DSC) were compared using the Wilcoxon signed-rank test.

RESULTS

Models only trained on LK images performed better on LK images than on MK images (median DSC = 0.62 [interquartile range (IQR): 0.54, 0.69] vs. 0.60 [IQR: 0.50, 0.70], P < 0.001 for COVID-19 and median DSC = 0.62 [IQR: 0.56, 0.69] vs. 0.50 [IQR 0.43, 0.57], P < 0.001 for ILD). Similarly, models only trained on MK images performed better on MK images (median DSC = 0.62 [IQR: 0.53, 0.68] vs. 0.54 [IQR: 0.47, 0.63], P < 0.001 for COVID-19 and 0.69 [IQR: 0.61, 0.73] vs. 0.63 [IQR: 0.53, 0.70], P < 0.001 for ILD). Models trained on both kernels performed better or similarly than those trained on only one kernel. For COVID-19, median DSC was 0.67 (IQR: =0.59, 0.73) when applied on LK images and 0.67 (IQR: 0.60, 0.74) when applied on MK images (P < 0.001 for both). For ILD, median DSC was 0.69 (IQR: 0.63, 0.73) when applied on LK images (P = 0.006) and 0.68 (IQR: 0.62, 0.72) when applied on MK images (P > 0.99).

CONCLUSION

Reconstruction kernels impact the performance of deep learning-based models for lung disease segmentation. Training on both LK and MK images improves the performance.

Deep Multi-Instance Learning Using Multi-Modal Data for Diagnosis of Lymphocytosis

We investigate the use of recent advances in deep learning and propose an end-to-end trainable multi-instance convolutional neural network within a mixture-of-experts formulation that combines information from two types of data-images and clinical attributes-for the diagnosis of lymphocytosis. The convolutional network learns to extract meaningful features from images of blood cells using an embedding level approach and aggregates them. Moreover, the mixture-of-experts model combines information from these images as well as clinical attributes to form an end-to-end trainable pipeline for diagnosis of lymphocytosis. Our results demonstrate that even the convolutional network by itself is able to discover meaningful associations between the images and the diagnosis, indicating the presence of important unexploited information in the images. The mixture-of-experts formulation is shown to be more robust while maintaining performance via. a repeatability study to assess the effect of variability in data acquisition on the predictions. The proposed methods are compared with different methods from literature based both on conventional handcrafted features and machine learning, and on recent deep learning models based on attention mechanisms. Our method reports a balanced accuracy of [Formula: see text] and outperfroms the handcrafted feature-based and attention-based approaches as well that of biologists which scored [Formula: see text], [Formula: see text] and [Formula: see text] respectively. These results give insights on the potentials of the applicability of the proposed method in clinical practice. Our code and datasets can be found at https://github.com/msahasrabudhe/lymphoMIL.

Abstract PS5-13: Holistic artificial intelligence-driven predictor in HER2-positive (HER2+) early breast cancer (BC) treated with neoadjuvant lapatinib and trastuzumab without chemotherapy: A correlative analysis from SOLTI-1114 PAMELA

Background: Positive partial response was observed in patients with primary HER2+ early BC with dual HER2 blockade that were not treated with chemotherapy. In this context and beyond, the low (partial) response rate of chemotherapy-free treatment strategies creates the necessity for patient stratification prior to treatment selection. Here, we tackle the challenging task of evaluating the ability of clinical, gene expression and histopathology data to predict response following dual HER2 blockade without chemotherapy. Our aim is through artificial intelligence to automatically decipher the complementarity of clinical, genomic and histopathology data through an evidence-driven approach towards a low dimensional holistic signature that determines outcomes and could be subsequently used as a clinical biomarker for treatment patient inclusion. Methods: PAMELA (Lancet Oncology 2017) was a prospective study in HER2+ BC designed to evaluate the ability of the PAM50 HER2-enriched intrinsic subtype to predict pCR following 18-weeks of neoadjuvant lapatinib and trastuzumab (and hormonal therapy if hormone receptor-positive [HR+]). Clinical-pathological variables (15) were included such as tumor cellularity, tumor-infiltrating lymphocytes (TILs), the expression of BC-related genes/signatures (567) along with histopathological data from pre-treatment samples. Imaging information was obtained from H/E slides, through an unsupervised deep learning approach using an attention network. The semantic segmentation was used to derive at the patch level image and shape characteristics resulted on a pathomics-derived feature vector of (300) variables. An integrative approach that harnessed clinical, genomics and pathomics data into a unified prediction framework was used. Patients were divided into a training set 80% and a testing set 20% with proportions of pCR and non-pCR corresponding to the ones observed. A 100-fold Cross-validation (CV) was performed on the training. Linear and non-linear robust feature selection were used to recover a low dimensional holistic signature along with an ensemble learning approach to select the top 5 machine learning/artificial intelligence methods for prognosis. Results: From the high dimensional feature (882) space, a low dimensional holistic signature of 8 predictive variables was automatically retrieved through. The signature consisted of 4 genomics variables (expression levels of ERBB2, ESR1, Luminal A signature and Risk of Relapse score), 2 clinical-pathological variables (histologic grade and ER-status) and 2 imaging variables (mean Short Run Low Gray Level Emphasis of the gray level run length matrix and the mean absolute deviation). To ensure the robustness and generalizability of the results, we present results averaged over 100 splits into training and test. On all cases the same holistic signature was uses and the same prediction methods/principles. The proposed AI-driven prognosis mechanism reached 75% balanced accuracy, 69% precision, 65% sensitivity, 86% specificity and 0.84 AUC demonstrating the relevance of the approach. It was observed a successful classification of 86% for the non-pCR and 65% for the pCR cases. Ablation studies were performed to determine the relevance of the different categories of variables. Genomics variables were the most informative since their removal led to the highest decrease of the metrics (11% in average). Conclusion: The proposed method has great potentials for an effective and clinically meaningful implementation of pre-selecting patients that will not achieve a pCR after neoadjuvant dual HER2 blockade. Besides, the generality of the method used here makes it transposable to any type of cancer or therapy.

Citation Format: Enzo Batistella, Laia Paré, Mihir Sahasrabudhe, Tomás Pascual, Maria Vakalopoulou, Patricia Villagrasa, Eric Deutsch, Núria Chic, Guillermo Villacampa, Paolo Nuciforo, Javier Cortes, Antonio Llombart-Cussac, Nikos Paragios, Aleix Prat. Holistic artificial intelligence-driven predictor in HER2-positive (HER2+) early breast cancer (BC) treated with neoadjuvant lapatinib and trastuzumab without chemotherapy: A correlative analysis from SOLTI-1114 PAMELA [abstract]. In: Proceedings of the 2020 San Antonio Breast Cancer Virtual Symposium; 2020 Dec 8-11; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2021;81(4 Suppl):Abstract nr PS5-13.

Radiomics to predict outcomes and abscopal response of patients with cancer treated with immunotherapy combined with radiotherapy using a validated signature of CD8 cells

BACKGROUND

Combining radiotherapy (RT) with immuno-oncology (IO) therapy (IORT) may enhance IO-induced antitumor response. Quantitative imaging biomarkers can be used to provide prognosis, predict tumor response in a non-invasive fashion and improve patient selection for IORT. A biologically inspired CD8 T-cells-associated radiomics signature has been developed on previous cohorts. We evaluated here whether this CD8 radiomic signature is associated with lesion response, whether it may help to assess disease spatial heterogeneity for predicting outcomes of patients treated with IORT. We also evaluated differences between irradiated and non-irradiated lesions.

METHODS

Clinical data from patients with advanced solid tumors in six independent clinical studies of IORT were investigated. Immunotherapy consisted of 4 different drugs (antiprogrammed death-ligand 1 or anticytotoxic T-lymphocyte-associated protein 4 in monotherapy). Most patients received stereotactic RT to one lesion. Irradiated and non-irradiated lesions were delineated from baseline and the first evaluation CT scans. Radiomic features were extracted from contrast-enhanced CT images and the CD8 radiomics signature was applied. A responding lesion was defined by a decrease in lesion size of at least 30%. Dispersion metrices of the radiomics signature were estimated to evaluate the impact of tumor heterogeneity in patient's response.

RESULTS

A total of 94 patients involving multiple lesions (100 irradiated and 189 non-irradiated lesions) were considered for a statistical interpretation. Lesions with high CD8 radiomics score at baseline were associated with significantly higher tumor response (area under the receiving operating characteristic curve (AUC)=0.63, p=0.0020). Entropy of the radiomics scores distribution on all lesions was shown to be associated with progression-free survival (HR=1.67, p=0.040), out-of-field abscopal response (AUC=0.70, p=0.014) and overall survival (HR=2.08, p=0.023), which remained significant in a multivariate analysis including clinical and biological variables.

CONCLUSIONS

These results enhance the predictive value of the biologically inspired CD8 radiomics score and suggests that tumor heterogeneity should be systematically considered in patients treated with IORT. This CD8 radiomics signature may help select patients who are most likely to benefit from IORT.

Elastic Registration-driven Deep Learning for Longitudinal Assessment of Systemic Sclerosis Interstitial Lung Disease at CT

Background Longitudinal follow-up of interstitial lung diseases (ILDs) at CT mainly relies on the evaluation of the extent of ILD, without accounting for lung shrinkage. Purpose To develop a deep learning-based method to depict worsening of ILD based on lung shrinkage detection from elastic registration of chest CT scans in patients with systemic sclerosis (SSc). Materials and Methods Patients with SSc evaluated between January 2009 and October 2017 who had undergone at least two unenhanced supine CT scans of the chest and pulmonary function tests (PFTs) performed within 3 months were retrospectively included. Morphologic changes on CT scans were visually assessed by two observers and categorized as showing improvement, stability, or worsening of ILD. Elastic registration between baseline and follow-up CT images was performed to obtain deformation maps of the whole lung. Jacobian determinants calculated from the deformation maps were given as input to a deep learning-based classifier to depict morphologic and functional worsening. For this purpose, the set was randomly split into training, validation, and test sets. Correlations between mean Jacobian values and changes in PFT measurements were evaluated with the Spearman correlation. Results A total of 212 patients (median age, 53 years; interquartile range, 45-62 years; 177 women) were included as follows: 138 for the training set (65%), 34 for the validation set (16%), and 40 for the test set (21%). Jacobian maps demonstrated lung parenchyma shrinkage of the posterior lung bases in patients found to have worsened ILD at visual assessment. The classifier detected morphologic and functional worsening with an accuracy of 80% (32 of 40 patients; 95% confidence interval [CI]: 64%, 91%) and 83% (33 of 40 patients; 95% CI: 67%, 93%), respectively. Jacobian values correlated with changes in forced vital capacity (R = -0.38; 95% CI: -0.25, -0.49; P < .001) and diffusing capacity for carbon monoxide (R = -0.42; 95% CI: -0.27, -0.54; P < .001). Conclusion Elastic registration of CT scans combined with a deep learning classifier aided in the diagnosis of morphologic and functional worsening of interstitial lung disease in patients with systemic sclerosis. © RSNA, 2020 Online supplemental material is available for this article. See also the editorial by Verschakelen in this issue.

Deep Learning-based Approach for Automated Assessment of Interstitial Lung Disease in Systemic Sclerosis on CT Images

Purpose

To develop a deep learning algorithm for the automatic assessment of the extent of systemic sclerosis (SSc)-related interstitial lung disease (ILD) on chest CT images.

Materials and Methods

This retrospective study included 208 patients with SSc (median age, 57 years; 167 women) evaluated between January 2009 and October 2017. A multicomponent deep neural network (AtlasNet) was trained on 6888 fully annotated CT images (80% for training and 20% for validation) from 17 patients with no, mild, or severe lung disease. The model was tested on a dataset of 400 images from another 20 patients, independently partially annotated by three radiologist readers. The ILD contours from the three readers and the deep learning neural network were compared by using the Dice similarity coefficient (DSC). The correlation between disease extent obtained from the deep learning algorithm and that obtained by using pulmonary function tests (PFTs) was then evaluated in the remaining 171 patients and in an external validation dataset of 31 patients based on the analysis of all slices of the chest CT scan. The Spearman rank correlation coefficient (ρ) was calculated to evaluate the correlation between disease extent and PFT results.

Results

The median DSCs between the readers and the deep learning ILD contours ranged from 0.74 to 0.75, whereas the median DSCs between contours from radiologists ranged from 0.68 to 0.71. The disease extent obtained from the algorithm, by analyzing the whole CT scan, correlated with the diffusion lung capacity for carbon monoxide, total lung capacity, and forced vital capacity (ρ = -0.76, -0.70, and -0.62, respectively; P < .001 for all) in the dataset for the correlation with PFT results. The disease extents correlated with diffusion lung capacity for carbon monoxide, total lung capacity, and forced vital capacity were ρ = -0.65, -0.70, and -0.57, respectively, in the external validation dataset (P < .001 for all).

Conclusion

The developed algorithm performed similarly to radiologists for disease-extent contouring, which correlated with pulmonary function to assess CT images from patients with SSc-related ILD.Supplemental material is available for this article.© RSNA, 2020.

Deep Learning-Based Concurrent Brain Registration and Tumor Segmentation

Image registration and segmentation are the two most studied problems in medical image analysis. Deep learning algorithms have recently gained a lot of attention due to their success and state-of-the-art results in variety of problems and communities. In this paper, we propose a novel, efficient, and multi-task algorithm that addresses the problems of image registration and brain tumor segmentation jointly. Our method exploits the dependencies between these tasks through a natural coupling of their interdependencies during inference. In particular, the similarity constraints are relaxed within the tumor regions using an efficient and relatively simple formulation. We evaluated the performance of our formulation both quantitatively and qualitatively for registration and segmentation problems on two publicly available datasets (BraTS 2018 and OASIS 3), reporting competitive results with other recent state-of-the-art methods. Moreover, our proposed framework reports significant amelioration (p < 0.005) for the registration performance inside the tumor locations, providing a generic method that does not need any predefined conditions (e.g., absence of abnormalities) about the volumes to be registered. Our implementation is publicly available online at https://github.com/TheoEst/joint_registration_tumor_segmentation.

Artificial intelligence applications for thoracic imaging

Artificial intelligence is a hot topic in medical imaging. The development of deep learning methods and in particular the use of convolutional neural networks (CNNs), have led to substantial performance gain over the classic machine learning techniques. Multiple usages are currently being evaluated, especially for thoracic imaging, such as such as lung nodule evaluation, tuberculosis or pneumonia detection or quantification of diffuse lung diseases. Chest radiography is a near perfect domain for the development of deep learning algorithms for automatic interpretation, requiring large annotated datasets, in view of the high number of procedures and increasing data availability. Current algorithms are able to detect up to 14 common anomalies, when present as isolated findings. Chest computed tomography is another major field of application for artificial intelligence, especially in the perspective of large scale lung cancer screening. It is important for radiologists to apprehend, contribute actively and lead this new era of radiology powered by artificial intelligence. Such a perspective requires understanding new terms and concepts associated with machine learning. The objective of this paper is to provide useful definitions for understanding the methods used and their possibilities, and report current and future developments for thoracic imaging. Prospective validation of AI tools will be required before reaching routine clinical implementation.

A radiomics approach to assess tumour-infiltrating CD8 cells and response to anti-PD-1 or anti-PD-L1 immunotherapy: an imaging biomarker, retrospective multicohort study

BACKGROUND

Because responses of patients with cancer to immunotherapy can vary in success, innovative predictors of response to treatment are urgently needed to improve treatment outcomes. We aimed to develop and independently validate a radiomics-based biomarker of tumour-infiltrating CD8 cells in patients included in phase 1 trials of anti-programmed cell death protein (PD)-1 or anti-programmed cell death ligand 1 (PD-L1) monotherapy. We also aimed to evaluate the association between the biomarker, and tumour immune phenotype and clinical outcomes of these patients.

METHODS

In this retrospective multicohort study, we used four independent cohorts of patients with advanced solid tumours to develop and validate a radiomic signature predictive of immunotherapy response by combining contrast-enhanced CT images and RNA-seq genomic data from tumour biopsies to assess CD8 cell tumour infiltration. To develop the radiomic signature of CD8 cells, we used the CT images and RNA sequencing data of 135 patients with advanced solid malignant tumours who had been enrolled into the MOSCATO trial between May 1, 2012, and March 31, 2016, in France (training set). The genomic data, which are based on the CD8B gene, were used to estimate the abundance of CD8 cells in the samples and data were then aligned with the images to generate the radiomic signatures. The concordance of the radiomic signature (primary endpoint) was validated in a Cancer Genome Atlas [TGCA] database dataset including 119 patients who had available baseline preoperative imaging data and corresponding transcriptomic data on June 30, 2017. From 84 input variables used for the machine-learning method (78 radiomic features, five location variables, and one technical variable), a radiomics-based predictor of the CD8 cell expression signature was built by use of machine learning (elastic-net regularised regression method). Two other independent cohorts of patients with advanced solid tumours were used to evaluate this predictor. The immune phenotype internal cohort (n=100), were randomly selected from the Gustave Roussy Cancer Campus database of patient medical records based on previously described, extreme tumour-immune phenotypes: immune-inflamed (with dense CD8 cell infiltration) or immune-desert (with low CD8 cell infiltration), irrespective of treatment delivered; these data were used to analyse the correlation of the immune phenotype with this biomarker. Finally, the immunotherapy-treated dataset (n=137) of patients recruited from Dec 1, 2011, to Jan 31, 2014, at the Gustave Roussy Cancer Campus, who had been treated with anti-PD-1 and anti-PD-L1 monotherapy in phase 1 trials, was used to assess the predictive value of this biomarker in terms of clinical outcome.

FINDINGS

We developed a radiomic signature for CD8 cells that included eight variables, which was validated with the gene expression signature of CD8 cells in the TCGA dataset (area under the curve [AUC]=0·67; 95% CI 0·57-0·77; p=0·0019). In the cohort with assumed immune phenotypes, the signature was also able to discriminate inflamed tumours from immune-desert tumours (0·76; 0·66-0·86; p<0·0001). In patients treated with anti-PD-1 and PD-L1, a high baseline radiomic score (relative to the median) was associated with a higher proportion of patients who achieved an objective response at 3 months (vs those with progressive disease or stable disease; p=0·049) and a higher proportion of patients who had an objective response (vs those with progressive disease or stable disease; p=0·025) or stable disease (vs those with progressive disease; p=0·013) at 6 months. A high baseline radiomic score was also associated with improved overall survival in univariate (median overall survival 24·3 months in the high radiomic score group, 95% CI 18·63-42·1; vs 11·5 months in the low radiomic score group, 7·98-15·6; hazard ratio 0·58, 95% CI 0·39-0·87; p=0·0081) and multivariate analyses (0·52, 0·35-0·79; p=0·0022).

INTERPRETATION

The radiomic signature of CD8 cells was validated in three independent cohorts. This imaging predictor provided a promising way to predict the immune phenotype of tumours and to infer clinical outcomes for patients with cancer who had been treated with anti-PD-1 and PD-L1. Our imaging biomarker could be useful in estimating CD8 cell count and predicting clinical outcomes of patients treated with immunotherapy, when validated by further prospective randomised trials.

FUNDING

Fondation pour la Recherche Médicale, and SIRIC-SOCRATE 2.0, French Society of Radiation Oncology.

Demystification of AI-driven medical image interpretation: past, present and future

The recent explosion of 'big data' has ushered in a new era of artificial intelligence (AI) algorithms in every sphere of technological activity, including medicine, and in particular radiology. However, the recent success of AI in certain flagship applications has, to some extent, masked decades-long advances in computational technology development for medical image analysis. In this article, we provide an overview of the history of AI methods for radiological image analysis in order to provide a context for the latest developments. We review the functioning, strengths and limitations of more classical methods as well as of the more recent deep learning techniques. We discuss the unique characteristics of medical data and medical science that set medicine apart from other technological domains in order to highlight not only the potential of AI in radiology but also the very real and often overlooked constraints that may limit the applicability of certain AI methods. Finally, we provide a comprehensive perspective on the potential impact of AI on radiology and on how to evaluate it not only from a technical point of view but also from a clinical one, so that patients can ultimately benefit from it. KEY POINTS: • Artificial intelligence (AI) research in medical imaging has a long history • The functioning, strengths and limitations of more classical AI methods is reviewed, together with that of more recent deep learning methods. • A perspective is provided on the potential impact of AI on radiology and on its evaluation from both technical and clinical points of view.

BENCHMARKING DEEP LEARNING FRAMEWORKS FOR THE CLASSIFICATION OF VERY HIGH RESOLUTION SATELLITE MULTISPECTRAL DATA

In this paper we evaluated deep-learning frameworks based on Convolutional Neural Networks for the accurate classification of multispectral remote sensing data. Certain state-of-the-art models have been tested on the publicly available SAT-4 and SAT-6 high resolution satellite multispectral datasets. In particular, the performed benchmark included the &lt;i&gt;AlexNet&lt;/i&gt;, &lt;i&gt;AlexNet-small&lt;/i&gt; and &lt;i&gt;VGG&lt;/i&gt; models which had been trained and applied to both datasets exploiting all the available spectral information. Deep Belief Networks, Autoencoders and other semi-supervised frameworks have been, also, compared. The high level features that were calculated from the tested models managed to classify the different land cover classes with significantly high accuracy rates &lt;i&gt;i.e.&lt;/i&gt;, above 99.9%. The experimental results demonstrate the great potentials of advanced deep-learning frameworks for the supervised classification of high resolution multispectral remote sensing data.