Handcrafted versus deep learning radiomics for prediction of cancer therapy response

Validated machine learning tools to distinguish immune checkpoint inhibitor, radiotherapy, COVID-19 and other infective pneumonitis

BACKGROUND

Pneumonitis is a well-described, potentially disabling, or fatal adverse effect associated with both immune checkpoint inhibitors (ICI) and thoracic radiotherapy. Accurate differentiation between checkpoint inhibitor pneumonitis (CIP) radiation pneumonitis (RP), and infective pneumonitis (IP) is crucial for swift, appropriate, and tailored management to achieve optimal patient outcomes. However, correct diagnosis is often challenging, owing to overlapping clinical presentations and radiological patterns.

METHODS

In this multi-centre study of 455 patients, we used machine learning with radiomic features extracted from chest CT imaging to develop and validate five models to distinguish CIP and RP from COVID-19, non-COVID-19 infective pneumonitis, and each other. Model performance was compared to that of two radiologists.

RESULTS

Models to distinguish RP from COVID-19, CIP from COVID-19 and CIP from non-COVID-19 IP out-performed radiologists (test set AUCs of 0.92 vs 0.8 and 0.8; 0.68 vs 0.43 and 0.4; 0.71 vs 0.55 and 0.63 respectively). Models to distinguish RP from non-COVID-19 IP and CIP from RP were not superior to radiologists but demonstrated modest performance, with test set AUCs of 0.81 and 0.8 respectively. The CIP vs RP model performed less well on patients with prior exposure to both ICI and radiotherapy (AUC 0.54), though the radiologists also had difficulty distinguishing this test cohort (AUC values 0.6 and 0.6).

CONCLUSION

Our results demonstrate the potential utility of such tools as a second or concurrent reader to support oncologists, radiologists, and chest physicians in cases of diagnostic uncertainty. Further research is required for patients with exposure to both ICI and thoracic radiotherapy.

A user-friendly deep learning application for accurate lung cancer diagnosis

BACKGROUND

Accurate diagnosis and subsequent delineated treatment planning require the experience of clinicians in the handling of their case numbers. However, applying deep learning in image processing is useful in creating tools that promise faster high-quality diagnoses, but the accuracy and precision of 3-D image processing from 2-D data may be limited by factors such as superposition of organs, distortion and magnification, and detection of new pathologies. The purpose of this research is to use radiomics and deep learning to develop a tool for lung cancer diagnosis.

METHODS

This study applies radiomics and deep learning in the diagnosis of lung cancer to help clinicians accurately analyze the images and thereby provide the appropriate treatment planning. 86 patients were recruited from Bach Mai Hospital, and 1012 patients were collected from an open-source database. First, deep learning has been applied in the process of segmentation by U-NET and cancer classification via the use of the DenseNet model. Second, the radiomics were applied for measuring and calculating diameter, surface area, and volume. Finally, the hardware also was designed by connecting between Arduino Nano and MFRC522 module for reading data from the tag. In addition, the displayed interface was created on a web platform using Python through Streamlit.

RESULTS

The applied segmentation model yielded a validation loss of 0.498, a train loss of 0.27, a cancer classification validation loss of 0.78, and a training accuracy of 0.98. The outcomes of the diagnostic capabilities of lung cancer (recognition and classification of lung cancer from chest CT scans) were quite successful.

CONCLUSIONS

The model provided means for storing and updating patients' data directly on the interface which allowed the results to be readily available for the health care providers. The developed system will improve clinical communication and information exchange. Moreover, it can manage efforts by generating correlated and coherent summaries of cancer diagnoses.

Application of deep learning in radiation therapy for cancer

In recent years, with the development of artificial intelligence, deep learning has been gradually applied to clinical treatment and research. It has also found its way into the applications in radiotherapy, a crucial method for cancer treatment. This study summarizes the commonly used and latest deep learning algorithms (including transformer, and diffusion models), introduces the workflow of different radiotherapy, and illustrates the application of different algorithms in different radiotherapy modules, as well as the defects and challenges of deep learning in the field of radiotherapy, so as to provide some help for the development of automatic radiotherapy for cancer.

Radiomics in Musculoskeletal Tumors

Sarcomas are heterogeneous rare tumors predominantly affecting the musculoskeletal (MSK) system. Due to significant variations in their natural history and variable response to conventional treatments, the discovery of novel diagnostic and prognostic biomarkers to guide therapeutic decision-making is an active and ongoing field of research. As new cellular, molecular, and metabolic biomarkers continue to be discovered, quantitative radiologic imaging is becoming increasingly important in sarcoma management. Radiomics offers the potential for discovering novel imaging diagnostic and predictive biomarkers using standard-of-care medical imaging. In this review, we detail the core concepts of radiomics and the application of radiomics to date in MSK sarcoma research. Also described are specific challenges related to radiomic studies, as well as viewpoints on clinical adoption and future perspectives in the field.

A critical moment in machine learning in medicine: on reproducible and interpretable learning

Over the past two decades, advances in computational power and data availability combined with increased accessibility to pre-trained models have led to an exponential rise in machine learning (ML) publications. While ML may have the potential to transform healthcare, this sharp increase in ML research output without focus on methodological rigor and standard reporting guidelines has fueled a reproducibility crisis. In addition, the rapidly growing complexity of these models compromises their interpretability, which currently impedes their successful and widespread clinical adoption. In medicine, where failure of such models may have severe implications for patients' health, the high requirements for accuracy, robustness, and interpretability confront ML researchers with a unique set of challenges. In this review, we discuss the semantics of reproducibility and interpretability, as well as related issues and challenges, and outline possible solutions to counteracting the "black box". To foster reproducibility, standard reporting guidelines need to be further developed and data or code sharing encouraged. Editors and reviewers may equally play a critical role by establishing high methodological standards and thus preventing the dissemination of low-quality ML publications. To foster interpretable learning, the use of simpler models more suitable for medical data can inform the clinician how results are generated based on input data. Model-agnostic explanation tools, sensitivity analysis, and hidden layer representations constitute further promising approaches to increase interpretability. Balancing model performance and interpretability are important to ensure clinical applicability. We have now reached a critical moment for ML in medicine, where addressing these issues and implementing appropriate solutions will be vital for the future evolution of the field.

Multi-institutional prognostic modeling of survival outcomes in NSCLC patients treated with first-line immunotherapy using radiomics

BACKGROUND

Immune checkpoint inhibitors (ICIs) have emerged as one of the most promising first-line therapeutics in the management of non-small cell lung cancer (NSCLC). However, only a subset of these patients responds to ICIs, highlighting the clinical need to develop better predictive and prognostic biomarkers. This study will leverage pre-treatment imaging profiles to develop survival risk models for NSCLC patients treated with first-line immunotherapy.

METHODS

Advanced NSCLC patients (n = 149) were retrospectively identified from two institutions who were treated with first-line ICIs. Radiomics features extracted from pretreatment imaging scans were used to build the predictive models for progression-free survival (PFS) and overall survival (OS). A compendium of five feature selection methods and seven machine learning approaches were utilized to build the survival risk models. The concordance index (C-index) was used to evaluate model performance.

RESULTS

From our results, we found several combinations of machine learning algorithms and feature selection methods to achieve similar performance. K-nearest neighbourhood (KNN) with ReliefF (RL) feature selection was the best-performing model to predict PFS (C-index = 0.61 and 0.604 in discovery and validation cohorts), while XGBoost with Mutual Information (MI) feature selection was the best-performing model for OS (C-index = 0.7 and 0.655 in discovery and validation cohorts).

CONCLUSION

The results of this study highlight the importance of implementing an appropriate feature selection method coupled with a machine learning strategy to develop robust survival models. With further validation of these models on external cohorts when available, this can have the potential to improve clinical decisions by systematically analyzing routine medical images.

AI in Orthodontics: Revolutionizing Diagnostics and Treatment Planning-A Comprehensive Review

The advent of artificial intelligence (AI) in medicine has transformed various medical specialties, including orthodontics. AI has shown promising results in enhancing the accuracy of diagnoses, treatment planning, and predicting treatment outcomes. Its usage in orthodontic practices worldwide has increased with the availability of various AI applications and tools. This review explores the principles of AI, its applications in orthodontics, and its implementation in clinical practice. A comprehensive literature review was conducted, focusing on AI applications in dental diagnostics, cephalometric evaluation, skeletal age determination, temporomandibular joint (TMJ) evaluation, decision making, and patient telemonitoring. Due to study heterogeneity, no meta-analysis was possible. AI has demonstrated high efficacy in all these areas, but variations in performance and the need for manual supervision suggest caution in clinical settings. The complexity and unpredictability of AI algorithms call for cautious implementation and regular manual validation. Continuous AI learning, proper governance, and addressing privacy and ethical concerns are crucial for successful integration into orthodontic practice.

Integration of dosimetric parameters, clinical factors, and radiomics to predict symptomatic radiation pneumonitis in lung cancer patients undergoing combined immunotherapy and radiotherapy

PURPOSE

This study aimed to combine clinical/dosimetric factors and handcrafted/deep learning radiomic features to establish a predictive model for symptomatic (grade ≥ 2) radiation pneumonitis (RP) in lung cancer patients who received immunotherapy followed by radiotherapy.

MATERIALS AND METHODS

This study retrospectively collected data of 73 lung cancer patients with prior receipt of ICIs who underwent thoracic radiotherapy (TRT). Of these 73 patients, 41 (56.2 %) developed symptomatic grade ≥ 2 RP. RP was defined per multidisciplinary clinician consensus using CTCAE v5.0. Regions of interest (ROIs) (from radiotherapy planning CT images) utilized herein were gross tumor volume (GTV), planning tumor volume (PTV), and PTV-GTV. Clinical/dosimetric (mean lung dose and V5-V30) parameters were collected, and 107 handcrafted radiomic (HCR) features were extracted from each ROI. Deep learning-based radiomic (DLR) features were also extracted based on pre-trained 3D residual network models. HCR models, Fusion HCR model, Fusion HCR + ResNet models, and Fusion HCR + ResNet + Clinical models were built and compared using the receiver operating characteristic (ROC) curve with measurement of the area under the curve (AUC). Five-fold cross-validation was performed to avoid model overfitting.

RESULTS

HCR models across various ROIs and the Fusion HCR model showed good predictive ability with AUCs from 0.740 to 0.808 and 0.740-0.802 in the training and testing cohorts, respectively. The addition of DLR features improved the effectiveness of HCR models (AUCs from 0.826 to 0.898 and 0.821-0.898 in both respective cohorts). The best performing prediction model (HCR + ResNet + Clinical) combined HCR & DLR features with 7 clinical/dosimetric characteristics and achieved an average AUC of 0.936 and 0.946 in both respective cohorts.

CONCLUSIONS

In patients undergoing combined immunotherapy/RT for lung cancer, integrating clinical/dosimetric factors and handcrafted/deep learning radiomic features can offer a high predictive capacity for RP, and merits further prospective validation.

Radiological artificial intelligence - predicting personalized immunotherapy outcomes in lung cancer

Personalized medicine has revolutionized approaches to treatment in the field of lung cancer by enabling therapies to be specific to each patient. However, physicians encounter an immense number of challenges in providing the optimal treatment regimen for the individual given the sheer complexity of clinical aspects such as tumor molecular profile, tumor microenvironment, expected adverse events, acquired or inherent resistance mechanisms, the development of brain metastases, the limited availability of biomarkers and the choice of combination therapy. The integration of innovative next-generation technologies such as deep learning-a subset of machine learning-and radiomics has the potential to transform the field by supporting clinical decision making in cancer treatment and the delivery of precision therapies while integrating numerous clinical considerations. In this review, we present a brief explanation of the available technologies, the benefits of using these technologies in predicting immunotherapy response in lung cancer, and the expected future challenges in the context of precision medicine.

Deep phenotyping the cervical spine: automatic characterization of cervical degenerative phenotypes based on T2-weighted MRI

PURPOSE

Radiological degenerative phenotypes provide insight into a patient's overall extent of disease and can be predictive for future pathological developments as well as surgical outcomes and complications. The objective of this study was to develop a reliable method for automatically classifying sagittal MRI image stacks of cervical spinal segments with respect to these degenerative phenotypes.

METHODS

We manually evaluated sagittal image data of the cervical spine of 873 patients (5182 motion segments) with respect to 5 radiological phenotypes. We then used this data set as ground truth for training a range of multi-class multi-label deep learning-based models to classify each motion segment automatically, on which we then performed hyper-parameter optimization.

RESULTS

The ground truth evaluations turned out to be relatively balanced for the labels disc displacement posterior, osteophyte anterior superior, osteophyte posterior superior, and osteophyte posterior inferior. Although we could not identify a single model that worked equally well across all the labels, the 3D-convolutional approach turned out to be preferable for classifying all labels.

CONCLUSIONS

Class imbalance in the training data and label noise made it difficult to achieve high predictive power for underrepresented classes. This shortcoming will be mitigated in the future versions by extending the training data set accordingly. Nevertheless, the classification performance rivals and in some cases surpasses that of human raters, while speeding up the evaluation process to only require a few seconds.

An MRI-based machine learning radiomics can predict short-term response to neoadjuvant chemotherapy in patients with cervical squamous cell carcinoma: A multicenter study

BACKGROUND AND PURPOSE

Neoadjuvant chemotherapy (NACT) has become an essential component of the comprehensive treatment of cervical squamous cell carcinoma (CSCC). However, not all patients respond to chemotherapy due to individual differences in sensitivity and tolerance to chemotherapy drugs. Therefore, accurately predicting the sensitivity of CSCC patients to NACT was vital for individual chemotherapy. This study aims to construct a machine learning radiomics model based on magnetic resonance imaging (MRI) to assess its efficacy in predicting NACT susceptibility among CSCC patients.

METHODS

This study included 234 patients with CSCC from two hospitals, who were divided into a training set (n = 180), a testing set (n = 20), and an external validation set (n = 34). Manual radiomic features were extracted from transverse section MRI images, and feature selection was performed using the recursive feature elimination (RFE) method. A prediction model was then generated using three machine learning algorithms, namely logistic regression, random forest, and support vector machines (SVM), for predicting NACT susceptibility. The model's performance was assessed based on the area under the receiver operating characteristic curve (AUC), accuracy, and sensitivity.

RESULTS

The SVM approach achieves the highest scores on both the testing set and the external validation set. In the testing set and external validation set, the AUC of the model was 0.88 and 0.764, and the accuracy was 0.90 and 0.853, the sensitivity was 0.93 and 0.962, respectively.

CONCLUSIONS

Machine learning radiomics models based on MRI images have achieved satisfactory performance in predicting the sensitivity of NACT in CSCC patients with high accuracy and robustness, which has great significance for the treatment and personalized medicine of CSCC patients.

Performance Analysis of Six Semi-Automated Tumour Delineation Methods on [18F] Fluorodeoxyglucose Positron Emission Tomography/Computed Tomography (FDG PET/CT) in Patients with Head and Neck Cancer

Background. Head and neck cancer (HNC) is the seventh most common neoplastic disorder at the global level. Contouring HNC lesions on [18F] Fluorodeoxyglucose positron emission tomography/computed tomography (FDG PET/CT) scans plays a fundamental role for diagnosis, risk assessment, radiotherapy planning and post-treatment evaluation. However, manual contouring is a lengthy and tedious procedure which requires significant effort from the clinician. Methods. We evaluated the performance of six hand-crafted, training-free methods (four threshold-based, two algorithm-based) for the semi-automated delineation of HNC lesions on FDG PET/CT. This study was carried out on a single-centre population of n=103 subjects, and the standard of reference was manual segmentation generated by nuclear medicine specialists. Figures of merit were the Sørensen-Dice coefficient (DSC) and relative volume difference (RVD). Results. Median DSC ranged between 0.595 and 0.792, median RVD between -22.0% and 87.4%. Click and draw and Nestle's methods achieved the best segmentation accuracy (median DSC, respectively, 0.792 ± 0.178 and 0.762 ± 0.107; median RVD, respectively, -21.6% ± 1270.8% and -32.7% ± 40.0%) and outperformed the other methods by a significant margin. Nestle's method also resulted in a lower dispersion of the data, hence showing stronger inter-patient stability. The accuracy of the two best methods was in agreement with the most recent state-of-the art results. Conclusions. Semi-automated PET delineation methods show potential to assist clinicians in the segmentation of HNC lesions on FDG PET/CT images, although manual refinement may sometimes be needed to obtain clinically acceptable ROIs.

Radiomics models for preoperative prediction of the histopathological grade of hepatocellular carcinoma: A systematic review and radiomics quality score assessment

OBJECTIVE

To systematically review the efficacy of radiomics models derived from computed tomography (CT) or magnetic resonance imaging (MRI) in preoperative prediction of the histopathological grade of hepatocellular carcinoma (HCC).

METHODS

Systematic literature search was performed at databases of PubMed, Web of Science, Embase, and Cochrane Library up to 30 December 2022. Studies that developed a radiomics model using preoperative CT/MRI for predicting the histopathological grade of HCC were regarded as eligible. A pre-defined table was used to extract the data related to study and patient characteristics, characteristics of radiomics modelling workflow, and the model performance metrics. Radiomics quality score and the Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) were applied for research quality evaluation.

RESULTS

Eleven eligible studies were included in this review, consisting of 2245 patients (range 53-494, median 165). No studies were prospectively designed and only two studies had an external test cohort. Half of the studies (five) used CT images and the other half MRI. The median number of extracted radiomics features was 328 (range: 40-1688), which was reduced to 11 (range: 1-50) after feature selection. The commonly used classifiers were logistic regression and support vector machine (both 4/11). When evaluated on the two external test cohorts, the area under the curve of the radiomics models was 0.70 and 0.77. The median radiomics quality score was 10 (range 2-13), corresponding to 28% (range 6-36%) of the full scale. Most studies showed an unclear risk of bias as evaluated by QUADAS-2.

CONCLUSION

Radiomics models based on preoperative CT or MRI have the potential to be used as an imaging biomarker for prediction of HCC histopathological grade. However, improved research and reporting quality is required to ensure sufficient reliability and reproducibility prior to implementation into clinical practice.

Radiomics and artificial intelligence for precision medicine in lung cancer treatment

Lung cancer is the leading cause of cancer-related deaths worldwide. It exhibits, at the mesoscopic scale, phenotypic characteristics that are generally indiscernible to the human eye but can be captured non-invasively on medical imaging as radiomic features, which can form a high dimensional data space amenable to machine learning. Radiomic features can be harnessed and used in an artificial intelligence paradigm to risk stratify patients, and predict for histological and molecular findings, and clinical outcome measures, thereby facilitating precision medicine for improving patient care. Compared to tissue sampling-driven approaches, radiomics-based methods are superior for being non-invasive, reproducible, cheaper, and less susceptible to intra-tumoral heterogeneity. This review focuses on the application of radiomics, combined with artificial intelligence, for delivering precision medicine in lung cancer treatment, with discussion centered on pioneering and groundbreaking works, and future research directions in the area.

Non-contrast CT synthesis using patch-based cycle-consistent generative adversarial network (Cycle-GAN) for radiomics and deep learning in the era of COVID-19

Handcrafted and deep learning (DL) radiomics are popular techniques used to develop computed tomography (CT) imaging-based artificial intelligence models for COVID-19 research. However, contrast heterogeneity from real-world datasets may impair model performance. Contrast-homogenous datasets present a potential solution. We developed a 3D patch-based cycle-consistent generative adversarial network (cycle-GAN) to synthesize non-contrast images from contrast CTs, as a data homogenization tool. We used a multi-centre dataset of 2078 scans from 1,650 patients with COVID-19. Few studies have previously evaluated GAN-generated images with handcrafted radiomics, DL and human assessment tasks. We evaluated the performance of our cycle-GAN with these three approaches. In a modified Turing-test, human experts identified synthetic vs acquired images, with a false positive rate of 67% and Fleiss' Kappa 0.06, attesting to the photorealism of the synthetic images. However, on testing performance of machine learning classifiers with radiomic features, performance decreased with use of synthetic images. Marked percentage difference was noted in feature values between pre- and post-GAN non-contrast images. With DL classification, deterioration in performance was observed with synthetic images. Our results show that whilst GANs can produce images sufficient to pass human assessment, caution is advised before GAN-synthesized images are used in medical imaging applications.

PET Radiomics and Response to Immunotherapy in Lung Cancer: A Systematic Review of the Literature

The aim of this review is to provide a comprehensive overview of the existing literature concerning the applications of positron emission tomography (PET) radiomics in lung cancer patient candidates or those undergoing immunotherapy.

MATERIALS AND METHODS

A systematic review was conducted on databases and web sources. English-language original articles were considered. The title and abstract were independently reviewed to evaluate study inclusion. Duplicate, out-of-topic, and review papers, or editorials, articles, and letters to editors were excluded. For each study, the radiomics analysis was assessed based on the radiomics quality score (RQS 2.0). The review was registered on the PROSPERO database with the number CRD42023402302.

RESULTS

Fifteen papers were included, thirteen were qualified as using conventional radiomics approaches, and two used deep learning radiomics. The content of each study was different; indeed, seven papers investigated the potential ability of radiomics to predict PD-L1 expression and tumor microenvironment before starting immunotherapy. Moreover, two evaluated the prediction of response, and four investigated the utility of radiomics to predict the response to immunotherapy. Finally, two papers investigated the prediction of adverse events due to immunotherapy.

CONCLUSIONS

Radiomics is promising for the evaluation of TME and for the prediction of response to immunotherapy, but some limitations should be overcome.

Unsupervised Machine Learning of MRI Radiomics Features Identifies Two Distinct Subgroups with Different Liver Function Reserve and Risks of Post-Hepatectomy Liver Failure in Patients with Hepatocellular Carcinoma

OBJECTIVE

To identify subgroups of patients with hepatocellular carcinoma (HCC) with different liver function reserves using an unsupervised machine-learning approach on the radiomics features from preoperative gadoxetic-acid-enhanced MRIs and to evaluate their association with the risk of post-hepatectomy liver failure (PHLF).

METHODS

Clinical data from 276 consecutive HCC patients who underwent liver resections between January 2017 and March 2019 were retrospectively collected. Radiomics features were extracted from the non-tumorous liver tissue at the gadoxetic-acid-enhanced hepatobiliary phase MRI. The reproducible and non-redundant features were selected for consensus clustering analysis to detect distinct subgroups. After that, clinical variables were compared between the identified subgroups to evaluate the clustering efficacy. The liver function reserve of the subgroups was compared and the correlations between the subgroups and PHLF, postoperative complications, and length of hospital stay were evaluated.

RESULTS

A total of 107 radiomics features were extracted and 37 were selected for unsupervised clustering analysis, which identified two distinct subgroups (138 patients in each subgroup). Compared with subgroup 1, subgroup 2 had significantly more patients with older age, albumin-bilirubin grades 2 and 3, a higher indocyanine green retention rate, and a lower indocyanine green plasma disappearance rate (all p < 0.05). Subgroup 2 was also associated with a higher risk of PHLF, postoperative complications, and longer hospital stays (>18 days) than that of subgroup 1, with an odds ratio of 2.83 (95% CI: 1.58-5.23), 2.41(95% CI: 1.15-5.35), and 2.14 (95% CI: 1.32-3.47), respectively. The odds ratio of our method was similar to the albumin-bilirubin grade for postoperative complications and length of hospital stay (2.41 vs. 2.29 and 2.14 vs. 2.16, respectively), but was inferior for PHLF (2.83 vs. 4.55).

CONCLUSIONS

Based on the radiomics features of gadoxetic-acid-enhanced MRI, unsupervised clustering analysis identified two distinct subgroups with different liver function reserves and risks of PHLF in HCC patients. Future studies are required to validate our findings.

Functional-structural sub-region graph convolutional network (FSGCN): Application to the prognosis of head and neck cancer with PET/CT imaging

BACKGROUND AND OBJECTIVE

Accurate risk stratification is crucial for enabling personalized treatment for head and neck cancer (HNC). Current PET/CT image-based prognostic methods include radiomics analysis and convolutional neural network (CNN), while extracting radiomics or deep features in grid Euclidean space has inherent limitations for risk stratification. Here, we propose a functional-structural sub-region graph convolutional network (FSGCN) for accurate risk stratification of HNC.

METHODS

This study collected 642 patients from 8 different centers in The Cancer Imaging Archive (TCIA), 507 patients from 5 centers were used for training, and 135 patients from 3 centers were used for testing. The tumor was first clustered into multiple sub-regions by using PET and CT voxel information, and radiomics features were extracted from each sub-region to characterize its functional and structural information, a graph was then constructed to format the relationship/difference among different sub-regions in non-Euclidean space for each patient, followed by a residual gated graph convolutional network, the prognostic score was finally generated to predict the progression-free survival (PFS).

RESULTS

In the testing cohort, compared with radiomics or FSGCN or clinical model alone, the model PETCT^Fea_CT^ROI + Cli that integrates FSGCN prognostic score and clinical parameter achieved the highest C-index and AUC of 0.767 (95% CI: 0.759-0.774) and 0.781 (95% CI: 0.774-0.788), respectively for PFS prediction. Besides, it also showed good prognostic performance on the secondary endpoints OS, RFS, and MFS in the testing cohort, with C-index of 0.786 (95% CI: 0.778-0.795), 0.775 (95% CI: 0.767-0.782) and 0.781 (95% CI: 0.772-0.789), respectively.

CONCLUSIONS

The proposed FSGCN can better capture the metabolic or anatomic difference/interaction among sub-regions of the whole tumor imaged with PET/CT. Extensive multi-center experiments demonstrated its capability and generalization of prognosis prediction in HNC over conventional radiomics analysis.

Deep learning radiomics under multimodality explore association between muscle/fat and metastasis and survival in breast cancer patients

Sarcopenia is correlated with poor clinical outcomes in breast cancer (BC) patients. However, there is no precise quantitative study on the correlation between body composition changes and BC metastasis and survival. The present study proposed a deep learning radiomics (DLR) approach to investigate the effects of muscle and fat on distant metastasis and death outcomes in BC patients. Image feature extraction was performed on 4th thoracic vertebra (T4) and 11th thoracic vertebra (T11) on computed tomography (CT) image levels by DLR, and image features were combined with clinical information to predict distant metastasis in BC patients. Clinical information combined with DLR significantly predicted distant metastasis in BC patients. In the test cohort, the area under the curve of model performance on clinical information combined with DLR was 0.960 (95% CI: 0.942-0.979, P < 0.001). The patients with distant metastases had a lower pectoral muscle index in T4 (PMI/T4) than in patients without metastases. PMI/T4 and visceral fat tissue area in T11 (VFA/T11) were independent prognostic factors for the overall survival in BC patients. The pectoralis muscle area in T4 (PMA/T4) and PMI/T4 is an independent prognostic factor for distant metastasis-free survival in BC patients. The current study further confirmed that muscle/fat of T4 and T11 levels have a significant effect on the distant metastasis of BC. Appending the network features of T4 and T11 to the model significantly enhances the prediction performance of distant metastasis of BC, providing a valuable biomarker for the early treatment of BC patients.

Radiomics-based approaches outperform visual analysis for differentiating lipoma from atypical lipomatous tumors: a review

BACKGROUND

Differentiating atypical lipomatous tumors (ALTs) and well-differentiated liposarcomas (WDLs) from benign lipomatous lesions is important for guiding clinical management, though conventional visual analysis of these lesions is challenging due to overlap of imaging features. Radiomics-based approaches may serve as a promising alternative and/or supplementary diagnostic approach to conventional imaging.

PURPOSE

The purpose of this study is to review the practice of radiomics-based imaging and systematically evaluate the literature available for studies evaluating radiomics applied to differentiating ALTs/WDLs from benign lipomas.

REVIEW

A background review of the radiomic workflow is provided, outlining the steps of image acquisition, segmentation, feature extraction, and model development. Subsequently, a systematic review of MEDLINE, EMBASE, Scopus, the Cochrane Library, and the grey literature was performed from inception to June 2022 to identify size studies using radiomics for differentiating ALTs/WDLs from benign lipomas. Radiomic models were shown to outperform conventional analysis in all but one model with a sensitivity ranging from 68 to 100% and a specificity ranging from 84 to 100%. However, current approaches rely on user input and no studies used a fully automated method for segmentation, contributing to interobserver variability and decreasing time efficiency.

CONCLUSION

Radiomic models may show improved performance for differentiating ALTs/WDLs from benign lipomas compared to conventional analysis. However, considerable variability between radiomic approaches exists and future studies evaluating a standardized radiomic model with a multi-institutional study design and preferably fully automated segmentation software are needed before clinical application can be more broadly considered.

Gross tumour volume radiomics for prognostication of recurrence & death following radical radiotherapy for NSCLC

Recurrence occurs in up to 36% of patients treated with curative-intent radiotherapy for NSCLC. Identifying patients at higher risk of recurrence for more intensive surveillance may facilitate the earlier introduction of the next line of treatment. We aimed to use radiotherapy planning CT scans to develop radiomic classification models that predict overall survival (OS), recurrence-free survival (RFS) and recurrence two years post-treatment for risk-stratification. A retrospective multi-centre study of >900 patients receiving curative-intent radiotherapy for stage I-III NSCLC was undertaken. Models using radiomic and/or clinical features were developed, compared with 10-fold cross-validation and an external test set, and benchmarked against TNM-stage. Respective validation and test set AUCs (with 95% confidence intervals) for the radiomic-only models were: (1) OS: 0.712 (0.592–0.832) and 0.685 (0.585–0.784), (2) RFS: 0.825 (0.733–0.916) and 0.750 (0.665–0.835), (3) Recurrence: 0.678 (0.554–0.801) and 0.673 (0.577–0.77). For the combined models: (1) OS: 0.702 (0.583–0.822) and 0.683 (0.586–0.78), (2) RFS: 0.805 (0.707–0.903) and 0·755 (0.672–0.838), (3) Recurrence: 0·637 (0.51–0.·765) and 0·738 (0.649–0.826). Kaplan-Meier analyses demonstrate OS and RFS difference of >300 and >400 days respectively between low and high-risk groups. We have developed validated and externally tested radiomic-based prediction models. Such models could be integrated into the routine radiotherapy workflow, thus informing a personalised surveillance strategy at the point of treatment. Our work lays the foundations for future prospective clinical trials for quantitative personalised risk-stratification for surveillance following curative-intent radiotherapy for NSCLC.

Prediction of 5-year progression-free survival in advanced nasopharyngeal carcinoma with pretreatment PET/CT using multi-modality deep learning-based radiomics

Objective

Deep learning-based radiomics (DLR) has achieved great success in medical image analysis and has been considered a replacement for conventional radiomics that relies on handcrafted features. In this study, we aimed to explore the capability of DLR for the prediction of 5-year progression-free survival (PFS) in advanced nasopharyngeal carcinoma (NPC) using pretreatment PET/CT images.

Methods

A total of 257 patients (170/87 patients in internal/external cohorts) with advanced NPC (TNM stage III or IVa) were enrolled. We developed an end-to-end multi-modality DLR model, in which a 3D convolutional neural network was optimized to extract deep features from pretreatment PET/CT images and predict the probability of 5-year PFS. The TNM stage, as a high-level clinical feature, could be integrated into our DLR model to further improve the prognostic performance. For a comparison between conventional radiomics and DLR, 1,456 handcrafted features were extracted, and optimal conventional radiomics methods were selected from 54 cross-combinations of six feature selection methods and nine classification methods. In addition, risk group stratification was performed with clinical signature, conventional radiomics signature, and DLR signature.

Results

Our multi-modality DLR model using both PET and CT achieved higher prognostic performance (area under the receiver operating characteristic curve (AUC) = 0.842 ± 0.034 and 0.823 ± 0.012 for the internal and external cohorts) than the optimal conventional radiomics method (AUC = 0.796 ± 0.033 and 0.782 ± 0.012). Furthermore, the multi-modality DLR model outperformed single-modality DLR models using only PET (AUC = 0.818 ± 0.029 and 0.796 ± 0.009) or only CT (AUC = 0.657 ± 0.055 and 0.645 ± 0.021). For risk group stratification, the conventional radiomics signature and DLR signature enabled significant difference between the high- and low-risk patient groups in both the internal and external cohorts (p < 0.001), while the clinical signature failed in the external cohort (p = 0.177).

Conclusion

Our study identified potential prognostic tools for survival prediction in advanced NPC, which suggests that DLR could provide complementary values to the current TNM staging.

Form Factors as Potential Imaging Biomarkers to Differentiate Benign vs. Malignant Lung Lesions on CT Scans

Indeterminate lung nodules detected on CT scans are common findings in clinical practice. Their correct assessment is critical, as early diagnosis of malignancy is crucial to maximise the treatment outcome. In this work, we evaluated the role of form factors as imaging biomarkers to differentiate benign vs. malignant lung lesions on CT scans. We tested a total of three conventional imaging features, six form factors, and two shape features for significant differences between benign and malignant lung lesions on CT scans. The study population consisted of 192 lung nodules from two independent datasets, containing 109 (38 benign, 71 malignant) and 83 (42 benign, 41 malignant) lung lesions, respectively. The standard of reference was either histological evaluation or stability on radiological followup. The statistical significance was determined via the Mann-Whitney U nonparametric test, and the ability of the form factors to discriminate a benign vs. a malignant lesion was assessed through multivariate prediction models based on Support Vector Machines. The univariate analysis returned four form factors (Angelidakis compactness and flatness, Kong flatness, and maximum projection sphericity) that were significantly different between the benign and malignant group in both datasets. In particular, we found that the benign lesions were on average flatter than the malignant ones; conversely, the malignant ones were on average more compact (isotropic) than the benign ones. The multivariate prediction models showed that adding form factors to conventional imaging features improved the prediction accuracy by up to 14.5 pp. We conclude that form factors evaluated on lung nodules on CT scans can improve the differential diagnosis between benign and malignant lesions.

Deep Radiomics-based Approach to the Diagnosis of Osteoporosis Using Hip Radiographs

PURPOSE

To develop and validate deep radiomics models for the diagnosis of osteoporosis using hip radiographs.

MATERIALS AND METHODS

A deep radiomics model was developed using 4924 hip radiographs from 4308 patients (3632 women; mean age, 62 years ± 13 [SD]) obtained between September 2009 and April 2020. Ten deep features, 16 texture features, and three clinical features were used to train the model. T score measured with dual-energy x-ray absorptiometry was used as a reference standard for osteoporosis. Seven deep radiomics models that combined different types of features were developed: clinical (model C); texture (model T); deep (model D); texture and clinical (model TC); deep and clinical (model DC); deep and texture (model DT); and deep, texture, and clinical features (model DTC). A total of 444 hip radiographs obtained between January 2019 and April 2020 from another institution were used for the external test. Six radiologists performed an observer performance test. The area under the receiver operating characteristic curve (AUC) was used to evaluate diagnostic performance.

RESULTS

For the external test set, model D (AUC, 0.92; 95% CI: 0.89, 0.95) demonstrated higher diagnostic performance than model T (AUC, 0.77; 95% CI: 0.70, 0.83; adjusted P < .001). Model DC (AUC, 0.95; 95% CI: 0.92, 0.97; adjusted P = .03) and model DTC (AUC, 0.95; 95% CI: 0.92, 0.97; adjusted P = .048) showed improved diagnostic performance compared with model D. When observer performance without and with the assistance of the model DTC prediction was compared, performance improved from a mean AUC of 0.77 to 0.87 (P = .002).

CONCLUSION

Deep radiomics models using hip radiographs could be used to diagnose osteoporosis with high performance.Keywords: Skeletal-Appendicular, Hip, Absorptiometry/Bone Densitometry© RSNA, 2022.

DeepMTS: Deep Multi-task Learning for Survival Prediction in Patients With Advanced Nasopharyngeal Carcinoma Using Pretreatment PET/CT

Nasopharyngeal Carcinoma (NPC) is a malignant epithelial cancer arising from the nasopharynx. Survival prediction is a major concern for NPC patients, as it provides early prognostic information to plan treatments. Recently, deep survival models based on deep learning have demonstrated the potential to outperform traditional radiomics-based survival prediction models. Deep survival models usually use image patches covering the whole target regions (e.g., nasopharynx for NPC) or containing only segmented tumor regions as the input. However, the models using the whole target regions will also include non-relevant background information, while the models using segmented tumor regions will disregard potentially prognostic information existing out of primary tumors (e.g., local lymph node metastasis and adjacent tissue invasion). In this study, we propose a 3D end-to-end Deep Multi-Task Survival model (DeepMTS) for joint survival prediction and tumor segmentation in advanced NPC from pretreatment PET/CT. Our novelty is the introduction of a hard-sharing segmentation backbone to guide the extraction of local features related to the primary tumors, which reduces the interference from non-relevant background information. In addition, we also introduce a cascaded survival network to capture the prognostic information existing out of primary tumors and further leverage the global tumor information (e.g., tumor size, shape, and locations) derived from the segmentation backbone. Our experiments with two clinical datasets demonstrate that our DeepMTS can consistently outperform traditional radiomics-based survival prediction models and existing deep survival models.

Spherical Convolutional Neural Networks for Survival Rate Prediction in Cancer Patients

Objective

Survival Rate Prediction (SRP) is a valuable tool to assist in the clinical diagnosis and treatment planning of lung cancer patients. In recent years, deep learning (DL) based methods have shown great potential in medical image processing in general and SRP in particular. This study proposes a fully-automated method for SRP from computed tomography (CT) images, which combines an automatic segmentation of the tumor and a DL-based method for extracting rotational-invariant features.

Methods

In the first stage, the tumor is segmented from the CT image of the lungs. Here, we use a deep-learning-based method that entails a variational autoencoder to provide more information to a U-Net segmentation model. Next, the 3D volumetric image of the tumor is projected onto 2D spherical maps. These spherical maps serve as inputs for a spherical convolutional neural network that approximates the log risk for a generalized Cox proportional hazard model.

Results

The proposed method is compared with 17 baseline methods that combine different feature sets and prediction models using three publicly-available datasets: Lung1 (n=422), Lung3 (n=89), and H&N1 (n=136). We observed comparable C-index scores compared to the best-performing baseline methods in a 5-fold cross-validation on Lung1 (0.59 ± 0.03 vs. 0.62 ± 0.04). In comparison, it slightly outperforms all methods in inter-data set evaluation (0.64 vs. 0.63). The best-performing method from the first experiment reduced its performance to 0.61 and 0.62 for Lung3 and H&N1, respectively.

Discussion

The experiments suggest that the performance of spherical features is comparable with previous approaches, but they generalize better when applied to unseen datasets. That might imply that orientation-independent shape features are relevant for SRP. The performance of the proposed method was very similar, using manual and automatic segmentation methods. This makes the proposed model useful in cases where expert annotations are not available or difficult to obtain.

PET/CT Based EGFR Mutation Status Classification of NSCLC Using Deep Learning Features and Radiomics Features

Purpose: This study aimed to compare the performance of radiomics and deep learning in predicting EGFR mutation status in patients with lung cancer based on PET/CT images, and tried to explore a model with excellent prediction performance to accurately predict EGFR mutation status in patients with non-small cell lung cancer (NSCLC). Method: PET/CT images of 194 NSCLC patients from Xijing Hospital were collected and divided into a training set and a validation set according to the ratio of 7:3. Statistics were made on patients' clinical characteristics, and a large number of features were extracted based on their PET/CT images (4306 radiomics features and 2048 deep learning features per person) with the pyradiomics toolkit and 3D convolutional neural network. Then a radiomics model (RM), a deep learning model (DLM), and a hybrid model (HM) were established. The performance of the three models was compared by receiver operating characteristic (ROC) curves, sensitivity, specificity, accuracy, calibration curves, and decision curves. In addition, a nomogram based on a deep learning score (DS) and the most significant clinical characteristic was plotted. Result: In the training set composed of 138 patients (64 with EGFR mutation and 74 without EGFR mutation), the area under the ROC curve (AUC) of HM (0.91, 95% CI: 0.86-0.96) was higher than that of RM (0.82, 95% CI: 0.75-0.89) and DLM (0.90, 95% CI: 0.85-0.95). In the validation set composed of 57 patients (32 with EGFR mutation and 25 without EGFR mutation), the AUC of HM (0.85, 95% CI: 0.77-0.93) was also higher than that of RM (0.68, 95% CI: 0.52-0.84) and DLM (0.79, 95% CI: 0.67-0.91). In all, HM achieved better diagnostic performance in predicting EGFR mutation status in NSCLC patients than two other models. Conclusion: Our study showed that the deep learning model based on PET/CT images had better performance than radiomics model in diagnosing EGFR mutation status of NSCLC patients based on PET/CT images. Combined with the most statistically significant clinical characteristic (smoking) and deep learning features, our hybrid model had better performance in predicting EGFR mutation types of patients than two other models, which could enable NSCLC patients to choose more personalized treatment schemes.

MR-Guided Adaptive Radiotherapy for OAR Sparing in Head and Neck Cancers

Simple Summary

Normal tissue toxicities in head and neck cancer persist as a cause of decreased quality of life and are associated with poorer treatment outcomes. The aim of this article is to review organ at risk (OAR) sparing approaches available in MR-guided adaptive radiotherapy and present future developments which hope to improve treatment outcomes. Increasing the spatial conformity of dose distributions in radiotherapy is an important first step in reducing normal tissue toxicities, and MR-guided treatment devices presents a new opportunity to use biological information to drive treatment decisions on a personalized basis.

Abstract

MR-linac devices offer the potential for advancements in radiotherapy (RT) treatment of head and neck cancer (HNC) by using daily MR imaging performed at the time and setup of treatment delivery. This article aims to present a review of current adaptive RT (ART) methods on MR-Linac devices directed towards the sparing of organs at risk (OAR) and a view of future adaptive techniques seeking to improve the therapeutic ratio. This ratio expresses the relationship between the probability of tumor control and the probability of normal tissue damage and is thus an important conceptual metric of success in the sparing of OARs. Increasing spatial conformity of dose distributions to target volume and OARs is an initial step in achieving therapeutic improvements, followed by the use of imaging and clinical biomarkers to inform the clinical decision-making process in an ART paradigm. Pre-clinical and clinical findings support the incorporation of biomarkers into ART protocols and investment into further research to explore imaging biomarkers by taking advantage of the daily MR imaging workflow. A coherent understanding of this road map for RT in HNC is critical for directing future research efforts related to sparing OARs using image-guided radiotherapy (IGRT).

CT Reconstruction Kernels and the Effect of Pre- and Post-Processing on the Reproducibility of Handcrafted Radiomic Features

Handcrafted radiomics features (HRFs) are quantitative features extracted from medical images to decode biological information to improve clinical decision making. Despite the potential of the field, limitations have been identified. The most important identified limitation, currently, is the sensitivity of HRF to variations in image acquisition and reconstruction parameters. In this study, we investigated the use of Reconstruction Kernel Normalization (RKN) and ComBat harmonization to improve the reproducibility of HRFs across scans acquired with different reconstruction kernels. A set of phantom scans (n = 28) acquired on five different scanner models was analyzed. HRFs were extracted from the original scans, and scans were harmonized using the RKN method. ComBat harmonization was applied on both sets of HRFs. The reproducibility of HRFs was assessed using the concordance correlation coefficient. The difference in the number of reproducible HRFs in each scenario was assessed using McNemar’s test. The majority of HRFs were found to be sensitive to variations in the reconstruction kernels, and only six HRFs were found to be robust with respect to variations in reconstruction kernels. The use of RKN resulted in a significant increment in the number of reproducible HRFs in 19 out of the 67 investigated scenarios (28.4%), while the ComBat technique resulted in a significant increment in 36 (53.7%) scenarios. The combination of methods resulted in a significant increment in 53 (79.1%) scenarios compared to the HRFs extracted from original images. Since the benefit of applying the harmonization methods depended on the data being harmonized, reproducibility analysis is recommended before performing radiomics analysis. For future radiomics studies incorporating images acquired with similar image acquisition and reconstruction parameters, except for the reconstruction kernels, we recommend the systematic use of the pre- and post-processing approaches (respectively, RKN and ComBat).

Progression Free Survival Prediction for Head and Neck Cancer Using Deep Learning Based on Clinical and PET/CT Imaging Data

Determining progression-free survival (PFS) for head and neck squamous cell carcinoma (HNSCC) patients is a challenging but pertinent task that could help stratify patients for improved overall outcomes. PET/CT images provide a rich source of anatomical and metabolic data for potential clinical biomarkers that would inform treatment decisions and could help improve PFS. In this study, we participate in the 2021 HECKTOR Challenge to predict PFS in a large dataset of HNSCC PET/CT images using deep learning approaches. We develop a series of deep learning models based on the DenseNet architecture using a negative log-likelihood loss function that utilizes PET/CT images and clinical data as separate input channels to predict PFS in days. Internal model validation based on 10-fold cross-validation using the training data (N = 224) yielded C-index values up to 0.622 (without) and 0.842 (with) censoring status considered in C-index computation, respectively. We then implemented model ensembling approaches based on the training data cross-validation folds to predict the PFS of the test set patients (N = 101). External validation on the test set for the best ensembling method yielded a C-index value of 0.694, placing 2^nd in the competition. Our results are a promising example of how deep learning approaches can effectively utilize imaging and clinical data for medical outcome prediction in HNSCC, but further work in optimizing these processes is needed.

Segmentation Uncertainty Estimation as a Sanity Check for Image Biomarker Studies

Problem. Image biomarker analysis, also known as radiomics, is a tool for tissue characterization and treatment prognosis that relies on routinely acquired clinical images and delineations. Due to the uncertainty in image acquisition, processing, and segmentation (delineation) protocols, radiomics often lack reproducibility. Radiomics harmonization techniques have been proposed as a solution to reduce these sources of uncertainty and/or their influence on the prognostic model performance. A relevant question is how to estimate the protocol-induced uncertainty of a specific image biomarker, what the effect is on the model performance, and how to optimize the model given the uncertainty. Methods. Two non-small cell lung cancer (NSCLC) cohorts, composed of 421 and 240 patients, respectively, were used for training and testing. Per patient, a Monte Carlo algorithm was used to generate three hundred synthetic contours with a surface dice tolerance measure of less than 1.18 mm with respect to the original GTV. These contours were subsequently used to derive 104 radiomic features, which were ranked on their relative sensitivity to contour perturbation, expressed in the parameter η. The top four (low η) and the bottom four (high η) features were selected for two models based on the Cox proportional hazards model. To investigate the influence of segmentation uncertainty on the prognostic model, we trained and tested the setup in 5000 augmented realizations (using a Monte Carlo sampling method); the log-rank test was used to assess the stratification performance and stability of segmentation uncertainty. Results. Although both low and high η setup showed significant testing set log-rank p-values (p = 0.01) in the original GTV delineations (without segmentation uncertainty introduced), in the model with high uncertainty, to effect ratio, only around 30% of the augmented realizations resulted in model performance with p < 0.05 in the test set. In contrast, the low η setup performed with a log-rank p < 0.05 in 90% of the augmented realizations. Moreover, the high η setup classification was uncertain in its predictions for 50% of the subjects in the testing set (for 80% agreement rate), whereas the low η setup was uncertain only in 10% of the cases. Discussion. Estimating image biomarker model performance based only on the original GTV segmentation, without considering segmentation, uncertainty may be deceiving. The model might result in a significant stratification performance, but can be unstable for delineation variations, which are inherent to manual segmentation. Simulating segmentation uncertainty using the method described allows for more stable image biomarker estimation, selection, and model development. The segmentation uncertainty estimation method described here is universal and can be extended to estimate other protocol uncertainties (such as image acquisition and pre-processing).

Computed tomography-based deep-learning prediction of induction chemotherapy treatment response in locally advanced nasopharyngeal carcinoma

BACKGROUND

Deep learning methods have great potential to predict treatment response. The objective of this study was to evaluate and validate the predictive performance of the computed tomography (CT)-based model using deep learning features for identification of responders and nonresponders to induction chemotherapy (IC) in nasopharyngeal carcinoma (NPC).

MATERIALS AND METHODS

All eligible patients were included retrospectively between January 2012 and December 2018, and assigned to the training (n = 208) or the testing cohort (n = 89). We extracted deep learning features of six pretrained convolutional neural networks (CNNs) via transfer learning method, and handcrafted radiomics features manually. Support vector machine (SVM) was adopted as the classifier. All predictive models were evaluated using the area under the receiver operating characteristics curve (AUC), by which an optimal model was selected. We also built clinical and clinical-radiological models for comparison.

RESULTS

The model with features extracted from ResNet50 (RN-SVM) had optimal performance among all models with features extracted from pretrained CNNs with an AUC of 0.811, accuracy of 68.54%, sensitivity of 61.54%, specificity of 87.50%, positive predictive value (PPV) of 93.02%, and negative predictive value (NPV) of 45.65% in the testing cohort. The handcrafted radiomics model was slightly inferior to the RN-SVM model with an AUC of 0.663 and accuracy of 60.67% in the testing cohort. All the imaging-derived models had better predictive performance than the clinical model.

CONCLUSION

The noninvasive deep learning method could provide efficient prediction of treatment response to IC in locally advanced NPC and might be a practicable approach in therapeutic strategy decision-making.

A review of explainable and interpretable AI with applications in COVID-19 imaging

The development of medical imaging artificial intelligence (AI) systems for evaluating COVID‐19 patients has demonstrated potential for improving clinical decision making and assessing patient outcomes during the recent COVID‐19 pandemic. These have been applied to many medical imaging tasks, including disease diagnosis and patient prognosis, as well as augmented other clinical measurements to better inform treatment decisions. Because these systems are used in life‐or‐death decisions, clinical implementation relies on user trust in the AI output. This has caused many developers to utilize explainability techniques in an attempt to help a user understand when an AI algorithm is likely to succeed as well as which cases may be problematic for automatic assessment, thus increasing the potential for rapid clinical translation. AI application to COVID‐19 has been marred with controversy recently. This review discusses several aspects of explainable and interpretable AI as it pertains to the evaluation of COVID‐19 disease and it can restore trust in AI application to this disease. This includes the identification of common tasks that are relevant to explainable medical imaging AI, an overview of several modern approaches for producing explainable output as appropriate for a given imaging scenario, a discussion of how to evaluate explainable AI, and recommendations for best practices in explainable/interpretable AI implementation. This review will allow developers of AI systems for COVID‐19 to quickly understand the basics of several explainable AI techniques and assist in the selection of an approach that is both appropriate and effective for a given scenario.

The application of radiomics in laryngeal cancer

OBJECTIVES

Radiomics is the conversion of medical images into quantitative high-dimensional data. Laryngeal cancer, one of the most common head and neck cancers, has risen globally by 58.7%. CT, MRI and PET are acquired during the diagnostic process providing potential data for radiomic analysis and correlation with outcomes.This review aims to examine the applications of this technique to laryngeal cancer and the future considerations for translation into clinical practice.

METHODS

A comprehensive systematic review-informed search of the MEDLINE and EMBASE databases was undertaken. Keywords "laryngeal cancer" OR "larynx" OR "larynx cancer" OR "head and neck cancer" were combined with "radiomic" OR "signature" OR "machine learning" OR "artificial intelligence". Additional articles were obtained from bibliographies using the "snowball method".

RESULTS

The included studies (n = 15) demonstrated that radiomic features are significantly associated with various clinical outcomes (including stage, overall survival, treatment response, progression-free survival) and that predictive models incorporating radiomic features are superior to those that do not. Two studies demonstrated radiomics could improve laryngeal cancer staging whilst 12 studies affirmed its predictive capability for clinical outcomes.

CONCLUSIONS

Radiomics has potential for improving multiple aspects of laryngeal cancer care; however, the heterogeneous cohorts and lack of data on laryngeal cancer exclusively inhibits firm conclusions. Large prospective well-designed studies in laryngeal cancer are required to progress this field. Furthermore, to implement radiomics into clinical practice, a unified research effort is required to standardise radiomics practice.

ADVANCES IN KNOWLEDGE

This review has highlighted the value of radiomics in enhancing laryngeal cancer care (including staging, prognosis and predicting treatment response).

Machine Learning and Radiomics Applications in Esophageal Cancers Using Non-Invasive Imaging Methods-A Critical Review of Literature

Esophageal cancer (EC) is of public health significance as one of the leading causes of cancer death worldwide. Accurate staging, treatment planning and prognostication in EC patients are of vital importance. Recent advances in machine learning (ML) techniques demonstrate their potential to provide novel quantitative imaging markers in medical imaging. Radiomics approaches that could quantify medical images into high-dimensional data have been shown to improve the imaging-based classification system in characterizing the heterogeneity of primary tumors and lymph nodes in EC patients. In this review, we aim to provide a comprehensive summary of the evidence of the most recent developments in ML application in imaging pertinent to EC patient care. According to the published results, ML models evaluating treatment response and lymph node metastasis achieve reliable predictions, ranging from acceptable to outstanding in their validation groups. Patients stratified by ML models in different risk groups have a significant or borderline significant difference in survival outcomes. Prospective large multi-center studies are suggested to improve the generalizability of ML techniques with standardized imaging protocols and harmonization between different centers.

Radiomic biomarkers of tumor immune biology and immunotherapy response

Immunotherapies are leading to improved outcomes for many cancers, including those with devastating prognoses. As therapies like immune checkpoint inhibitors (ICI) become a mainstay in treatment regimens, many concurrent challenges have arisen - for instance, delineating clinical responders from non-responders. Predicting response has proven to be difficult given a lack of consistent and accurate biomarkers, heterogeneity of the tumor microenvironment (TME), and a poor understanding of resistance mechanisms. For the most part, imaging data have remained an untapped, yet abundant, resource to address these challenges. In recent years, quantitative image analyses have highlighted the utility of medical imaging in predicting tumor phenotypes, prognosis, and therapeutic response. These studies have been fueled by an explosion of resources in high-throughput mining of image features (i.e. radiomics) and artificial intelligence. In this review, we highlight current progress in radiomics to understand tumor immune biology and predict clinical responses to immunotherapies. We also discuss limitations in these studies and future directions for the field, particularly if high-dimensional imaging data are to play a larger role in precision medicine.

Radiomics in Renal Cell Carcinoma-A Systematic Review and Meta-Analysis

Simple Summary

Radiomics may answer questions where the conventional interpretation of medical imaging has limitations. The aim of our systematic review and meta-analysis was to assess the (current) status of evidence in the application of radiomics in the field of renal masses. We focused on its role in diagnosis, sub-entity discrimination and treatment response assessment in renal cell carcinoma (RCC) and benign renal masses. Our quantitative synthesis showed promising results in discrimination of tumor dignity, nevertheless, the value added to human assessment remains unclear and should be the focus of future research. Furthermore, the benefit regarding treatment response assessment remains unclear as well, since the existing studies are investigating already abandoned systemic therapies (ST), which no longer represent the current “reference” standard. Open science could enable to establish technical and clinical validity of radiomic signatures prior to the incorporation of radiomics into everyday clinical practice.

Abstract

Radiomics may increase the diagnostic accuracy of medical imaging for localized and metastatic RCC (mRCC). A systematic review and meta-analysis was performed. Doing so, we comprehensively searched literature databases until May 2020. Studies investigating the diagnostic value of radiomics in differentiation of localized renal tumors and assessment of treatment response to ST in mRCC were included and assessed with respect to their quality using the radiomics quality score (RQS). A total of 113 out of 1098 identified studies met the criteria and were included in qualitative synthesis. Median RQS of all studies was 13.9% (5.0 points, IQR 0.25–7.0 points), and RQS increased over time. Thirty studies were included into the quantitative synthesis: For distinguishing angiomyolipoma, oncocytoma or unspecified benign tumors from RCC, the random effects model showed a log odds ratio (OR) of 2.89 (95%-CI 2.40–3.39, p < 0.001), 3.08 (95%-CI 2.09–4.06, p < 0.001) and 3.57 (95%-CI 2.69–4.45, p < 0.001), respectively. For the general discrimination of benign tumors from RCC log OR was 3.17 (95%-CI 2.73–3.62, p < 0.001). Inhomogeneity of the available studies assessing treatment response in mRCC prevented any meaningful meta-analysis. The application of radiomics seems promising for discrimination of renal tumor dignity. Shared data and open science may assist in improving reproducibility of future studies.

Deep learning classification of lung cancer histology using CT images

Tumor histology is an important predictor of therapeutic response and outcomes in lung cancer. Tissue sampling for pathologist review is the most reliable method for histology classification, however, recent advances in deep learning for medical image analysis allude to the utility of radiologic data in further describing disease characteristics and for risk stratification. In this study, we propose a radiomics approach to predicting non-small cell lung cancer (NSCLC) tumor histology from non-invasive standard-of-care computed tomography (CT) data. We trained and validated convolutional neural networks (CNNs) on a dataset comprising 311 early-stage NSCLC patients receiving surgical treatment at Massachusetts General Hospital (MGH), with a focus on the two most common histological types: adenocarcinoma (ADC) and Squamous Cell Carcinoma (SCC). The CNNs were able to predict tumor histology with an AUC of 0.71(p = 0.018). We also found that using machine learning classifiers such as k-nearest neighbors (kNN) and support vector machine (SVM) on CNN-derived quantitative radiomics features yielded comparable discriminative performance, with AUC of up to 0.71 (p = 0.017). Our best performing CNN functioned as a robust probabilistic classifier in heterogeneous test sets, with qualitatively interpretable visual explanations to its predictions. Deep learning based radiomics can identify histological phenotypes in lung cancer. It has the potential to augment existing approaches and serve as a corrective aid for diagnosticians.

Radiomics in immuno-oncology

With the ongoing advances in imaging techniques, increasing volumes of anatomical and functional data are being generated as part of the routine clinical workflow. This surge of available imaging data coincides with increasing research in quantitative imaging, particularly in the domain of imaging features. An important and novel approach is radiomics, where high-dimensional image properties are extracted from routine medical images. The fundamental principle of radiomics is the hypothesis that biomedical images contain predictive information, not discernible to the human eye, that can be mined through quantitative image analysis. In this review, a general outline of radiomics and artificial intelligence (AI) will be provided, along with prominent use cases in immunotherapy (e.g. response and adverse event prediction) and targeted therapy (i.e. radiogenomics). While the increased use and development of radiomics and AI in immuno-oncology is highly promising, the technology is still in its early stages, and different challenges still need to be overcome. Nevertheless, novel AI algorithms are being constructed with an ever-increasing scope of applications.

Computed tomography-based deep-learning prediction of neoadjuvant chemoradiotherapy treatment response in esophageal squamous cell carcinoma

BACKGROUND

Deep learning is promising to predict treatment response. We aimed to evaluate and validate the predictive performance of the CT-based model using deep learning features for predicting pathologic complete response to neoadjuvant chemoradiotherapy (nCRT) in esophageal squamous cell carcinoma (ESCC).

MATERIALS AND METHODS

Patients were retrospectively enrolled between April 2007 and December 2018 from two institutions. We extracted deep learning features of six pre-trained convolutional neural networks, respectively, from pretreatment CT images in the training cohort (n = 161). Support vector machine was adopted as the classifier. Validation was performed in an external testing cohort (n = 70). We assessed the performance using the area under the receiver operating characteristics curve (AUC) and selected an optimal model, which was compared with a radiomics model developed from the training cohort. A clinical model consisting of clinical factors only was also built for baseline comparison. We further conducted a radiogenomics analysis using gene expression profiles to reveal underlying biology associated with radiological prediction.

RESULTS

The optimal model with features extracted from ResNet50 achieved an AUC and accuracy of 0.805 (95% CI, 0.696-0.913) and 77.1% (65.6%-86.3%) in the testing cohort, compared with 0.725 (0.605-0.846)) and 67.1% (54.9%-77.9%) for the radiomics model. All the radiological models showed better predictive performance than the clinical model. Radiogenomics analysis suggested a potential association mainly with WNT signaling pathway and tumor microenvironment.

CONCLUSIONS

The novel and noninvasive deep learning approach could provide efficient and accurate prediction of treatment response to nCRT in ESCC, and benefit clinical decision making of therapeutic strategy.

Integrative radiogenomics for virtual biopsy and treatment monitoring in ovarian cancer

BACKGROUND

Ovarian cancer survival rates have not changed in the last 20 years. The majority of cases are High-grade serous ovarian carcinomas (HGSOCs), which are typically diagnosed at an advanced stage with multiple metastatic lesions. Taking biopsies of all sites of disease is infeasible, which challenges the implementation of stratification tools based on molecular profiling.

MAIN BODY

In this review, we describe how these challenges might be overcome by integrating quantitative features extracted from medical imaging with the analysis of paired genomic profiles, a combined approach called radiogenomics, to generate virtual biopsies. Radiomic studies have been used to model different imaging phenotypes, and some radiomic signatures have been associated with paired molecular profiles to monitor spatiotemporal changes in the heterogeneity of tumours. We describe different strategies to integrate radiogenomic information in a global and local manner, the latter by targeted sampling of tumour habitats, defined as regions with distinct radiomic phenotypes.

CONCLUSION

Linking radiomics and biological correlates in a targeted manner could potentially improve the clinical management of ovarian cancer. Radiogenomic signatures could be used to monitor tumours during the course of therapy, offering additional information for clinical decision making. In summary, radiogenomics may pave the way to virtual biopsies and treatment monitoring tools for integrative tumour analysis.

[Formula: see text]: deep learning-based radiomics for the time-to-event outcome prediction in lung cancer

Hand-crafted radiomics has been used for developing models in order to predict time-to-event clinical outcomes in patients with lung cancer. Hand-crafted features, however, are pre-defined and extracted without taking the desired target into account. Furthermore, accurate segmentation of the tumor is required for development of a reliable predictive model, which may be objective and a time-consuming task. To address these drawbacks, we propose a deep learning-based radiomics model for the time-to-event outcome prediction, referred to as DRTOP that takes raw images as inputs, and calculates the image-based risk of death or recurrence, for each patient. Our experiments on an in-house dataset of 132 lung cancer patients show that the obtained image-based risks are significant predictors of the time-to-event outcomes. Computed Tomography (CT)-based features are predictors of the overall survival (OS), with the hazard ratio (HR) of 1.35, distant control (DC), with HR of 1.06, and local control (LC), with HR of 2.66. The Positron Emission Tomography (PET)-based features are predictors of OS and recurrence free survival (RFS), with hazard ratios of 1.67 and 1.18, respectively. The concordance indices of [Formula: see text], [Formula: see text], and [Formula: see text] for predicting the OS, DC, and RFS show that the deep learning-based radiomics model is as accurate or better in predicting predefined clinical outcomes compared to hand-crafted radiomics, with concordance indices of [Formula: see text], [Formula: see text], and [Formula: see text], for predicting the OS, DC, and RFS, respectively. Deep learning-based radiomics has the potential to offer complimentary predictive information in the personalized management of lung cancer patients.

Radiomics: A primer for the radiation oncologist

PURPOSE

Radiomics are a set of methods used to leverage medical imaging and extract quantitative features that can characterize a patient's phenotype. All modalities can be used with several different software packages. Specific informatics methods can then be used to create meaningful predictive models. In this review, we will explain the major steps of a radiomics analysis pipeline and then present the studies published in the context of radiation therapy.

METHODS

A literature review was performed on Medline using the search engine PubMed. The search strategy included the search terms "radiotherapy", "radiation oncology" and "radiomics". The search was conducted in July 2019 and reference lists of selected articles were hand searched for relevance to this review.

RESULTS

A typical radiomics workflow always includes five steps: imaging and segmenting, data curation and preparation, feature extraction, exploration and selection and finally modeling. In radiation oncology, radiomics studies have been published to explore different clinical outcome in lung (n=5), head and neck (n=5), esophageal (n=3), rectal (n=3), pancreatic (n=2) cancer and brain metastases (n=2). The quality of these retrospective studies is heterogeneous and their results have not been translated to the clinic.

CONCLUSION

Radiomics has a great potential to predict clinical outcome and better personalize treatment. But the field is still young and constantly evolving. Improvement in bias reduction techniques and multicenter studies will hopefully allow more robust and generalizable models.

[A primer on radiomics]

CLINICAL ISSUE

The reproducible and exhaustive extraction of information from radiological images is a central task in the practice of radiology. Dynamic developments in the fields of artificial intelligence (AI) and machine learning are introducing new methods for this task. Radiomics is one such method and offers new opportunities and challenges for the future of radiology.

METHODOLOGICAL INNOVATIONS

Radiomics describes the quantitative evaluation, interpretation, and clinical assessment of imaging markers in radiological data. Components of a radiomics analysis are data acquisition, data preprocessing, data management, segmentation of regions of interest, computation and selection of imaging markers, as well as the development of a radiomics model used for diagnosis and prognosis. This article explains these components and aims at providing an introduction to the field of radiomics while highlighting existing limitations.

MATERIALS AND METHODS

This article is based on a selective literature search with the PubMed search engine.

ASSESSMENT

Even though radiomics applications have yet to arrive in routine clinical practice, the quantification of radiological data in terms of radiomics is underway and will increase in the future. This holds the potential for lasting change in the discipline of radiology. Through the successful extraction and interpretation of all the information encoded in radiological images the next step in the direction of a more personalized, future-oriented form of medicine can be taken.

Deep Learning: A Review for the Radiation Oncologist

Introduction: Deep Learning (DL) is a machine learning technique that uses deep neural networks to create a model. The application areas of deep learning in radiation oncology include image segmentation and detection, image phenotyping, and radiomic signature discovery, clinical outcome prediction, image dose quantification, dose-response modeling, radiation adaptation, and image generation. In this review, we explain the methods used in DL and perform a literature review using the Medline database to identify studies using deep learning in radiation oncology. The search was conducted in April 2018, and identified studies published between 1997 and 2018, strongly skewed toward 2015 and later. Methods: A literature review was performed using PubMed/Medline in order to identify important recent publications to be synthesized into a review of the current status of Deep Learning in radiation oncology, directed at a clinically-oriented reader. The search strategy included the search terms "radiotherapy" and "deep learning." In addition, reference lists of selected articles were hand searched for further potential hits of relevance to this review. The search was conducted in April 2018, and identified studies published between 1997 and 2018, strongly skewed toward 2015 and later. Results: Studies using DL for image segmentation were identified in Brain (n = 2), Head and Neck (n = 3), Lung (n = 6), Abdominal (n = 2), and Pelvic (n = 6) cancers. Use of Deep Learning has also been reported for outcome prediction, such as toxicity modeling (n = 3), treatment response and survival (n = 2), or treatment planning (n = 5). Conclusion: Over the past few years, there has been a significant number of studies assessing the performance of DL techniques in radiation oncology. They demonstrate how DL-based systems can aid clinicians in their daily work, be it by reducing the time required for or the variability in segmentation, or by helping to predict treatment outcomes and toxicities. It still remains to be seen when these techniques will be employed in routine clinical practice.