Radiological tumor classification across imaging modality and histology

Nomogram for preoperative differentiation of benign and malignant breast tumors using contrast-enhanced cone-beam breast CT (CE CB-BCT) quantitative imaging and assessment features

PURPOSE

Breast cancer's impact necessitates refined diagnostic approaches. This study develops a nomogram using radiology quantitative features from contrast-enhanced cone-beam breast CT for accurate preoperative classification of benign and malignant breast tumors.

MATERIAL AND METHODS

A retrospective study enrolled 234 females with breast tumors, split into training and test sets. Contrast-enhanced cone-beam breast CT-images were acquired using Koning Breast CT-1000. Quantitative assessment features were extracted via 3D-slicer software, identifying independent predictors. The nomogram was constructed to preoperative differentiation benign and malignant breast tumors. Calibration curve was used to assess whether the model showed favorable correspondence with pathological confirmation. Decision curve analysis confirmed the model's superiority.

RESULTS

The study enrolled 234 female patients with a mean age of 50.2 years (SD ± 9.2). The training set had 164 patients (89 benign, 75 malignant), and the test set had 70 patients (29 benign, 41 malignant). The nomogram achieved excellent predictive performance in distinguishing benign and malignant breast lesions with an AUC of 0.940 (95% CI 0.900-0.940) in the training set and 0.970 (95% CI 0.940-0.970) in the test set.

CONCLUSION

This study illustrates the effectiveness of quantitative radiology features derived from contrast-enhanced cone-beam breast CT in distinguishing between benign and malignant breast tumors. Incorporating these features into a nomogram-based diagnostic model allows for breast tumor diagnoses that are objective and possess good accuracy. The application of these insights could substantially increase reliability and efficacy in the management of breast tumors, offering enhanced diagnostic capability.

Enhancing NSCLC recurrence prediction with PET/CT habitat imaging, ctDNA, and integrative radiogenomics-blood insights

While we recognize the prognostic importance of clinicopathological measures and circulating tumor DNA (ctDNA), the independent contribution of quantitative image markers to prognosis in non-small cell lung cancer (NSCLC) remains underexplored. In our multi-institutional study of 394 NSCLC patients, we utilize pre-treatment computed tomography (CT) and 18F-fluorodeoxyglucose positron emission tomography (FDG-PET) to establish a habitat imaging framework for assessing regional heterogeneity within individual tumors. This framework identifies three PET/CT subtypes, which maintain prognostic value after adjusting for clinicopathologic risk factors including tumor volume. Additionally, these subtypes complement ctDNA in predicting disease recurrence. Radiogenomics analysis unveil the molecular underpinnings of these imaging subtypes, highlighting downregulation in interferon alpha and gamma pathways in the high-risk subtype. In summary, our study demonstrates that these habitat imaging subtypes effectively stratify NSCLC patients based on their risk levels for disease recurrence after initial curative surgery or radiotherapy, providing valuable insights for personalized treatment approaches.

Artificial intelligence in neuro-oncology: advances and challenges in brain tumor diagnosis, prognosis, and precision treatment

This review delves into the most recent advancements in applying artificial intelligence (AI) within neuro-oncology, specifically emphasizing work on gliomas, a class of brain tumors that represent a significant global health issue. AI has brought transformative innovations to brain tumor management, utilizing imaging, histopathological, and genomic tools for efficient detection, categorization, outcome prediction, and treatment planning. Assessing its influence across all facets of malignant brain tumor management- diagnosis, prognosis, and therapy- AI models outperform human evaluations in terms of accuracy and specificity. Their ability to discern molecular aspects from imaging may reduce reliance on invasive diagnostics and may accelerate the time to molecular diagnoses. The review covers AI techniques, from classical machine learning to deep learning, highlighting current applications and challenges. Promising directions for future research include multimodal data integration, generative AI, large medical language models, precise tumor delineation and characterization, and addressing racial and gender disparities. Adaptive personalized treatment strategies are also emphasized for optimizing clinical outcomes. Ethical, legal, and social implications are discussed, advocating for transparency and fairness in AI integration for neuro-oncology and providing a holistic understanding of its transformative impact on patient care.

Synthetic PET from CT improves diagnosis and prognosis for lung cancer: Proof of concept

[18F]Fluorodeoxyglucose positron emission tomography (FDG-PET) and computed tomography (CT) are indispensable components in modern medicine. Although PET can provide additional diagnostic value, it is costly and not universally accessible, particularly in low-income countries. To bridge this gap, we have developed a conditional generative adversarial network pipeline that can produce FDG-PET from diagnostic CT scans based on multi-center multi-modal lung cancer datasets (n = 1,478). Synthetic PET images are validated across imaging, biological, and clinical aspects. Radiologists confirm comparable imaging quality and tumor contrast between synthetic and actual PET scans. Radiogenomics analysis further proves that the dysregulated cancer hallmark pathways of synthetic PET are consistent with actual PET. We also demonstrate the clinical values of synthetic PET in improving lung cancer diagnosis, staging, risk prediction, and prognosis. Taken together, this proof-of-concept study testifies to the feasibility of applying deep learning to obtain high-fidelity PET translated from CT.

Circulating tumor DNA and radiological tumor volume identify patients at risk for relapse with resected, early-stage non-small-cell lung cancer

BACKGROUND

Predicting relapse and overall survival (OS) in early-stage non-small-cell lung cancer (NSCLC) patients remains challenging. Therefore, we hypothesized that detection of circulating tumor DNA (ctDNA) can identify patients with increased risk of relapse and that integrating radiological tumor volume measurement along with ctDNA detectability improves prediction of outcome.

PATIENTS AND METHODS

We analyzed 366 serial plasma samples from 85 patients who underwent surgical resections and assessed ctDNA using a next-generation sequencing liquid biopsy assay, and measured tumor volume using a computed tomography-based three-dimensional annotation.

RESULTS

Our results showed that patients with detectable ctDNA at baseline or after treatment and patients who did not clear ctDNA after treatment had a significantly worse clinical outcome. Integrating radiological analysis allowed the stratification in risk groups prognostic of clinical outcome as confirmed in an independent cohort of 32 patients.

CONCLUSIONS

Our findings suggest ctDNA and radiological monitoring could be valuable tools for guiding follow-up care and treatment decisions for early-stage NSCLC patients.

Predictive value of triple negative breast cancer based on DCE-MRI multi-phase full-volume ROI clinical radiomics model

BACKGROUND

Since no studies compared the value of radiomics features of distinct phases of dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) for predicting triple-negative breast cancer (TNBC).

PURPOSE

To identify the optimal phase of DCE-MRI for diagnosing TNBC and, in combination with clinical factors, to develop a clinical-radiomics model to well predict TNBC.

MATERIAL AND METHODS

This retrospective study included 158 patients with pathology-confirmed breast cancer, including 38 cases of TNBC. The patients were randomly divided into the training and validation set (7:3). Eight radiomics models were built based on eight DCE-MR phases, and their performances were evaluated using receiver operating characteristic curve (ROC) and DeLong's test. The Radscore derived from the best radiomics model was integrated with independent clinical risk factors to construct a clinical-radiomics predictive model, and evaluate its performance using ROC analysis, calibration, and decision curve analyses.

RESULTS

WHO classification, margin, and T2-weighted (T2W) imaging signals were significantly correlated with TNBC and independent risk factors for TNBC (P<0.05). The clinical model yielded areas under the curve (AUCs) of 0.867 and 0.843 in the training and validation sets, respectively. The radiomics model based on DCEphase7 achieved the highest efficacy, with an AUC of 0.818 and 0.777. The AUC of the clinical-radiomics model was 0.936 and 0.886 in the training and validation sets, respectively. The decision curve showed the clinical utility of the clinical-radiomics model.

CONCLUSION

The radiomics features of DCE-MRI had the potential to predict TNBC and could improve the performance of clinical risk factors for preoperative personalized prediction of TNBC.

CT-based deep learning radiomics and hematological biomarkers in the assessment of pathological complete response to neoadjuvant chemoradiotherapy in patients with esophageal squamous cell carcinoma: A two-center study

PURPOSE

To evaluate and validate CT-based models using pre- and posttreatment deep learning radiomics features and hematological biomarkers for assessing esophageal squamous cell carcinoma (ESCC) pathological complete response (pCR) after neoadjuvant chemoradiotherapy (nCRT).

MATERIAL AND METHODS

This retrospective study recruited patients with biopsy-proven ESCC who underwent nCRT from two Chinese hospitals between May 2017 and May 2022, divided into a training set (hospital I, 111 cases), an internal validation set (hospital I, 47 cases), and an external validation set (hospital II, 33 cases). We used minimum redundancy maximum relevance (mRMR) and least absolute shrinkage and selection operator (LASSO) as feature selection methods and three classifiers as model construction methods. The assessment of models was performed using area under the receiver operating characteristic (ROC) curve (AUC) and decision curve analysis (DCA).

RESULTS

A total 190 patients were included in our study (60.8 ± 7.08 years, 133 men), and seventy-seven of them (40.5 %) achieved pCR. The logistic regression (LR)-based combined model incorporating neutrophil to lymphocyte ratio, lymphocyte to monocyte ratio, albumin, and radscores performed well both in the internal and external validation sets with AUCs of 0.875 and 0.857 (95 % CI, 0.776-0.964; 0.731-0.984, P <0.05), respectively. DCA demonstrated that nomogram was useful for pCR prediction and produced clinical net benefits.

CONCLUSION

The incorporation of radscores and hematological biomarkers into LR-based model improved pCR prediction after nCRT in ESCC. Enhanced pCR predictability may improve patients selection before surgery, providing clinical application value for the use of active surveillance.

Radiogenomic-based multiomic analysis reveals imaging intratumor heterogeneity phenotypes and therapeutic targets

Intratumor heterogeneity (ITH) profoundly affects therapeutic responses and clinical outcomes. However, the widespread methods for assessing ITH based on genomic sequencing or pathological slides, which rely on limited tissue samples, may lead to inaccuracies due to potential sampling biases. Using a newly established multicenter breast cancer radio-multiomic dataset (n = 1474) encompassing radiomic features extracted from dynamic contrast-enhanced magnetic resonance images, we formulated a noninvasive radiomics methodology to effectively investigate ITH. Imaging ITH (IITH) was associated with genomic and pathological ITH, predicting poor prognosis independently in breast cancer. Through multiomic analysis, we identified activated oncogenic pathways and metabolic dysregulation in high-IITH tumors. Integrated metabolomic and transcriptomic analyses highlighted ferroptosis as a vulnerability and potential therapeutic target of high-IITH tumors. Collectively, this work emphasizes the superiority of radiomics in capturing ITH. Furthermore, we provide insights into the biological basis of IITH and propose therapeutic targets for breast cancers with elevated IITH.

Survival Prediction via Hierarchical Multimodal Co-Attention Transformer: A Computational Histology-Radiology Solution

The rapid advances in deep learning-based computational pathology and radiology have demonstrated the promise of using whole slide images (WSIs) and radiology images for survival prediction in cancer patients. However, most image-based survival prediction methods are limited to using either histology or radiology alone, leaving integrated approaches across histology and radiology relatively underdeveloped. There are two main challenges in integrating WSIs and radiology images: (1) the gigapixel nature of WSIs and (2) the vast difference in spatial scales between WSIs and radiology images. To address these challenges, in this work, we propose an interpretable, weakly-supervised, multimodal learning framework, called Hierarchical Multimodal Co-Attention Transformer (HMCAT), to integrate WSIs and radiology images for survival prediction. Our approach first uses hierarchical feature extractors to capture various information including cellular features, cellular organization, and tissue phenotypes in WSIs. Then the hierarchical radiology-guided co- attention (HRCA) in HMCAT characterizes the multimodal interactions between hierarchical histology-based visual concepts and radiology features and learns hierarchical co- attention mappings for two modalities. Finally, HMCAT combines their complementary information into a multimodal risk score and discovers prognostic features from two modalities by multimodal interpretability. We apply our approach to two cancer datasets (365 WSIs with matched magnetic resonance [MR] images and 213 WSIs with matched computed tomography [CT] images). Our results demonstrate that the proposed HMCAT consistently achieves superior performance over the unimodal approaches trained on either histology or radiology data alone, as well as other state-of-the-art methods.

Biology-guided deep learning predicts prognosis and cancer immunotherapy response

Substantial progress has been made in using deep learning for cancer detection and diagnosis in medical images. Yet, there is limited success on prediction of treatment response and outcomes, which has important implications for personalized treatment strategies. A significant hurdle for clinical translation of current data-driven deep learning models is lack of interpretability, often attributable to a disconnect from the underlying pathobiology. Here, we present a biology-guided deep learning approach that enables simultaneous prediction of the tumor immune and stromal microenvironment status as well as treatment outcomes from medical images. We validate the model for predicting prognosis of gastric cancer and the benefit from adjuvant chemotherapy in a multi-center international study. Further, the model predicts response to immune checkpoint inhibitors and complements clinically approved biomarkers. Importantly, our model identifies a subset of mismatch repair-deficient tumors that are non-responsive to immunotherapy and may inform the selection of patients for combination treatments.

SwarmDeepSurv: swarm intelligence advances deep survival network for prognostic radiomics signatures in four solid cancers

Survival models exist to study relationships between biomarkers and treatment effects. Deep learning-powered survival models supersede the classical Cox proportional hazards (CoxPH) model, but substantial performance drops were observed on high-dimensional features because of irrelevant/redundant information. To fill this gap, we proposed SwarmDeepSurv by integrating swarm intelligence algorithms with the deep survival model. Furthermore, four objective functions were designed to optimize prognostic prediction while regularizing selected feature numbers. When testing on multicenter sets (n = 1,058) of four different cancer types, SwarmDeepSurv was less prone to overfitting and achieved optimal patient risk stratification compared with popular survival modeling algorithms. Strikingly, SwarmDeepSurv selected different features compared with classical feature selection algorithms, including the least absolute shrinkage and selection operator (LASSO), with nearly no feature overlapping across these models. Taken together, SwarmDeepSurv offers an alternative approach to model relationships between radiomics features and survival endpoints, which can further extend to study other input data types including genomics.

Predicting benefit from immune checkpoint inhibitors in patients with non-small-cell lung cancer by CT-based ensemble deep learning: a retrospective study

BACKGROUND

Only around 20-30% of patients with non-small-cell lung cancer (NCSLC) have durable benefit from immune-checkpoint inhibitors. Although tissue-based biomarkers (eg, PD-L1) are limited by suboptimal performance, tissue availability, and tumour heterogeneity, radiographic images might holistically capture the underlying cancer biology. We aimed to investigate the application of deep learning on chest CT scans to derive an imaging signature of response to immune checkpoint inhibitors and evaluate its added value in the clinical context.

METHODS

In this retrospective modelling study, 976 patients with metastatic, EGFR/ALK negative NSCLC treated with immune checkpoint inhibitors at MD Anderson and Stanford were enrolled from Jan 1, 2014, to Feb 29, 2020. We built and tested an ensemble deep learning model on pretreatment CTs (Deep-CT) to predict overall survival and progression-free survival after treatment with immune checkpoint inhibitors. We also evaluated the added predictive value of the Deep-CT model in the context of existing clinicopathological and radiological metrics.

FINDINGS

Our Deep-CT model demonstrated robust stratification of patient survival of the MD Anderson testing set, which was validated in the external Stanford set. The performance of the Deep-CT model remained significant on subgroup analyses stratified by PD-L1, histology, age, sex, and race. In univariate analysis, Deep-CT outperformed the conventional risk factors, including histology, smoking status, and PD-L1 expression, and remained an independent predictor after multivariate adjustment. Integrating the Deep-CT model with conventional risk factors demonstrated significantly improved prediction performance, with overall survival C-index increases from 0·70 (clinical model) to 0·75 (composite model) during testing. On the other hand, the deep learning risk scores correlated with some radiomics features, but radiomics alone could not reach the performance level of deep learning, indicating that the deep learning model effectively captured additional imaging patterns beyond known radiomics features.

INTERPRETATION

This proof-of-concept study shows that automated profiling of radiographic scans through deep learning can provide orthogonal information independent of existing clinicopathological biomarkers, bringing the goal of precision immunotherapy for patients with NSCLC closer.

FUNDING

National Institutes of Health, Mark Foundation Damon Runyon Foundation Physician Scientist Award, MD Anderson Strategic Initiative Development Program, MD Anderson Lung Moon Shot Program, Andrea Mugnaini, and Edward L C Smith.

Radiomics for characterization of the glioma immune microenvironment

Increasing evidence suggests that besides mutational and molecular alterations, the immune component of the tumor microenvironment also substantially impacts tumor behavior and complicates treatment response, particularly to immunotherapies. Although the standard method for characterizing tumor immune profile is through performing integrated genomic analysis on tissue biopsies, the dynamic change in the immune composition of the tumor microenvironment makes this approach not feasible, especially for brain tumors. Radiomics is a rapidly growing field that uses advanced imaging techniques and computational algorithms to extract numerous quantitative features from medical images. Recent advances in machine learning methods are facilitating biological validation of radiomic signatures and allowing them to "mine" for a variety of significant correlates, including genetic, immunologic, and histologic data. Radiomics has the potential to be used as a non-invasive approach to predict the presence and density of immune cells within the microenvironment, as well as to assess the expression of immune-related genes and pathways. This information can be essential for patient stratification, informing treatment decisions and predicting patients' response to immunotherapies. This is particularly important for tumors with difficult surgical access such as gliomas. In this review, we provide an overview of the glioma microenvironment, describe novel approaches for clustering patients based on their tumor immune profile, and discuss the latest progress on utilization of radiomics for immune profiling of glioma based on current literature.

Bibliometric research on the developments of artificial intelligence in radiomics toward nervous system diseases

Background

The growing interest suggests that the widespread application of radiomics has facilitated the development of neurological disease diagnosis, prognosis, and classification. The application of artificial intelligence methods in radiomics has increasingly achieved outstanding prediction results in recent years. However, there are few studies that have systematically analyzed this field through bibliometrics. Our destination is to study the visual relationships of publications to identify the trends and hotspots in radiomics research and encourage more researchers to participate in radiomics studies.

Methods

Publications in radiomics in the field of neurological disease research can be retrieved from the Web of Science Core Collection. Analysis of relevant countries, institutions, journals, authors, keywords, and references is conducted using Microsoft Excel 2019, VOSviewer, and CiteSpace V. We analyze the research status and hot trends through burst detection.

Results

On October 23, 2022, 746 records of studies on the application of radiomics in the diagnosis of neurological disorders were retrieved and published from 2011 to 2023. Approximately half of them were written by scholars in the United States, and most were published in Frontiers in Oncology, European Radiology, Cancer, and SCIENTIFIC REPORTS. Although China ranks first in the number of publications, the United States is the driving force in the field and enjoys a good academic reputation. NORBERT GALLDIKS and JIE TIAN published the most relevant articles, while GILLIES RJ was cited the most. RADIOLOGY is a representative and influential journal in the field. "Glioma" is a current attractive research hotspot. Keywords such as "machine learning," "brain metastasis," and "gene mutations" have recently appeared at the research frontier.

Conclusion

Most of the studies focus on clinical trial outcomes, such as the diagnosis, prediction, and prognosis of neurological disorders. The radiomics biomarkers and multi-omics studies of neurological disorders may soon become a hot topic and should be closely monitored, particularly the relationship between tumor-related non-invasive imaging biomarkers and the intrinsic micro-environment of tumors.

Machine Learning Models for the Identification of Prognostic and Predictive Cancer Biomarkers: A Systematic Review

The identification of biomarkers plays a crucial role in personalized medicine, both in the clinical and research settings. However, the contrast between predictive and prognostic biomarkers can be challenging due to the overlap between the two. A prognostic biomarker predicts the future outcome of cancer, regardless of treatment, and a predictive biomarker predicts the effectiveness of a therapeutic intervention. Misclassifying a prognostic biomarker as predictive (or vice versa) can have serious financial and personal consequences for patients. To address this issue, various statistical and machine learning approaches have been developed. The aim of this study is to present an in-depth analysis of recent advancements, trends, challenges, and future prospects in biomarker identification. A systematic search was conducted using PubMed to identify relevant studies published between 2017 and 2023. The selected studies were analyzed to better understand the concept of biomarker identification, evaluate machine learning methods, assess the level of research activity, and highlight the application of these methods in cancer research and treatment. Furthermore, existing obstacles and concerns are discussed to identify prospective research areas. We believe that this review will serve as a valuable resource for researchers, providing insights into the methods and approaches used in biomarker discovery and identifying future research opportunities.

Imaging phenotypes from MRI for the prediction of glioma immune subtypes from RNA sequencing: A multicenter study

Tumor subtyping based on its immune landscape may guide precision immunotherapy. The aims of this study were to identify immune subtypes of adult diffuse gliomas with RNA sequencing data, and to noninvasively predict this subtype using a biologically interpretable radiomic signature from MRI. A subtype discovery dataset (n = 210) from a public database and two radiogenomic datasets (n = 130 and 55, respectively) from two local hospitals were included. Brain tumor microenvironment-specific signatures were constructed from RNA sequencing to identify the immune types. A radiomic signature was built from MRI to predict the identified immune subtypes. The pathways underlying the radiomic signature were identified to annotate their biological meanings. The reproducibility of the findings was verified externally in multicenter datasets. Three distinctive immune subtypes were identified, including an inflamed subtype marked by elevated hypoxia-induced immunosuppression, a "cold" subtype that exhibited scarce immune infiltration with downregulated antigen presentation, and an intermediate subtype that showed medium immune infiltration. A 10-feature radiomic signature was developed to predict immune subtypes, achieving an AUC of 0.924 in the validation dataset. The radiomic features correlated with biological functions underpinning immune suppression, which substantiated the hypothesis that molecular changes can be reflected by radiomic features. The immune subtypes, predictive radiomic signature, and radiomics-correlated biological pathways were validated externally. Our data suggest that adult-type diffuse gliomas harbor three distinctive immune subtypes that can be predicted by MRI radiomic features with clear biological significance. The immune subtypes, radiomic signature, and radiogenomic links can be replicated externally.

Towards precision medicine: Omics approach for COVID-19

The coronavirus disease 2019 (COVID-19) pandemic had a devastating impact on human society. Beginning with genome surveillance of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the development of omics technologies brought a clearer understanding of the complex SARS-CoV-2 and COVID-19. Here, we reviewed how omics, including genomics, proteomics, single-cell multi-omics, and clinical phenomics, play roles in answering biological and clinical questions about COVID-19. Large-scale sequencing and advanced analysis methods facilitate COVID-19 discovery from virus evolution and severity risk prediction to potential treatment identification. Omics would indicate precise and globalized prevention and medicine for the COVID-19 pandemic under the utilization of big data capability and phenotypes refinement. Furthermore, decoding the evolution rule of SARS-CoV-2 by deep learning models is promising to forecast new variants and achieve more precise data to predict future pandemics and prevent them on time.

Synthesis and Bioevaluation of Novel Technetium-99m-Labeled Complexes with Norfloxacin HYNIC Derivatives for Bacterial Infection Imaging

To seek a novel ^99mTc-labeled quinolone derivative for bacterial infection SPECT imaging that aims to lower nontarget organ uptake, a novel norfloxacin 6-hydrazinoicotinamide (HYNIC) derivative (HYNICNF) was designed and synthesized. It was radiolabeled with different coligands, such as tricine, trisodium triphenylphosphine-3,3',3″-trisulfonate (TPPTS), sodium triphenylphosphine-3-monosulfonate (TPPMS), and ethylenediamine-N,N'-diacetic acid (EDDA), to obtain three ^99mTc-labeled norfloxacin HYNIC complexes, namely, [^99mTc]Tc-tricine-TPPTS-HYNICNF, [^99mTc]Tc-tricine-TPPMS-HYNICNF, and [^99mTc]Tc-EDDA-HYNICNF. These complexes were purified (RCP > 95%) and evaluated in vitro and in vivo for targeting bacteria. All three complexes are hydrophilic, maintain good stability, and specifically bind Staphylococcus aureus in vitro. The biodistribution in mice with bacterial infection demonstrated that [^99mTc]Tc-EDDA-HYNICNF showed a higher abscess uptake and lower nontarget organ uptake and was able to distinguish bacterial infection and sterile inflammation. Single photon emission computed tomography (SPECT) image study in bacterial infection mice showed there was a visible accumulation in the infection site, suggesting that [^99mTc]Tc-EDDA-HYNICNF is a potential radiotracer for bacterial infection imaging.

Multimodal integration of radiology, pathology and genomics for prediction of response to PD-(L)1 blockade in patients with non-small cell lung cancer

Immunotherapy is used to treat almost all patients with advanced non-small cell lung cancer (NSCLC); however, identifying robust predictive biomarkers remains challenging. Here we show the predictive capacity of integrating medical imaging, histopathologic and genomic features to predict immunotherapy response using a cohort of 247 patients with advanced NSCLC with multimodal baseline data obtained during diagnostic clinical workup, including computed tomography scan images, digitized programmed death ligand-1 immunohistochemistry slides and known outcomes to immunotherapy. Using domain expert annotations, we developed a computational workflow to extract patient-level features and used a machine-learning approach to integrate multimodal features into a risk prediction model. Our multimodal model (area under the curve (AUC) = 0.80, 95% confidence interval (CI) 0.74-0.86) outperformed unimodal measures, including tumor mutational burden (AUC = 0.61, 95% CI 0.52-0.70) and programmed death ligand-1 immunohistochemistry score (AUC = 0.73, 95% CI 0.65-0.81). Our study therefore provides a quantitative rationale for using multimodal features to improve prediction of immunotherapy response in patients with NSCLC using expert-guided machine learning.

Integrative Histology-Genomic Analysis Predicts Hepatocellular Carcinoma Prognosis Using Deep Learning

Cancer prognosis analysis is of essential interest in clinical practice. In order to explore the prognostic power of computational histopathology and genomics, this paper constructs a multi-modality prognostic model for survival prediction. We collected 346 patients diagnosed with hepatocellular carcinoma (HCC) from The Cancer Genome Atlas (TCGA), each patient has 1-3 whole slide images (WSIs) and an mRNA expression file. WSIs were processed by a multi-instance deep learning model to obtain the patient-level survival risk scores; mRNA expression data were processed by weighted gene co-expression network analysis (WGCNA), and the top hub genes of each module were extracted as risk factors. Information from two modalities was integrated by Cox proportional hazard model to predict patient outcomes. The overall survival predictions of the multi-modality model (Concordance index (C-index): 0.746, 95% confidence interval (CI): ±0.077) outperformed these based on histopathology risk score or hub genes, respectively. Furthermore, in the prediction of 1-year and 3-year survival, the area under curve of the model achieved 0.816 and 0.810. In conclusion, this paper provides an effective workflow for multi-modality prognosis of HCC, the integration of histopathology and genomic information has the potential to assist clinical prognosis management.

Artificial intelligence in oncologic imaging

Radiology is integral to cancer care. Compared to molecular assays, imaging has its advantages. Imaging as a noninvasive tool can assess the entirety of tumor unbiased by sampling error and is routinely acquired at multiple time points in oncological practice. Imaging data can be digitally post-processed for quantitative assessment. The ever-increasing application of Artificial intelligence (AI) to clinical imaging is challenging radiology to become a discipline with competence in data science, which plays an important role in modern oncology. Beyond streamlining certain clinical tasks, the power of AI lies in its ability to reveal previously undetected or even imperceptible radiographic patterns that may be difficult to ascertain by the human sensory system. Here, we provide a narrative review of the emerging AI applications relevant to the oncological imaging spectrum and elaborate on emerging paradigms and opportunities. We envision that these technical advances will change radiology in the coming years, leading to the optimization of imaging acquisition and discovery of clinically relevant biomarkers for cancer diagnosis, staging, and treatment monitoring. Together, they pave the road for future clinical translation in precision oncology.

Integrated multimodal artificial intelligence framework for healthcare applications

Artificial intelligence (AI) systems hold great promise to improve healthcare over the next decades. Specifically, AI systems leveraging multiple data sources and input modalities are poised to become a viable method to deliver more accurate results and deployable pipelines across a wide range of applications. In this work, we propose and evaluate a unified Holistic AI in Medicine (HAIM) framework to facilitate the generation and testing of AI systems that leverage multimodal inputs. Our approach uses generalizable data pre-processing and machine learning modeling stages that can be readily adapted for research and deployment in healthcare environments. We evaluate our HAIM framework by training and characterizing 14,324 independent models based on HAIM-MIMIC-MM, a multimodal clinical database (N = 34,537 samples) containing 7279 unique hospitalizations and 6485 patients, spanning all possible input combinations of 4 data modalities (i.e., tabular, time-series, text, and images), 11 unique data sources and 12 predictive tasks. We show that this framework can consistently and robustly produce models that outperform similar single-source approaches across various healthcare demonstrations (by 6–33%), including 10 distinct chest pathology diagnoses, along with length-of-stay and 48 h mortality predictions. We also quantify the contribution of each modality and data source using Shapley values, which demonstrates the heterogeneity in data modality importance and the necessity of multimodal inputs across different healthcare-relevant tasks. The generalizable properties and flexibility of our Holistic AI in Medicine (HAIM) framework could offer a promising pathway for future multimodal predictive systems in clinical and operational healthcare settings.

Radiogenomic analysis reveals tumor heterogeneity of triple-negative breast cancer

Triple-negative breast cancer (TNBC) is a subset of breast cancer with an adverse prognosis and significant tumor heterogeneity. Here, we extract quantitative radiomic features from contrast-enhanced magnetic resonance images to construct a breast cancer radiomic dataset (n = 860) and a TNBC radiogenomic dataset (n = 202). We develop and validate radiomic signatures that can fairly differentiate TNBC from other breast cancer subtypes and distinguish molecular subtypes within TNBC. A radiomic feature that captures peritumoral heterogeneity is determined to be a prognostic factor for recurrence-free survival (p = 0.01) and overall survival (p = 0.004) in TNBC. Combined with the established matching TNBC transcriptomic and metabolomic data, we demonstrate that peritumoral heterogeneity is associated with immune suppression and upregulated fatty acid synthesis in tumor samples. Collectively, this multi-omic dataset serves as a useful public resource to promote precise subtyping of TNBC and helps to understand the biological significance of radiomics.

•

A radiomic signature identifies TNBC from other subtypes of breast cancer

•

Radiomic features are predictive of TNBC molecular subtypes

•

A radiomic feature reflecting peritumoral heterogeneity indicates TNBC prognosis

•

Peritumoral heterogeneity correlates with metabolic and immune abnormalities in TNBC

Automated detection and segmentation of non-small cell lung cancer computed tomography images

Detection and segmentation of abnormalities on medical images is highly important for patient management including diagnosis, radiotherapy, response evaluation, as well as for quantitative image research. We present a fully automated pipeline for the detection and volumetric segmentation of non-small cell lung cancer (NSCLC) developed and validated on 1328 thoracic CT scans from 8 institutions. Along with quantitative performance detailed by image slice thickness, tumor size, image interpretation difficulty, and tumor location, we report an in-silico prospective clinical trial, where we show that the proposed method is faster and more reproducible compared to the experts. Moreover, we demonstrate that on average, radiologists & radiation oncologists preferred automatic segmentations in 56% of the cases. Additionally, we evaluate the prognostic power of the automatic contours by applying RECIST criteria and measuring the tumor volumes. Segmentations by our method stratified patients into low and high survival groups with higher significance compared to those methods based on manual contours.

Correct interpretation of computer tomography (CT) scans is important for the correct assessment of a patient’s disease but can be subjective and timely. Here, the authors develop a system that can automatically segment the non-small cell lung cancer on CT images of patients and show in an in silico trial that the method was faster and more reproducible than clinicians.

Artificial Intelligence in Digital Pathology to Advance Cancer Immunotherapy

Immune-checkpoint inhibitors (ICIs) have revolutionized the treatment of many malignancies. For instance, in lung cancer, however, only 20~30% of patients can achieve durable clinical benefits from ICI monotherapy. Histopathologic and molecular features such as histological type, PD-L1 expression, and tumor mutation burden (TMB), play a paramount role in selecting appropriate regimens for cancer treatment in the era of immunotherapy. Unfortunately, none of the existing features are exclusive predictive biomarkers. Thus, there is an imperative need to pinpoint more effective biomarkers to identify patients who may achieve the most benefit from ICIs. The adoption of digital pathology in clinical flow, as being powered by artificial intelligence (AI) especially deep learning, has catalyzed the automated analysis of tissue slides. With the breakthrough of multiplex bioimaging technology, researchers can comprehensively characterize the tumor microenvironment, including the different immune cells' distribution, function, and interaction. Here, we briefly summarize recent AI studies in digital pathology and share our perspective on emerging paradigms and directions to advance the development of immunotherapy biomarkers.

Chronic Lymphocytic Leukemia Progression Diagnosis with Intrinsic Cellular Patterns via Unsupervised Clustering

Simple Summary

Distinguishing between chronic lymphocytic leukemia (CLL), accelerated CLL (aCLL), and full-blown transformation to diffuse large B-cell lymphoma (Richter transformation; RT) has significant clinical implications. Identifying cellular phenotypes via unsupervised clustering provides the most robust analytic performance in analyzing digitized pathology slides. This study serves as a proof of concept that using an unsupervised machine learning scheme can enhance diagnostic accuracy.

Abstract

Identifying the progression of chronic lymphocytic leukemia (CLL) to accelerated CLL (aCLL) or transformation to diffuse large B-cell lymphoma (Richter transformation; RT) has significant clinical implications as it prompts a major change in patient management. However, the differentiation between these disease phases may be challenging in routine practice. Unsupervised learning has gained increased attention because of its substantial potential in data intrinsic pattern discovery. Here, we demonstrate that cellular feature engineering, identifying cellular phenotypes via unsupervised clustering, provides the most robust analytic performance in analyzing digitized pathology slides (accuracy = 0.925, AUC = 0.978) when compared to alternative approaches, such as mixed features, supervised features, unsupervised/mixed/supervised feature fusion and selection, as well as patch-based convolutional neural network (CNN) feature extraction. We further validate the reproducibility and robustness of unsupervised feature extraction via stability and repeated splitting analysis, supporting its utility as a diagnostic aid in identifying CLL patients with histologic evidence of disease progression. The outcome of this study serves as proof of principle using an unsupervised machine learning scheme to enhance the diagnostic accuracy of the heterogeneous histology patterns that pathologists might not easily see.

Generative models improve radiomics performance in different tasks and different datasets: An experimental study

PURPOSE

Radiomics is an active area of research focusing on high throughput feature extraction from medical images with a wide array of applications in clinical practice, such as clinical decision support in oncology. However, noise in low dose computed tomography (CT) scans can impair the accurate extraction of radiomic features. In this article, we investigate the possibility of using deep learning generative models to improve the performance of radiomics from low dose CTs.

METHODS

We used two datasets of low dose CT scans - NSCLC Radiogenomics and LIDC-IDRI - as test datasets for two tasks - pre-treatment survival prediction and lung cancer diagnosis. We used encoder-decoder networks and conditional generative adversarial networks (CGANs) trained in a previous study as generative models to transform low dose CT images into full dose CT images. Radiomic features extracted from the original and improved CT scans were used to build two classifiers - a support vector machine (SVM) and a deep attention based multiple instance learning model - for survival prediction and lung cancer diagnosis respectively. Finally, we compared the performance of the models derived from the original and improved CT scans.

RESULTS

Denoising with the encoder-decoder network and the CGAN improved the area under the curve (AUC) of survival prediction from 0.52 to 0.57 (p-value < 0.01). On the other hand, the encoder-decoder network and the CGAN improved the AUC of lung cancer diagnosis from 0.84 to 0.88 and 0.89 respectively (p-value < 0.01). Finally, there are no statistically significant improvements in AUC using encoder-decoder networks and CGAN (p-value = 0.34) when networks trained at 75 and 100 epochs.

CONCLUSION

Generative models can improve the performance of low dose CT-based radiomics in different tasks. Hence, denoising using generative models seems to be a necessary pre-processing step for calculating radiomic features from low dose CTs.

Lesion-Based Radiomics Signature in Pretherapy 18F-FDG PET Predicts Treatment Response to Ibrutinib in Lymphoma

PURPOSE

The aim of this study was to develop a pretherapy PET/CT-based prediction model for treatment response to ibrutinib in lymphoma patients.

PATIENTS AND METHODS

One hundred sixty-nine lymphoma patients with 2441 lesions were studied retrospectively. All eligible lymphomas on pretherapy 18F-FDG PET images were contoured and segmented for radiomic analysis. Lesion- and patient-based responsiveness to ibrutinib was determined retrospectively using the Lugano classification. PET radiomic features were extracted. A radiomic model was built to predict ibrutinib response. The prognostic significance of the radiomic model was evaluated independently in a test cohort and compared with conventional PET metrics: SUVmax, metabolic tumor volume, and total lesion glycolysis.

RESULTS

The radiomic model had an area under the receiver operating characteristic curve (ROC AUC) of 0.860 (sensitivity, 92.9%, specificity, 81.4%; P < 0.001) for predicting response to ibrutinib, outperforming the SUVmax (ROC AUC, 0.519; P = 0.823), metabolic tumor volume (ROC AUC, 0.579; P = 0.412), total lesion glycolysis (ROC AUC, 0.576; P = 0.199), and a composite model built using all 3 (ROC AUC, 0.562; P = 0.046). The radiomic model increased the probability of accurately predicting ibrutinib-responsive lesions from 84.8% (pretest) to 96.5% (posttest). At the patient level, the model's performance (ROC AUC = 0.811; P = 0.007) was superior to that of conventional PET metrics. Furthermore, the radiomic model showed robustness when validated in treatment subgroups: first (ROC AUC, 0.916; P < 0.001) versus second or greater (ROC AUC, 0.842; P < 0.001) line of defense and single treatment (ROC AUC, 0.931; P < 0.001) versus multiple treatments (ROC AUC, 0.824; P < 0.001).

CONCLUSIONS

We developed and validated a pretherapy PET-based radiomic model to predict response to treatment with ibrutinib in a diverse cohort of lymphoma patients.

Computer-Assisted Image Processing System for Early Assessment of Lung Nodule Malignancy

Simple Summary

Lung cancer is the second most common cancer in men after prostate cancer and in women after breast cancer, but it is the leading cause of cancer death among both genders. This manuscript proposes a new computer-aided diagnosis system that uses only a single computed tomography scan to diagnose the pulmonary nodule as benign or malignant. This system helps in the early detection of the pulmonary nodules and shows its ability to identify the pulmonary nodules precisely.

Abstract

Lung cancer is one of the most dreadful cancers, and its detection in the early stage is very important and challenging. This manuscript proposes a new computer-aided diagnosis system for lung cancer diagnosis from chest computed tomography scans. The proposed system extracts two different kinds of features, namely, appearance features and shape features. For the appearance features, a Histogram of oriented gradients, a Multi-view analytical Local Binary Pattern, and a Markov Gibbs Random Field are developed to give a good description of the lung nodule texture, which is one of the main distinguishing characteristics between benign and malignant nodules. For the shape features, Multi-view Peripheral Sum Curvature Scale Space, Spherical Harmonics Expansion, and a group of some fundamental morphological features are implemented to describe the outer contour complexity of the nodules, which is main factor in lung nodule diagnosis. Each feature is fed into a stacked auto-encoder followed by a soft-max classifier to generate the initial malignancy probability. Finally, all these probabilities are combined together and fed to the last network to give the final diagnosis. The system is validated using 727 nodules which are subset from the Lung Image Database Consortium (LIDC) dataset. The system shows very high performance measures and achieves 92.55%, 91.70%, and 93.40% for the accuracy, sensitivity, and specificity, respectively. This high performance shows the ability of the system to distinguish between the malignant and benign nodules precisely.

Deep Learning With Radiomics for Disease Diagnosis and Treatment: Challenges and Potential

The high-throughput extraction of quantitative imaging features from medical images for the purpose of radiomic analysis, i.e., radiomics in a broad sense, is a rapidly developing and emerging research field that has been attracting increasing interest, particularly in multimodality and multi-omics studies. In this context, the quantitative analysis of multidimensional data plays an essential role in assessing the spatio-temporal characteristics of different tissues and organs and their microenvironment. Herein, recent developments in this method, including manually defined features, data acquisition and preprocessing, lesion segmentation, feature extraction, feature selection and dimension reduction, statistical analysis, and model construction, are reviewed. In addition, deep learning-based techniques for automatic segmentation and radiomic analysis are being analyzed to address limitations such as rigorous workflow, manual/semi-automatic lesion annotation, and inadequate feature criteria, and multicenter validation. Furthermore, a summary of the current state-of-the-art applications of this technology in disease diagnosis, treatment response, and prognosis prediction from the perspective of radiology images, multimodality images, histopathology images, and three-dimensional dose distribution data, particularly in oncology, is presented. The potential and value of radiomics in diagnostic and therapeutic strategies are also further analyzed, and for the first time, the advances and challenges associated with dosiomics in radiotherapy are summarized, highlighting the latest progress in radiomics. Finally, a robust framework for radiomic analysis is presented and challenges and recommendations for future development are discussed, including but not limited to the factors that affect model stability (medical big data and multitype data and expert knowledge in medical), limitations of data-driven processes (reproducibility and interpretability of studies, different treatment alternatives for various institutions, and prospective researches and clinical trials), and thoughts on future directions (the capability to achieve clinical applications and open platform for radiomics analysis).

Radiomics, deep learning and early diagnosis in oncology

Medical imaging, including X-ray, computed tomography (CT), and magnetic resonance imaging (MRI), plays a critical role in early detection, diagnosis, and treatment response prediction of cancer. To ease radiologists' task and help with challenging cases, computer-aided diagnosis has been developing rapidly in the past decade, pioneered by radiomics early on, and more recently, driven by deep learning. In this mini-review, I use breast cancer as an example and review how medical imaging and its quantitative modeling, including radiomics and deep learning, have improved the early detection and treatment response prediction of breast cancer. I also outline what radiomics and deep learning share in common and how they differ in terms of modeling procedure, sample size requirement, and computational implementation. Finally, I discuss the challenges and efforts entailed to integrate deep learning models and software in clinical practice.

Lack of Benefit of Extending Temozolomide Treatment in Patients with High Vascular Glioblastoma with Methylated MGMT

In this study, we evaluated the benefit on survival of the combination of methylation of O6-methylguanine-DNA methyltransferase (MGMT) promotor gene and moderate vascularity in glioblastoma using a retrospective dataset of 123 patients from a multicenter cohort. MRI processing and calculation of relative cerebral blood volume (rCBV), used to define moderate- and high-vascular groups, were performed with the automatic ONCOhabitats method. We assessed the previously proposed rCBV threshold (10.7) and the new calculated ones (9.1 and 9.8) to analyze the association with survival for different populations according to vascularity and MGMT methylation status. We found that patients included in the moderate-vascular group had longer survival when MGMT is methylated (significant median survival difference of 174 days, p = 0.0129*). However, we did not find significant differences depending on the MGMT methylation status for the high-vascular group (p = 0.9119). In addition, we investigated the combined correlation of MGMT methylation status and rCBV with the prognostic effect of the number of temozolomide cycles, and only significant results were found for the moderate-vascular group. In conclusion, there is a lack of benefit of extending temozolomide treatment for patients with high vascular glioblastomas, even presenting MGMT methylation. Preliminary results suggest that patients with moderate vascularity and methylated MGMT glioblastomas would benefit more from prolonged adjuvant chemotherapy.

At the Cutting Edge against Cancer: A Perspective on Immunoproteasome and Immune Checkpoints Modulation as a Potential Therapeutic Intervention

Immunoproteasome is a noncanonical form of proteasome with enzymological properties optimized for the generation of antigenic peptides presented in complex with class I MHC molecules. This enzymatic property makes the modulation of its activity a promising area of research. Nevertheless, immunotherapy has emerged as a front-line treatment of advanced/metastatic tumors providing outstanding improvement of life expectancy, even though not all patients achieve a long-lasting clinical benefit. To enhance the efficacy of the currently available immunotherapies and enable the development of new strategies, a broader knowledge of the dynamics of antigen repertoire processing by cancer cells is needed. Therefore, a better understanding of the role of immunoproteasome in antigen processing and of the therapeutic implication of its modulation is mandatory. Studies on the potential crosstalk between proteasome modulators and immune checkpoint inhibitors could provide novel perspectives and an unexplored treatment option for a variety of cancers.