Artificial intelligence for multimodal data integration in oncology

Cross-modal alignment and contrastive learning for enhanced cancer survival prediction

BACKGROUND AND OBJECTIVE

Integrating multimodal data, such as pathology images and genomics, is crucial for understanding cancer heterogeneity, personalized treatment complexity, and enhancing survival prediction. However, most current prognostic methods are limited to a single domain of histopathology or genomics, inevitably reducing their potential for accurate patient outcome prediction. Despite advancements in the concurrent analysis of pathology and genomic data, existing approaches inadequately address the intricate intermodal relationships.

METHODS

This paper introduces the CPathomic method for multimodal data-based survival prediction. By leveraging whole slide pathology images to guide local pathological features, the method effectively mitigates significant intermodal differences through a cross-modal representational contrastive learning module. Furthermore, it facilitates interactive learning between different modalities through cross-modal and gated attention modules.

RESULTS

The extensive experiments on five public TCGA datasets demonstrate that CPathomic framework effectively bridges modality gaps, consistently outperforming alternative multimodal survival prediction methods.

CONCLUSION

The model we propose, CPathomic, unveils the potential of contrastive learning and cross-modal attention in the representation and fusion of multimodal data, enhancing the performance of patient survival prediction.

Multimodal radiopathological integration for prognosis and prediction of adjuvant chemotherapy benefit in resectable lung adenocarcinoma: A multicentre study

Lung adenocarcinoma (LUAD) has a heterogeneous prognosis and controversial postoperative treatment protocols. We aim to develop and validate a multimodal analysis framework that integrates CT images with H&E-stained whole-slide images (WSIs) to enhance risk stratification and predict adjuvant chemotherapy benefit in LUAD patients. We retrospectively collected data from 1039 resectable LUAD patients (stage I-III) across four centres, forming a training dataset (n = 303), two testing datasets (n = 197 and n = 228) for survival analysis, and a feature testing dataset (n = 311) for interpretability analysis. We extracted 487 tumour/peritumour radiomics features from CT images and 783 multiscale pathomics features from WSIs, characterising the shape of tumour (CT) and cancer nuclei (WSIs), as well as the intensity and texture of tumour/peritumour regions (CT) and tumour regions/epithelium/stroma (WSIs). A survival support vector machine (SVM) was employed to establish a radiopathomics signature using the optimal set of multimodal features, including 2 tumour radiomics features, 3 peritumour radiomics features, and 4 nuclei heterogeneity pathomics features. The radiopathomics signature outperformed both radiomics and pathomics signatures in predicting disease-free survival (DFS) (C-index: training dataset, 0.744 vs. 0.734 and 0.692; testing dataset 1, 0.719 vs. 0.701 and 0.638; testing dataset 2, 0.711 vs. 0.689 and 0.684), demonstrating greater robustness compared to the state-of-the-art deep learning integration approaches. It provided additional prognostic information beyond clinical risk factors (C-index of clinical plus radiopathomics vs. clinical models: training dataset, 0.763 vs. 0.676; testing dataset 1, 0.739 vs. 0.676; testing dataset 2, 0.711 vs. 0.699, p < 0.001). Compared to low-risk patients categorised by the radiopathomics signature, high-risk patients achieved comparable DFS when receiving adjuvant chemotherapy (training dataset, HR = 1.53, 95 % CI 0.85-2.73, p = 0.153; testing dataset 1 and 2, HR = 1.62, 95 % CI 0.92-2.85, p = 0.096), but had significantly worse DFS when only observed after surgery (training dataset, HR = 4.46, 95 % CI 2.82-7.05, p < 0.001; testing datasets 1 and 2, HR = 3.52, 95 % CI 2.26-5.49, p < 0.001), indicating the predictive value of the radiopathomics signature for adjuvant chemotherapy benefit (interaction p < 0.05). Further interpretability analysis revealed that the radiopathomics signature was associated with various prognostic/treatment-related biomarkers, including differentiation, immune phenotypes, and EGFR status. The multimodal integration framework offered a cost-effective approach for LUAD characterisation by leveraging complementary information from radiological and histopathological imaging. The radiopathomics signature demonstrated robust prognostic capabilities, providing valuable insights for postoperative treatment decisions.

Angiogenic and immune predictors of neoadjuvant axitinib response in renal cell carcinoma with venous tumour thrombus

Venous tumour thrombus (VTT), where the primary tumour invades the renal vein and inferior vena cava, affects 10-15% of renal cell carcinoma (RCC) patients. Curative surgery for VTT is high-risk, but neoadjuvant therapy may improve outcomes. The NAXIVA trial demonstrated a 35% VTT response rate after 8 weeks of neoadjuvant axitinib, a VEGFR-directed therapy. However, understanding non-response is critical for better treatment. Here we show that response to axitinib in this setting is characterised by a distinct and predictable set of features. We conduct a multiparametric investigation of samples collected during NAXIVA using digital pathology, flow cytometry, plasma cytokine profiling and RNA sequencing. Responders have higher baseline microvessel density and increased induction of VEGF-A and PlGF during treatment. A multi-modal machine learning model integrating features predict response with an AUC of 0.868, improving to 0.945 when using features from week 3. Key predictive features include plasma CCL17 and IL-12. These findings may guide future treatment strategies for VTT, improving the clinical management of this challenging scenario.

DNA methylation in breast cancer: early detection and biomarker discovery through current and emerging approaches

Breast cancer remains one of the most common cancers in women worldwide. Early detection is critical for improving patient outcomes, yet current screening methods have limitations. Therefore, there is a pressing need for more sensitive and specific approaches to detect breast cancer in its earliest stages. Liquid biopsy has emerged as a promising non-invasive method for early cancer detection and management. DNA methylation, an epigenetic alteration that often precedes genetic changes, has been observed in precancerous or early cancer stages, making it a valuable biomarker. This review explores the role of DNA methylation in breast cancer and its potential for developing blood-based tests. We discuss advancements in DNA methylation detection methods, recent discoveries of potential DNA methylation biomarkers from both single-omics and multi-omics integration studies, and the role of machine learning in enhancing diagnostic accuracy. Challenges and future directions are also addressed. Although challenges remain, advances in multi-omics integration and machine learning continue to enhance the clinical potential of methylation-based biomarkers. Ongoing research is crucial to further refine these approaches and improve early detection and patient outcomes.

Emerging artificial intelligence-driven precision therapies in tumor drug resistance: recent advances, opportunities, and challenges

Drug resistance is one of the main reasons for cancer treatment failure, leading to a rapid recurrence/disease progression of the cancer. Recently, artificial intelligence (AI) has empowered physicians to use its powerful data processing and pattern recognition capabilities to extract and mine valuable drug resistance information from large amounts of clinical or omics data, to study drug resistance mechanisms, to evaluate and predict drug resistance, and to develop innovative therapeutic strategies to reduce drug resistance. In this review, we proposed a feasible workflow for incorporating AI into tumor drug resistance research, highlighted current AI-driven tumor drug resistance applications, and discussed the opportunities and challenges encountered in the process. Based on a comprehensive literature analysis, we systematically summarized the role of AI in tumor drug resistance research, including drug development, resistance mechanism elucidation, drug sensitivity prediction, combination therapy optimization, resistance phenotype identification, and clinical biomarker discovery. With the continuous advancement of AI technology and rigorous validation of clinical data, AI models are expected to fuel the development of precision oncology by improving efficacy, guiding therapeutic decisions, and optimizing patient prognosis. In summary, by leveraging clinical and omics data, AI models are expected to pioneer new therapy strategies to mitigate tumor drug resistance, improve efficacy and patient survival, and provide novel perspectives and tools for oncology treatment.

Hybrid intelligence for environmental pollution: biodegradability assessment of organic compounds through multimodal integration of graph attention networks and QSAR models

Computational methods are crucial for assessing chemical biodegradability, given their significant impact on both environmental and human health. Organic compounds that are not biodegradable can persist in the environment, contributing to pollution. Our novel approach leverages graph attention networks (GATs) and incorporates node and edge attributes for biodegradability prediction. Quantitative Structure-Activity Relationship (QSAR) models using two-dimensional descriptors alongside weighted average and stacking approaches were employed to generate ensemble models. The GAT models demonstrated a stable function and generally higher specificity on the validation set compared to a graph convolutional network, although definitive superiority is challenging to establish owing to overlapping standard deviations. However, the sensitivities tended to decrease with potential performance overlap owing to the interval intersection. Ensemble learning enhanced several performance metrics compared with individual models and base models, with the combination of extreme Gradient Boosting and GAT achieving the highest precision and specificity. Combining GAT with random forest and Gradient Boosting may be preferable for accurately predicting biodegradable molecules, whereas the stacking approach may be suitable for prioritizing the correct classification of nonbiodegradable substances. Important descriptors, such as SpMax1_Bh(m) and SAscore, were identified in at least two QSAR models. Despite inherent complexities, the ease of implementation depends on factors such as data availability, and domain knowledge. Assessing the biodegradability of organic compounds is essential for reducing their environmental impact, assessing risks, ensuring regulatory compliance, promoting sustainable development, and supporting effective pollution remediation. It assists in making informed decisions about chemical use, waste management, and environmental protection.

Radiomics and machine learning for predicting metachronous liver metastasis in rectal cancer

A recent study by Long et al used a predictive model to explore the efficacy of radiomics based on multiparametric magnetic resonance imaging in predicting metachronous liver metastasis (MLM) in newly diagnosed rectal cancer (RC) patients. The machine learning algorithms, particularly the random forest model (RFM), appeared well-matched to the complex nature of radiomics data. The predictive capabilities of the RFM, as evidenced by the area under the curve of 0.919 in the training cohort and 0.901 in the validation cohort, highlighted its potential clinical utility. However, we highlighted several methodological limitations, including excluding genomic markers, potential biases from the retrospective design, limited generalizability due to a single-center study, and variability in image interpretation. We propose further investigation into integrating multi-omic data, conducting larger multicenter studies, and utilizing advanced imaging techniques. Additionally, we highlighted the importance of interdisciplinary collaboration to improve predictive model development and advocate for cost-effectiveness analyses to facilitate clinical integration. Overall, this predictive model may improve the early detection and management of MLM in RC patients, with promising avenues for future exploration. Ongoing research in this domain can potentially improve clinical outcomes and the quality of care for RC patients.

Artificial Intelligence Advancements in Oncology: A Review of Current Trends and Future Directions

Cancer remains one of the leading causes of mortality worldwide, driving the need for innovative approaches in research and treatment. Artificial intelligence (AI) has emerged as a powerful tool in oncology, with the potential to revolutionize cancer diagnosis, treatment, and management. This paper reviews recent advancements in AI applications within cancer research, focusing on early detection through computer-aided diagnosis, personalized treatment strategies, and drug discovery. We survey AI-enhanced diagnostic applications and explore AI techniques such as deep learning, as well as the integration of AI with nanomedicine and immunotherapy for cancer care. Comparative analyses of AI-based models versus traditional diagnostic methods are presented, highlighting AI's superior potential. Additionally, we discuss the importance of integrating social determinants of health to optimize cancer care. Despite these advancements, challenges such as data quality, algorithmic biases, and clinical validation remain, limiting widespread adoption. The review concludes with a discussion of the future directions of AI in oncology, emphasizing its potential to reshape cancer care by enhancing diagnosis, personalizing treatments and targeted therapies, and ultimately improving patient outcomes.

Multimodal fusion of radio-pathology and proteogenomics identify integrated glioma subtypes with prognostic and therapeutic opportunities

Integrating multimodal data can uncover causal features hidden in single-modality analyses, offering a comprehensive understanding of disease complexity. This study introduces a multimodal fusion subtyping (MOFS) framework that integrates radiological, pathological, genomic, transcriptomic, and proteomic data from 122 patients with IDH-wildtype adult glioma, identifying three subtypes: MOFS1 (proneural) with favorable prognosis, elevated neurodevelopmental activity, and abundant neurocyte infiltration; MOFS2 (proliferative) with the worst prognosis, superior proliferative activity, and genome instability; MOFS3 (TME-rich) with intermediate prognosis, abundant immune and stromal components, and sensitive to anti-PD-1 immunotherapy. STRAP emerges as a prognostic biomarker and potential therapeutic target for MOFS2, associated with its proliferative phenotype. Stromal infiltration in MOFS3 serves as a crucial prognostic indicator, allowing for further prognostic stratification. Additionally, we develop a deep neural network (DNN) classifier based on radiological features to further enhance the clinical translatability, providing a non-invasive tool for predicting MOFS subtypes. Overall, these findings highlight the potential of multimodal fusion in improving the classification, prognostic accuracy, and precision therapy of IDH-wildtype glioma, offering an avenue for personalized management.

A critical assessment of artificial intelligence in magnetic resonance imaging of cancer

Given the enormous output and pace of development of artificial intelligence (AI) methods in medical imaging, it can be challenging to identify the true success stories to determine the state-of-the-art of the field. This report seeks to provide the magnetic resonance imaging (MRI) community with an initial guide into the major areas in which the methods of AI are contributing to MRI in oncology. After a general introduction to artificial intelligence, we proceed to discuss the successes and current limitations of AI in MRI when used for image acquisition, reconstruction, registration, and segmentation, as well as its utility for assisting in diagnostic and prognostic settings. Within each section, we attempt to present a balanced summary by first presenting common techniques, state of readiness, current clinical needs, and barriers to practical deployment in the clinical setting. We conclude by presenting areas in which new advances must be realized to address questions regarding generalizability, quality assurance and control, and uncertainty quantification when applying MRI to cancer to maintain patient safety and practical utility.

Deep learning in the precise assessment of primary Sjögren's syndrome based on ultrasound images

OBJECTIVES

This study aimed to investigate the value of a deep learning (DL) model based on greyscale ultrasound (US) images for precise assessment and accurate diagnosis of primary Sjögren's syndrome (pSS).

METHODS

This was a multicentre prospective analysis. All pSS patients were diagnosed according to 2016 ACR/EULAR criteria. A total of 72 pSS patients and 72 sex- and age-matched healthy controls recruited between January 2022 and April 2023, together with 41 patients and 41 healthy controls recruited from June 2023 to February 2024 were used for DL model development and validation, respectively. The DL model was constructed based on the ResNet 50 input with preprocessed all participants' bilateral submandibular glands (SMGs), parotid glands (PGs), and lacrimal glands (LGs) greyscale US images. Diagnostic performance of the model was compared with two radiologists. The accuracy of prediction and identification performance of DL model were evaluated by calibration curve.

RESULTS

A total of 864 and 164 greyscale US images of SMGs, PGs, and LGs were collected for development and validation of the model. The area under the ROC (AUCs) of DL model in the SMGs, PGs, and LGs were 0.92, 0.93, 0.91 in the model cohort, and were 0.90, 0.88, 0.87 in the validation cohort, respectively, outperforming both radiologists. Calibration curves showed the prediction probability of the DL model was consistent with the actual probability in both model cohort and validation cohort.

CONCLUSION

The DL model based on greyscale US images showed diagnostic potential in the precise assessment of pSS patients in the SMGs, PGs and LGs, outperforming conventional radiologist evaluation.

Deep learning informed multimodal fusion of radiology and pathology to predict outcomes in HPV-associated oropharyngeal squamous cell carcinoma

BACKGROUND

We aim to predict outcomes of human papillomavirus (HPV)-associated oropharyngeal squamous cell carcinoma (OPSCC), a subtype of head and neck cancer characterized with improved clinical outcome and better response to therapy. Pathology and radiology focused AI-based prognostic models have been independently developed for OPSCC, but their integration incorporating both primary tumour (PT) and metastatic cervical lymph node (LN) remains unexamined.

METHODS

We investigate the prognostic value of an AI approach termed the swintransformer-based multimodal and multi-region data fusion framework (SMuRF). SMuRF integrates features from CT corresponding to the PT and LN, as well as whole slide pathology images from the PT as a predictor of survival and tumour grade in HPV-associated OPSCC. SMuRF employs cross-modality and cross-region window based multi-head self-attention mechanisms to capture interactions between features across tumour habitats and image scales.

FINDINGS

Developed and tested on a cohort of 277 patients with OPSCC with matched radiology and pathology images, SMuRF demonstrated strong performance (C-index = 0.81 for DFS prediction and AUC = 0.75 for tumour grade classification) and emerged as an independent prognostic biomarker for DFS (hazard ratio [HR] = 17, 95% confidence interval [CI], 4.9-58, p < 0.0001) and tumour grade (odds ratio [OR] = 3.7, 95% CI, 1.4-10.5, p = 0.01) controlling for other clinical variables (i.e., T-, N-stage, age, smoking, sex and treatment modalities). Importantly, SMuRF outperformed unimodal models derived from radiology or pathology alone.

INTERPRETATION

Our findings underscore the potential of multimodal deep learning in accurately stratifying OPSCC risk, informing tailored treatment strategies and potentially refining existing treatment algorithms.

FUNDING

The National Institutes of Health, the U.S. Department of Veterans Affairs and National Institute of Biomedical Imaging and Bioengineering.

Multimodal data integration in early-stage breast cancer

The use of biomarkers in breast cancer has significantly improved patient outcomes through targeted therapies, such as hormone therapy anti-Her2 therapy and CDK4/6 or PARP inhibitors. However, existing knowledge does not fully encompass the diverse nature of breast cancer, particularly in triple-negative tumors. The integration of multi-omics and multimodal data has the potential to provide new insights into biological processes, to improve breast cancer patient stratification, enhance prognosis and response prediction, and identify new biomarkers. This review presents a comprehensive overview of the state-of-the-art multimodal (including molecular and image) data integration algorithms developed and with applicability to breast cancer stratification, prognosis, or biomarker identification. We examined the primary challenges and opportunities of these multimodal data integration algorithms, including their advantages, limitations, and critical considerations for future research. We aimed to describe models that are not only academically and preclinically relevant, but also applicable to clinical settings.

Future of Artificial Intelligence-Machine Learning Trends in Pathology and Medicine

Artificial intelligence (AI) and machine learning (ML) are transforming the field of medicine. Health care organizations are now starting to establish management strategies for integrating such platforms (AI-ML toolsets) that leverage the computational power of advanced algorithms to analyze data and to provide better insights that ultimately translate to enhanced clinical decision-making and improved patient outcomes. Emerging AI-ML platforms and trends in pathology and medicine are reshaping the field by offering innovative solutions to enhance diagnostic accuracy, operational workflows, clinical decision support, and clinical outcomes. These tools are also increasingly valuable in pathology research in which they contribute to automated image analysis, biomarker discovery, drug development, clinical trials, and productive analytics. Other related trends include the adoption of ML operations for managing models in clinical settings, the application of multimodal and multiagent AI to utilize diverse data sources, expedited translational research, and virtualized education for training and simulation. As the final chapter of our AI educational series, this review article delves into the current adoption, future directions, and transformative potential of AI-ML platforms in pathology and medicine, discussing their applications, benefits, challenges, and future perspectives.

Self-interactive learning: Fusion and evolution of multi-scale histomorphology features for molecular traits prediction in computational pathology

Predicting disease-related molecular traits from histomorphology brings great opportunities for precision medicine. Despite the rich information present in histopathological images, extracting fine-grained molecular features from standard whole slide images (WSI) is non-trivial. The task is further complicated by the lack of annotations for subtyping and contextual histomorphological features that might span multiple scales. This work proposes a novel multiple-instance learning (MIL) framework capable of WSI-based cancer morpho-molecular subtyping by fusion of different-scale features. Our method, debuting as Inter-MIL, follows a weakly-supervised scheme. It enables the training of the patch-level encoder for WSI in a task-aware optimisation procedure, a step normally not modelled in most existing MIL-based WSI analysis frameworks. We demonstrate that optimising the patch-level encoder is crucial to achieving high-quality fine-grained and tissue-level subtyping results and offers a significant improvement over task-agnostic encoders. Our approach deploys a pseudo-label propagation strategy to update the patch encoder iteratively, allowing discriminative subtype features to be learned. This mechanism also empowers extracting fine-grained attention within image tiles (the small patches), a task largely ignored in most existing weakly supervised-based frameworks. With Inter-MIL, we carried out four challenging cancer molecular subtyping tasks in the context of ovarian, colorectal, lung, and breast cancer. Extensive evaluation results show that Inter-MIL is a robust framework for cancer morpho-molecular subtyping with superior performance compared to several recently proposed methods, in small dataset scenarios where the number of available training slides is less than 100. The iterative optimisation mechanism of Inter-MIL significantly improves the quality of the image features learned by the patch embedded and generally directs the attention map to areas that better align with experts' interpretation, leading to the identification of more reliable histopathology biomarkers. Moreover, an external validation cohort is used to verify the robustness of Inter-MIL on molecular trait prediction.

A comprehensive study on the radiomic score derived from perineural invasion in gastric cancer and its correlation with the overall survival of patients

BACKGROUND

Perineural invasion (PNI) is closely related to the prognosis of gastric cancer (GC) patients. However, a noninvasive tool for accurately and reliably predicting the PNI is lacking.

METHODS

The clinical and imaging data of 278 patients from institution I and 39 patients from institution II were retrospectively analyzed. Radiomic features were extracted from the intratumoral and peritumoral regions. Seven independent machine learning (ML) algorithms are used to develop the models. Kaplan-Meier survival analysis and Cox proportional hazards analysis were carried out to compare 3-year and 5-year overall survival (OS) differences among various subgroups based on PNI and radiomic scores.

RESULTS

T stage and lymphovascular invasion (LVI) were significantly correlated with the PNI (P < 0.01). The OS of patients with different PNI status was significantly different (P < 0.05). Gradient boosting tree is the best ML algorithm. The area-under-the-curve (AUC) values of the optimal radiomics model in the internal test set and external test set were 0.901 and 0.886, respectively. After the introduction of clinical variables T stage and LVI, the performance of the model further improved in predicting the PNI of GC patients, with the AUC of 0.904 in the internal test set and 0.886 in the external test set. The difference in 3-year OS (P = 0.005) and 5-year OS (P = 0.015) among patients with varying radiomic scores was statistically significant.

CONCLUSION

Radiomics combined with intratumoral and peritumoral features is feasible for evaluating the PNI of GC patients. The prognosis of patients with different radiomic scores was statistically significant.

Deep learning-based prediction of cervical canal stenosis from mid-sagittal T2-weighted MRI

OBJECTIVE

This study aims to establish a large degenerative cervical myelopathy cohort and develop deep learning models for predicting cervical canal stenosis from sagittal T2-weighted MRI.

MATERIALS AND METHODS

Data was collected retrospectively from patients who underwent a cervical spine MRI from January 2007 to December 2022 at a single institution. Ground truth labels for cervical canal stenosis were obtained from sagittal T2-weighted MRI using Kang's grade, a four-level scoring system that classifies stenosis with the degree of subarachnoid space obliteration and cord indentation. ResNet50, VGG16, MobileNetV3, and EfficientNetV2 were trained using threefold cross-validation, and the models exhibiting the largest area under the receiver operating characteristic curve (AUC) were selected to produce the ensemble model. Gradient-weighted class activation mapping was adopted for qualitative assessment. Models that incorporate demographic features were trained, and their corresponding AUCs on the test set were evaluated.

RESULTS

Of 8676 patients, 7645 were eligible for developing deep learning models, where 6880 (mean age, 56.0 ± 14.3 years, 3480 men) were used for training while 765 (mean age, 56.5 ± 14.4 years, 386 men) were set aside for testing. The ensemble model exhibited the largest AUC of 0.95 (0.94-0.97). Accuracy was 0.875 (0.851-0.898), sensitivity was 0.885 (0.855-0.915), and specificity was 0.861 (0.824-0.898). Qualitative analyses demonstrated that the models accurately pinpoint radiologic findings suggestive of cervical canal stenosis and myelopathy. Incorporation of demographic features did not result in a gain of AUC.

CONCLUSION

We have developed deep learning models from a large degenerative cervical myelopathy cohort and thoroughly explored their robustness and explainability.

Unravelling key signaling pathways for the therapeutic targeting of non-small cell lung cancer

Lung cancer (LC) remains the foremost cause of cancer-related mortality across the globe. Non-small cell lung cancer (NSCLC) is a type of LC that exhibits significant heterogeneity at histological and molecular levels. Genetic alterations in upstream signaling molecules activate cascades affecting apoptosis, proliferation, and differentiation. Disruption of these signaling pathways leads to the proliferation of cancer-promoting cells, progression of cancer, and resistance to its treatment. Recent insights into the function of signaling pathways and their fundamental mechanisms in the onset of various diseases could pave the way for new therapeutic approaches. Recently, numerous drug molecules have been created that target these cell signaling pathways and could be used alongside other standard therapies to achieve synergistic effects in mitigating the pathophysiology of NSCLC. Additionally, many researchers have identified several predictive biomarkers, and alterations in transcription factors and related pathways are employed to create new therapeutic strategies for NSCLC. Findings suggest using specific inhibitors to target cellular signaling pathways in tumor progression to treat NSCLC. This review investigates the role of signaling pathways in NSCLC development and explores novel therapeutic strategies to enhance clinical treatment options for NSCLC.

AI-driven biomarker discovery: enhancing precision in cancer diagnosis and prognosis

Cancer remains a significant health issue, resulting in around 10 million deaths per year, particularly in developing nations. Demographic changes, socio-economic variables, and lifestyle choices are responsible for the rise in cancer cases. Despite the potential to mitigate the adverse effects of cancer by early detection and the implementation of cancer prevention methods, several nations have limited screening facilities. In oncology, the use of artificial intelligence (AI) represents a transformative advancement in cancer diagnosis, prognosis, and treatment. The use of AI in biomarker discovery improves precision medicine by uncovering biomarker signatures that are essential for early detection and treatment of diseases within vast and diverse datasets. Deep learning and machine learning diagnostics are two examples of AI technologies that are changing the way biomarkers are made by finding patterns in large datasets and making new technologies that make it possible to deliver accurate and effective therapies. Existing gaps include data quality, algorithmic transparency, and ethical concerns around privacy, among others. The advancement of biomarker discovery methodologies with AI seeks to transform cancer by improving patient survival rates through enhanced early diagnosis and targeted therapy. This commentary aims to clarify how AI is improving the identification of novel biomarkers for optimal early diagnosis, focused treatment, and improved clinical outcomes, while also addressing certain obstacles and ethical issues related to the application of artificial intelligence in oncology. Data from reputable scientific databases such as PubMed, Scopus, and ScienceDirect were utilized.

Prognostic and predictive values of a multimodal nomogram incorporating tumor and peritumor morphology with immune status in resectable lung adenocarcinoma

BACKGROUND

Current prognostic and predictive biomarkers for lung adenocarcinoma (LUAD) predominantly rely on unimodal approaches, limiting their characterization ability. There is an urgent need for a comprehensive and accurate biomarker to guide individualized adjuvant therapy decisions.

METHODS

In this retrospective study, data from patients with resectable LUAD (stage I-III) were collected from two hospitals and a publicly available dataset, forming a training dataset (n=223), a validation dataset (n=95), a testing dataset (n=449), and the non-small cell lung cancer (NSCLC) Radiogenomics dataset (n=59). Tumor and peritumor scores were constructed from preoperative CT radiomics features (shape/intensity/texture). An immune score was derived from the density of tumor-infiltrating lymphocytes (TILs) within the cancer epithelium and stroma on hematoxylin and eosin-stained whole-slide images. A clinical score was constructed based on clinicopathological risk factors. A Cox regression model was employed to integrate these scores, thereby constructing a multimodal nomogram to predict disease-free survival (DFS). The adjuvant chemotherapy benefit rate was subsequently calculated based on this nomogram.

RESULTS

The multimodal nomogram outperformed each of the unimodal scores in predicting DFS, with a C-index of 0.769 (vs 0.634-0.731) in the training dataset, 0.730 (vs 0.548-0.713) in the validation dataset, and 0.751 (vs 0.660-0.692) in the testing dataset. It was independently associated with DFS after adjusting for other clinicopathological risk factors (training dataset: HR=3.02, p<0.001; validation dataset: HR=2.33, p<0.001; testing dataset: HR=2.03, p=0.001). The adjuvant chemotherapy benefit rate effectively distinguished between patients benefiting from adjuvant chemotherapy and those from observation alone (interaction p<0.001). Furthermore, the high-/low-risk groups defined by the multimodal nomogram provided refined stratification of candidates for adjuvant chemotherapy identified by current guidelines (p<0.001). Gene set enrichment analyses using the NSCLC Radiogenomics dataset revealed associations between tumor/peritumor scores and pathways involved in epithelial-mesenchymal transition, angiogenesis, IL6-JAK-STAT3 signaling, and reactive oxidative species.

CONCLUSION

The multimodal nomogram, which incorporates tumor and peritumor morphology with anti-tumor immune response, provides superior prognostic accuracy compared with unimodal scores. Its defined adjuvant chemotherapy benefit rates can inform individualized adjuvant therapy decisions.

A Multicenter Study on Intraoperative Glioma Grading via Deep Learning on Cryosection Pathology

Intraoperative glioma grading remains a significant challenge primarily due to the diminished diagnostic attributable to the suboptimal quality of cryosectioned slides. Precise intraoperative diagnosis is instrumental in guiding the surgical strategy to balance resection extent and neurologic function preservation, thereby optimizing patient prognoses. This study developed a model for intraoperative glioma grading via deep learning on cryosectioned images, termed intraoperative glioma grading on cryosection (IGGC). The model was trained and validated on The Cancer Genome Atlas data sets and 1 cohort (ntrain = 1603 and nvalidate = 628), and tested on 5 cohorts (ntest = 213). The IGGC model achieved an area under the receiver operating characteristic curve value of 0.99 in differentiating between high-grade glioma and low-grade glioma, and an area under the receiver operating characteristic curve value of 0.96 in identifying grade 4 glioma. Integrated into the clinical workflow, the IGGC model-assisted pathologists of varying experience levels in reducing interobserver variability and enhancing diagnostic consistency. This integrated diagnostic model possesses the potential for clinical implementation, offering a time-efficient and highly accurate method for the 3-grade classification of adult-type diffuse gliomas based on intraoperative cryosectioned slides.

A Multimodal Data Fusion and Embedding Attention Mechanism-Based Method for Eggplant Disease Detection

A novel eggplant disease detection method based on multimodal data fusion and attention mechanisms is proposed in this study, aimed at improving both the accuracy and robustness of disease detection. The method integrates image and sensor data, optimizing the fusion of multimodal features through an embedded attention mechanism, which enhances the model's ability to focus on disease-related features. Experimental results demonstrate that the proposed method excels across various evaluation metrics, achieving a precision of 0.94, recall of 0.90, accuracy of 0.92, and mAP@75 of 0.91, indicating excellent classification accuracy and object localization capability. Further experiments, through ablation studies, evaluated the impact of different attention mechanisms and loss functions on model performance, all of which showed superior performance for the proposed approach. The multimodal data fusion combined with the embedded attention mechanism effectively enhances the accuracy and robustness of the eggplant disease detection model, making it highly suitable for complex disease identification tasks and demonstrating significant potential for widespread application.

Multimodal histopathologic models stratify hormone receptor-positive early breast cancer

The Oncotype DX® Recurrence Score (RS) is an assay for hormone receptor-positive early breast cancer with extensively validated predictive and prognostic value. However, its cost and lag time have limited global adoption, and previous attempts to estimate it using clinicopathologic variables have had limited success. To address this, we assembled 6172 cases across three institutions and developed Orpheus, a multimodal deep learning tool to infer the RS from H&E whole-slide images. Our model identifies TAILORx high-risk cases (RS > 25) with an area under the curve (AUC) of 0.89, compared to a leading clinicopathologic nomogram with 0.73. Furthermore, in patients with RS ≤ 25, Orpheus ascertains risk of metastatic recurrence more accurately than the RS itself (0.75 vs 0.49 mean time-dependent AUC). These findings have the potential to guide adjuvant therapy for high-risk cases and tailor surveillance for patients at elevated metastatic recurrence risk.

Synthetic data generation in healthcare: A scoping review of reviews on domains, motivations, and future applications

BACKGROUND

The development of Artificial Intelligence in the healthcare sector is generating a great impact. However, one of the primary challenges for the implementation of this technology is the access to high-quality data due to issues in data collection and regulatory constraints, for which synthetic data is an emerging alternative. While previous research has reviewed synthetic data generation techniques, there is limited focus on their applications and the motivations driving their synthesis. A comprehensive review is needed to expand the potential of synthetic data into less explored healthcare areas.

OBJECTIVE

This review aims to identify the healthcare domains where synthetic data are currently generated, the motivations behind their creation, their future uses, limitations, and types of data.

MATERIALS AND METHODS

Following the PRISMA-ScR framework, this review analysed literature from the last 10 years within PubMed, Scopus, and Web of Science. Reviews containing information on synthetic data generation in healthcare were screened and analysed. Key healthcare domains, motivations, future uses, and gaps in the literature were identified through a structured data extraction process.

RESULTS

Of the 346 reviews identified, 42 were included for data extraction. Thirteen main domains were identified, with Oncology, Neurology, and Cardiology being the most frequently mentioned. Five primary motivations for synthetic data generation and three major categories of future applications were highlighted. Additionally, unstructured data, particularly images, were found to be the predominant type of synthetic data generated.

DISCUSSION AND CONCLUSION

Synthetic data are currently being generated across diverse healthcare domains, showcasing their adaptability and potential. Despite their early stage, synthetic data technologies hold significant promise for future applications. Expanding their use into new domains and less common data types (e.g., video and text) could further enhance their impact. Future work should focus on developing evaluation benchmarks and standardized generative models tailored to specific healthcare domains.

Generation of synthetic whole-slide image tiles of tumours from RNA-sequencing data via cascaded diffusion models

Training machine-learning models with synthetically generated data can alleviate the problem of data scarcity when acquiring diverse and sufficiently large datasets is costly and challenging. Here we show that cascaded diffusion models can be used to synthesize realistic whole-slide image tiles from latent representations of RNA-sequencing data from human tumours. Alterations in gene expression affected the composition of cell types in the generated synthetic image tiles, which accurately preserved the distribution of cell types and maintained the cell fraction observed in bulk RNA-sequencing data, as we show for lung adenocarcinoma, kidney renal papillary cell carcinoma, cervical squamous cell carcinoma, colon adenocarcinoma and glioblastoma. Machine-learning models pretrained with the generated synthetic data performed better than models trained from scratch. Synthetic data may accelerate the development of machine-learning models in scarce-data settings and allow for the imputation of missing data modalities.

Hallmarks of artificial intelligence contributions to precision oncology

The integration of artificial intelligence (AI) into oncology promises to revolutionize cancer care. In this Review, we discuss ten AI hallmarks in precision oncology, organized into three groups: (1) cancer prevention and diagnosis, encompassing cancer screening, detection and profiling; (2) optimizing current treatments, including patient outcome prediction, treatment planning and monitoring, clinical trial design and matching, and developing response biomarkers; and (3) advancing new treatments by identifying treatment combinations, discovering cancer vulnerabilities and designing drugs. We also survey AI applications in interventional clinical trials and address key challenges to broader clinical adoption of AI: data quality and quantity, model accuracy, clinical relevance and patient benefit, proposing actionable solutions for each.

Quantitative vascular feature-based multimodality prediction model for multi-origin malignant cervical lymphadenopathy

Background

The precise prediction of multi-origin malignant cervical lymphadenopathy is limited by the low inter-reader reproducibility of imaging interpretation, and a quantitative method to improve this aspect is lacking. This study aimed to develop and validate an artificial intelligence framework integrating quantitative vascular features for assessing cervical lymphadenopathy and explore its utility among radiologists.

Methods

For this retrospective study, a total of 21,298 ultrasound images of 10,649 cervical lymph nodes (LNs) from 10,386 patients and 2366 images of 1183 LNs from 1151 patients at the Sun Yat-sen University Cancer Center between January 2011 and July 2022 were used for model development and internal testing, respectively. For external model testing, we used 776 images of 388 LNs from 360 patients at the Chongqing University Cancer Hospital between January and December 2022. Quantitative features used to characterize the vascular distribution and degree of richness were fused with morphological and semantic features on B-mode and color Doppler ultrasound images to develop a dual-modality, multi-feature, fusion lymph node network (DMFLNN). Subsequently, the performance of DMFLNN was compared with that of six radiologists, and its auxiliary value was assessed in test cohorts.

Findings

DMFLNN achieved an area under the receiver operating characteristic curve (AUC) of 0.937 for the internal test cohort and 0.875 for the external test cohort. Using the internal test cohort with assistance from DMFLNN, the average AUC improved from 0.814 to 0.836 for senior radiologists (P = 0.00018), and from 0.778 to 0.847 for junior radiologists (P < 0.0001). Additionally, the average inter-radiologist agreement improved from fair to moderate (improvement in kappa: from 0.590 to 0.696 for senior radiologists; from 0.571 to 0.750 for junior radiologists). Similar trends were observed for the external test cohort. Moreover, the radiologists' average false-positive rate decreased by 3.8% and 9.8% for the internal and external test cohorts, respectively.

Interpretation

DMFLNN could improve radiologists' performance and potentially reduce unnecessary biopsies of cervical lymphadenopathy. However, further testing is warranted before its wide adoption in clinical practice.

Funding

The National Natural Science Foundation of China (82171955; 62371476; 82441027); the China Department of Science and Technology (2023YFE0204300); and the R&D project of Pazhou Lab (HuangPu) (2023K0606).

Exploring a decade of deep learning in dentistry: A comprehensive mapping review

OBJECTIVES

Artificial Intelligence (AI), particularly deep learning, has significantly impacted healthcare, including dentistry, by improving diagnostics, treatment planning, and prognosis prediction. This systematic mapping review explores the current applications of deep learning in dentistry, offering a comprehensive overview of trends, models, and their clinical significance.

MATERIALS AND METHODS

Following a structured methodology, relevant studies published from January 2012 to September 2023 were identified through database searches in PubMed, Scopus, and Embase. Key data, including clinical purpose, deep learning tasks, model architectures, and data modalities, were extracted for qualitative synthesis.

RESULTS

From 21,242 screened studies, 1,007 were included. Of these, 63.5% targeted diagnostic tasks, primarily with convolutional neural networks (CNNs). Classification (43.7%) and segmentation (22.9%) were the main methods, and imaging data-such as cone-beam computed tomography and orthopantomograms-were used in 84.4% of cases. Most studies (95.2%) applied fully supervised learning, emphasizing the need for annotated data. Pathology (21.5%), radiology (17.5%), and orthodontics (10.2%) were prominent fields, with 24.9% of studies relating to more than one specialty.

CONCLUSION

This review explores the advancements in deep learning in dentistry, particulary for diagnostics, and identifies areas for further improvement. While CNNs have been used successfully, it is essential to explore emerging model architectures, learning approaches, and ways to obtain diverse and reliable data. Furthermore, fostering trust among all stakeholders by advancing explainable AI and addressing ethical considerations is crucial for transitioning AI from research to clinical practice.

CLINICAL RELEVANCE

This review offers a comprehensive overview of a decade of deep learning in dentistry, showcasing its significant growth in recent years. By mapping its key applications and identifying research trends, it provides a valuable guide for future studies and highlights emerging opportunities for advancing AI-driven dental care.

Enhancing rectal cancer liver metastasis prediction: Magnetic resonance imaging-based radiomics, bias mitigation, and regulatory considerations

In this article, we comment on the article by Long et al published in the recent issue of the World Journal of Gastrointestinal Oncology. Rectal cancer patients are at risk for developing metachronous liver metastasis (MLM), yet early prediction remains challenging due to variations in tumor heterogeneity and the limitations of traditional diagnostic methods. Therefore, there is an urgent need for non-invasive techniques to improve patient outcomes. Long et al's study introduces an innovative magnetic resonance imaging (MRI)-based radiomics model that integrates high-throughput imaging data with clinical variables to predict MLM. The study employed a 7:3 split to generate training and validation datasets. The MLM prediction model was constructed using the training set and subsequently validated on the validation set using area under the curve (AUC) and dollar-cost averaging metrics to assess performance, robustness, and generalizability. By employing advanced algorithms, the model provides a non-invasive solution to assess tumor heterogeneity for better metastasis prediction, enabling early intervention and personalized treatment planning. However, variations in MRI parameters, such as differences in scanning resolutions and protocols across facilities, patient heterogeneity (e.g., age, comorbidities), and external factors like carcinoembryonic antigen levels introduce biases. Additionally, confounding factors such as diagnostic staging methods and patient comorbidities require further validation and adjustment to ensure accuracy and generalizability. With evolving Food and Drug Administration regulations on machine learning models in healthcare, compliance and careful consideration of these regulatory requirements are essential to ensuring safe and effective implementation of this approach in clinical practice. In the future, clinicians may be able to utilize data-driven, patient-centric artificial intelligence (AI)-enhanced imaging tools integrated with clinical data, which would help improve early detection of MLM and optimize personalized treatment strategies. Combining radiomics, genomics, histological data, and demographic information can significantly enhance the accuracy and precision of predictive models.

Current Advancements in Digital Neuropathology and Machine Learning for the Study of Neurodegenerative Diseases

Computational neurodegenerative neuropathology represents a transformative approach in the analysis and understanding of neurodegenerative diseases through utilization of whole slide images (WSIs) and advanced machine learning/artificial intelligence (ML/AI) techniques. This review explores the emerging field of computational neurodegenerative neuropathology, emphasizing its potential to enhance neuropathologic assessment, diagnosis, and research. Recent advancements in ML/AI technologies have significantly affected image-based medical fields, including anatomic pathology, by automating disease staging, identifying novel morphologic biomarkers, and uncovering new clinical insights via multi-modal AI approaches. Despite its promise, the field faces several challenges, including limited expert annotations, slide scanning inaccessibility, inter-institutional variability, and the complexities of sharing large WSI data sets. This review discusses the importance of improving deep learning model accuracy and efficiency for better interpretation of neuropathologic data. It highlights the potential of unsupervised learning to identify patterns in unannotated data. Furthermore, the development of explainable AI models is crucial for experimental neuropathology. By addressing these challenges and leveraging cutting-edge AI techniques, computational neurodegenerative neuropathology has the potential to revolutionize the field and significantly advance our understanding of disease.

Progressive role of artificial intelligence in treatment decision-making in the field of medical oncology

This article explores the role of artificial intelligence (AI) in medical oncology, emphasizing its impact on treatment decision-making for adult and pediatric cancer care. AI applications, including advanced imaging, drug discovery, and clinical decision support systems, enhance precision, personalization, and efficiency. Pediatric oncology benefits from improved diagnostics, risk stratification, and targeted therapies, despite unique challenges. AI-driven personalized medicine optimizes treatment strategies, improving patient outcomes and reducing costs. Ethical considerations, such as data privacy, algorithmic bias, and explainability, remain critical for responsible AI integration. Future advancements, including explainable AI and quantum computing, promise to redefine cancer care through data-driven insights.

Advancing Precision Oncology Through Modeling of Longitudinal and Multimodal Data

Cancer evolves continuously over time through a complex interplay of genetic, epigenetic, microenvironmental, and phenotypic changes. This dynamic behavior drives uncontrolled cell growth, metastasis, immune evasion, and therapy resistance, posing challenges for effective monitoring and treatment. However, today's data-driven research in oncology has primarily focused on cross-sectional analysis using data from a single modality, limiting the ability to fully characterize and interpret the disease's dynamic heterogeneity. Advances in multiscale data collection and computational methods now enable the discovery of longitudinal multimodal biomarkers for precision oncology. Longitudinal data reveal patterns of disease progression and treatment response that are not evident from single-timepoint data, enabling timely abnormality detection and dynamic treatment adaptation. Multimodal data integration offers complementary information from diverse sources for more precise risk assessment and targeting of cancer therapy. In this review, we survey methods of longitudinal and multimodal modeling, highlighting their synergy in providing multifaceted insights for personalized care tailored to the unique characteristics of a patient's cancer. We summarize the current challenges and future directions of longitudinal multimodal analysis in advancing precision oncology.

Applications of Machine Learning-Driven Molecular Models for Advancing Ophthalmic Precision Medicine

Ophthalmic diseases such as glaucoma, age-related macular degeneration (ARMD), and optic neuritis involve complex molecular and cellular disruptions that challenge current diagnostic and therapeutic approaches. Advanced artificial intelligence (AI) and machine learning (ML) models offer a novel lens to analyze these diseases by integrating diverse datasets, identifying patterns, and enabling precision medicine strategies. Over the past decade, applications of AI in ophthalmology have expanded from imaging-based diagnostics to molecular-level modeling, bridging critical gaps in understanding disease mechanisms. This paper systematically reviews the application of AI-driven methods, including reinforcement learning (RL), graph neural networks (GNNs), Bayesian inference, and generative adversarial networks (GANs), in the context of these ophthalmic conditions. RL models simulate transcription factor dynamics in hypoxic or inflammatory environments, offering insights into disrupted molecular pathways. GNNs map intricate molecular networks within affected tissues, identifying key inflammatory or degenerative drivers. Bayesian inference provides probabilistic models for predicting disease progression and response to therapies, while GANs generate synthetic datasets to explore therapeutic interventions. By contextualizing these AI tools within the broader framework of ophthalmic disease management, this review highlights their potential to transform diagnostic precision and therapeutic outcomes. Ultimately, this work underscores the need for continued interdisciplinary collaboration to harness AI's potential in advancing the field of ophthalmology and improving patient care.

Framework for the Pathology Workup of Metastatic Castration-Resistant Prostate Cancer Biopsies

Lineage plasticity and histologic transformation from prostate adenocarcinoma to neuroendocrine (NE) prostate cancer (NEPC) occur in up to 15% to 20% of patients with castration-resistant prostate cancer (CRPC) as a mechanism of treatment resistance and are associated with aggressive disease and poor prognosis. NEPC tumors typically display small cell carcinoma morphology with loss of androgen receptor (AR) expression and gain of NE lineage markers. However, there is a spectrum of phenotypes that are observed during the lineage plasticity process, and the clinical significance of mixed histologies or those that co-express AR and NE markers or lack all markers is not well defined. Translational research studies investigating NEPC have used variable definitions, making clinical trial design challenging. In this manuscript, we discuss the diagnostic workup of metastatic biopsies to help guide the reproducible classification of phenotypic CRPC subtypes. We recommend classifying CRPC tumors based on histomorphology (adenocarcinoma, small cell carcinoma, poorly differentiated carcinoma, other morphologic variant, or mixed morphology) and IHC markers with a priority for AR, NK3 homeobox 1, insulinoma-associated protein 1, synaptophysin, and cell proliferation based on Ki-67 positivity, with additional markers to be considered based on the clinical context. Ultimately, a unified workup of metastatic CRPC biopsies can improve clinical trial design and eventually practice.

Artificial intelligence for diagnosis and predictive biomarkers in Non-Small cell lung cancer Patients: New promises but also new hurdles for the pathologist

The rapid development of artificial intelligence (AI) based tools in pathology laboratories has brought forward unlimited opportunities for pathologists. Promising AI applications used for accomplishing diagnostic, prognostic and predictive tasks are being developed at a high pace. This is notably true in thoracic oncology, given the significant and rapid therapeutic progress made recently for lung cancer patients. Advances have been based on drugs targeting molecular alterations, immunotherapies, and, more recently antibody-drug conjugates which are soon to be introduced. For over a decade, many proof-of-concept studies have explored the use of AI algorithms in thoracic oncology to improve lung cancer patient care. However, despite the enthusiasm in this domain, the set-up and use of AI algorithms in daily practice of thoracic pathologists has not been operative until now, due to several constraints. The purpose of this review is to describe the potential but also the current barriers of AI applications in routine thoracic pathology for non-small cell lung cancer patient care and to suggest practical solutions for rapid future implementation.

Let it shine: Autofluorescence of Papanicolaou-stain improves AI-based cytological oral cancer detection

BACKGROUND AND OBJECTIVES

Oral cancer is a global health challenge. The disease can be successfully treated if detected early, but the survival rate drops significantly for late stage cases. There is a growing interest in a shift from the current standard of invasive and time-consuming tissue sampling and histological examination, towards non-invasive brush biopsies and cytological examination, facilitating continued risk group monitoring. For cost effective and accurate cytological analysis there is a great need for reliable computer-assisted data-driven approaches. However, infeasibility of accurate cell-level annotation hinders model performance, and limits evaluation and interpretation of the results. This study aims to improve AI-based oral cancer detection by introducing additional information through multimodal imaging and deep multimodal information fusion.

METHODS

We combine brightfield and fluorescence whole slide microscopy imaging to analyze Papanicolaou-stained liquid-based cytology slides of brush biopsies collected from both healthy and cancer patients. Given the challenge of detailed cytological annotations, we utilize a weakly supervised deep learning approach only relying on patient-level labels. We evaluate various multimodal information fusion strategies, including early, late, and three recent intermediate fusion methods.

RESULTS

Our experiments demonstrate that: (i) there is substantial diagnostic information to gain from fluorescence imaging of Papanicolaou-stained cytological samples, (ii) multimodal information fusion improves classification performance and cancer detection accuracy, compared to single-modality approaches. Intermediate fusion emerges as the leading method among the studied approaches. Specifically, the Co-Attention Fusion Network (CAFNet) model achieves impressive results, with an F1 score of 83.34% and an accuracy of 91.79% at cell level, surpassing human performance on the task. Additional tests highlight the importance of accurate image registration to maximize the benefits of the multimodal analysis.

CONCLUSION

This study advances the field of cytopathology by integrating deep learning methods, multimodal imaging and information fusion to enhance non-invasive early detection of oral cancer. Our approach not only improves diagnostic accuracy, but also allows an efficient, yet uncomplicated, clinical workflow. The developed pipeline has potential applications in other cytological analysis settings. We provide a validated open-source analysis framework and share a unique multimodal oral cancer dataset to support further research and innovation.

Decoding pan-cancer treatment outcomes using multimodal real-world data and explainable artificial intelligence

Despite advances in precision oncology, clinical decision-making still relies on limited variables and expert knowledge. To address this limitation, we combined multimodal real-world data and explainable artificial intelligence (xAI) to introduce AI-derived (AID) markers for clinical decision support. We used xAI to decode the outcome of 15,726 patients across 38 solid cancer entities based on 350 markers, including clinical records, image-derived body compositions, and mutational tumor profiles. xAI determined the prognostic contribution of each clinical marker at the patient level and identified 114 key markers that accounted for 90% of the neural network's decision process. Moreover, xAI enabled us to uncover 1,373 prognostic interactions between markers. Our approach was validated in an independent cohort of 3,288 patients with lung cancer from a US nationwide electronic health record-derived database. These results show the potential of xAI to transform the assessment of clinical variables and enable personalized, data-driven cancer care.

A vision-language foundation model for precision oncology

Clinical decision-making is driven by multimodal data, including clinical notes and pathological characteristics. Artificial intelligence approaches that can effectively integrate multimodal data hold significant promise in advancing clinical care1,2. However, the scarcity of well-annotated multimodal datasets in clinical settings has hindered the development of useful models. In this study, we developed the Multimodal transformer with Unified maSKed modeling (MUSK), a vision-language foundation model designed to leverage large-scale, unlabelled, unpaired image and text data. MUSK was pretrained on 50 million pathology images from 11,577 patients and one billion pathology-related text tokens using unified masked modelling. It was further pretrained on one million pathology image-text pairs to efficiently align the vision and language features. With minimal or no further training, MUSK was tested in a wide range of applications and demonstrated superior performance across 23 patch-level and slide-level benchmarks, including image-to-text and text-to-image retrieval, visual question answering, image classification and molecular biomarker prediction. Furthermore, MUSK showed strong performance in outcome prediction, including melanoma relapse prediction, pan-cancer prognosis prediction and immunotherapy response prediction in lung and gastro-oesophageal cancers. MUSK effectively combined complementary information from pathology images and clinical reports and could potentially improve diagnosis and precision in cancer therapy.

Parallel CNN-Deep Learning Clinical-Imaging Signature for Assessing Pathologic Grade and Prognosis of Soft Tissue Sarcoma Patients

BACKGROUND

Traditional biopsies pose risks and may not accurately reflect soft tissue sarcoma (STS) heterogeneity. MRI provides a noninvasive, comprehensive alternative.

PURPOSE

To assess the diagnostic accuracy of histological grading and prognosis in STS patients when integrating clinical-imaging parameters with deep learning (DL) features from preoperative MR images.

STUDY TYPE

Retrospective/prospective.

POPULATION

354 pathologically confirmed STS patients (226 low-grade, 128 high-grade) from three hospitals and the Cancer Imaging Archive (TCIA), divided into training (n = 185), external test (n = 125), and TCIA cohorts (n = 44). 12 patients (6 low-grade, 6 high-grade) were enrolled into prospective validation cohort.

FIELD STRENGTH/SEQUENCE

1.5 T and 3.0 T/Unenhanced T1-weighted and fat-suppressed-T2-weighted.

ASSESSMENT

DL features were extracted from MR images using a parallel ResNet-18 model to construct DL signature. Clinical-imaging characteristics included age, gender, tumor-node-metastasis stage and MRI semantic features (depth, number, heterogeneity at T1WI/FS-T2WI, necrosis, and peritumoral edema). Logistic regression analysis identified significant risk factors for the clinical model. A DL clinical-imaging signature (DLCS) was constructed by incorporating DL signature with risk factors, evaluated for risk stratification, and assessed for progression-free survival (PFS) in retrospective cohorts, with an average follow-up of 23 ± 22 months.

STATISTICAL TESTS

Logistic regression, Cox regression, Kaplan-Meier curves, log-rank test, area under the receiver operating characteristic curve (AUC)，and decision curve analysis. A P-value <0.05 was considered significant.

RESULTS

The AUC values for DLCS in the external test, TCIA, and prospective test cohorts (0.834, 0.838, 0.819) were superior to clinical model (0.662, 0.685, 0.694). Decision curve analysis showed that the DLCS model provided greater clinical net benefit over the DL and clinical models. Also, the DLCS model was able to risk-stratify patients and assess PFS.

DATA CONCLUSION

The DLCS exhibited strong capabilities in histological grading and prognosis assessment for STS patients, and may have potential to aid in the formulation of personalized treatment plans.

LEVEL OF EVIDENCE: 4

TECHNICAL EFFICACY

Stage 2.

Tumor-Targeted Catalytic Immunotherapy

Cancer immunotherapy holds significant promise for improving cancer treatment efficacy; however, the low response rate remains a considerable challenge. To overcome this limitation, advanced catalytic materials offer potential in augmenting catalytic immunotherapy by modulating the immunosuppressive tumor microenvironment (TME) through precise biochemical reactions. Achieving optimal targeting precision and therapeutic efficacy necessitates a thorough understanding of the properties and underlying mechanisms of tumor-targeted catalytic materials. This review provides a comprehensive and systematic overview of recent advancements in tumor-targeted catalytic materials and their critical role in enhancing catalytic immunotherapy. It highlights the types of catalytic reactions, the construction strategies of catalytic materials, and their fundamental mechanisms for tumor targeting, including passive, bioactive, stimuli-responsive, and biomimetic targeting approaches. Furthermore, this review outlines various tumor-specific targeting strategies, encompassing tumor tissue, tumor cell, exogenous stimuli-responsive, TME-responsive, and cellular TME targeting strategies. Finally, the discussion addresses the challenges and future perspectives for transitioning catalytic materials into clinical applications, offering insights that pave the way for next-generation cancer therapies and provide substantial benefits to patients in clinical settings.

Convergence of evolving artificial intelligence and machine learning techniques in precision oncology

The confluence of new technologies with artificial intelligence (AI) and machine learning (ML) analytical techniques is rapidly advancing the field of precision oncology, promising to improve diagnostic approaches and therapeutic strategies for patients with cancer. By analyzing multi-dimensional, multiomic, spatial pathology, and radiomic data, these technologies enable a deeper understanding of the intricate molecular pathways, aiding in the identification of critical nodes within the tumor's biology to optimize treatment selection. The applications of AI/ML in precision oncology are extensive and include the generation of synthetic data, e.g., digital twins, in order to provide the necessary information to design or expedite the conduct of clinical trials. Currently, many operational and technical challenges exist related to data technology, engineering, and storage; algorithm development and structures; quality and quantity of the data and the analytical pipeline; data sharing and generalizability; and the incorporation of these technologies into the current clinical workflow and reimbursement models.

A morphometric signature to identify ductal carcinoma in situ with a low risk of progression

Ductal carcinoma in situ (DCIS) may progress to ipsilateral invasive breast cancer (iIBC), but often never will. Because DCIS is treated as early breast cancer, many women with harmless DCIS face overtreatment. To identify features associated with progression, we developed an artificial intelligence-based DCIS morphometric analysis pipeline (AIDmap) on hematoxylin-eosin-stained (H&E) tissue sections. We analyzed 689 digitized H&Es of pure primary DCIS of which 226 were diagnosed with subsequent iIBC and 463 were not. The distribution of 15 duct morphological measurements was summarized in 55 morphometric variables. A ridge regression classifier with cross validation predicted 5-years-free of iIBC with an area-under the curve of 0.67 (95% CI 0.57-0.77). A combined clinical-morphometric signature, characterized by small-sized ducts, a low number of cells and a low DCIS/stroma ratio, was associated with outcome (HR = 0.56; 95% CI 0.28-0.78). AIDmap has potential to identify harmless DCIS that may not need treatment.

The clinical application of artificial intelligence in cancer precision treatment

BACKGROUND

Artificial intelligence has made significant contributions to oncology through the availability of high-dimensional datasets and advances in computing and deep learning. Cancer precision medicine aims to optimize therapeutic outcomes and reduce side effects for individual cancer patients. However, a comprehensive review describing the impact of artificial intelligence on cancer precision medicine is lacking.

OBSERVATIONS

By collecting and integrating large volumes of data and applying it to clinical tasks across various algorithms and models, artificial intelligence plays a significant role in cancer precision medicine. Here, we describe the general principles of artificial intelligence, including machine learning and deep learning. We further summarize the latest developments in artificial intelligence applications in cancer precision medicine. In tumor precision treatment, artificial intelligence plays a crucial role in individualizing both conventional and emerging therapies. In specific fields, including target prediction, targeted drug generation, immunotherapy response prediction, neoantigen prediction, and identification of long non-coding RNA, artificial intelligence offers promising perspectives. Finally, we outline the current challenges and ethical issues in the field.

CONCLUSIONS

Recent clinical studies demonstrate that artificial intelligence is involved in cancer precision medicine and has the potential to benefit cancer healthcare, particularly by optimizing conventional therapies, emerging targeted therapies, and individual immunotherapies. This review aims to provide valuable resources to clinicians and researchers and encourage further investigation in this field.

Histopathology and proteomics are synergistic for high-grade serous ovarian cancer platinum response prediction

Patients with High-Grade Serous Ovarian Cancer (HGSOC) exhibit varied responses to treatment, with 20-30% showing de novo resistance to platinum-based chemotherapy. While hematoxylin-eosin (H&E)-stained pathological slides are used for routine diagnosis of cancer type, they may also contain diagnostically useful information about treatment response. Our study demonstrates that combining H&E-stained whole slide images (WSIs) with proteomic signatures using a multimodal deep learning framework significantly improves the prediction of platinum response in both discovery and validation cohorts. This method outperforms the Homologous Recombination Deficiency (HRD) score in predicting platinum response and overall patient survival. Our study suggests that histology and proteomics contain complementary information about biological processes determining response to first line platinum treatment in HGSOC. This integrative approach has the potential to improve personalized treatment and provide insights into the therapeutic vulnerabilities of HGSOC.

Digital pathology and artificial intelligence in renal cell carcinoma focusing on feature extraction: a literature review

The integrated application of artificial intelligence (AI) and digital pathology (DP) technology has opened new avenues for advancements in oncology and molecular pathology. Consequently, studies in renal cell carcinoma (RCC) have emerged, highlighting potential in histological subtype classification, molecular aberration identification, and outcome prediction by extracting high-throughput features. However, reviews of these studies are still rare. To address this gap, we conducted a thorough literature review on DP and AI applications in RCC through database searches. Notably, we found that AI models based on deep learning achieved area under the curve (AUC) of over 0.93 in subtype classification, 0.89-0.96 in grading of clear cell RCC, 0.70-0,89 in molecular prediction, and over 0.78 in survival prediction. This review finally discussed the current state of researches and potential future directions.

Machine learning techniques for predicting neurodevelopmental impairments in premature infants: a systematic review

Background and objective

Very preterm infants are highly susceptible to Neurodevelopmental Impairments (NDIs), including cognitive, motor, and language deficits. This paper presents a systematic review of the application of Machine Learning (ML) techniques to predict NDIs in premature infants.

Methods

This review presents a comparative analysis of existing studies from January 2018 to December 2023, highlighting their strengths, limitations, and future research directions.

Results

We identified 26 studies that fulfilled the inclusion criteria. In addition, we explore the potential of ML algorithms and discuss commonly used data sources, including clinical and neuroimaging data. Furthermore, the inclusion of omics data as a contemporary approach employed, in other diagnostic contexts is proposed.

Conclusions

We identified limitations and emphasized the significance of employing multimodal data models and explored various alternatives to address the limitations identified in the reviewed studies. The insights derived from this review guide researchers and clinicians toward improving early identification and intervention strategies for NDIs in this vulnerable population.

A practical guide to FAIR data management in the age of multi-OMICS and AI

Multi-cellular biological systems, including the immune system, are highly complex, dynamic, and adaptable. Systems biologists aim to understand such complexity at a quantitative level. However, these ambitious efforts are often limited by access to a variety of high-density intra-, extra- and multi-cellular measurements resolved in time and space and across a variety of perturbations. The advent of automation, OMICs and single-cell technologies now allows high dimensional multi-modal data acquisition from the same biological samples multiplexed at scale (multi-OMICs). As a result, systems biologists -theoretically- have access to more data than ever. However, the mathematical frameworks and computational tools needed to analyze and interpret such data are often still nascent, limiting the biological insights that can be obtained without years of computational method development and validation. More pressingly, much of the data sits in silos in formats that are incomprehensible to other scientists or machines limiting its value to the vaster scientific community, especially the computational biologists tasked with analyzing these vast amounts of data in more nuanced ways. With the rapid development and increasing interest in using artificial intelligence (AI) for the life sciences, improving how biologic data is organized and shared is more pressing than ever for scientific progress. Here, we outline a practical approach to multi-modal data management and FAIR sharing, which are in line with the latest US and EU funders' data sharing policies. This framework can help extend the longevity and utility of data by allowing facile use and reuse, accelerating scientific discovery in the biomedical sciences.

Deep Learning for Biomarker Discovery in Cancer Genomes

Background

Genomic data is essential for clinical decision-making in precision oncology. Bioinformatic algorithms are widely used to analyze next-generation sequencing (NGS) data, but they face two major challenges. First, these pipelines are highly complex, involving multiple steps and the integration of various tools. Second, they generate features that are human-interpretable but often result in information loss by focusing only on predefined genetic properties. This limitation restricts the full potential of NGS data in biomarker extraction and slows the discovery of new biomarkers in precision oncology.

Methods

We propose an end-to-end deep learning (DL) approach for analyzing NGS data. Specifically, we developed a multiple instance learning DL framework that integrates somatic mutation sequences to predict two compound biomarkers: microsatellite instability (MSI) and homologous recombination deficiency (HRD). To achieve this, we utilized data from 3,184 cancer patients obtained from two public databases: The Cancer Genome Atlas (TCGA) and the Clinical Proteome Tumor Analysis Consortium (CPTAC).

Results

Our proposed deep learning method demonstrated high accuracy in identifying clinically relevant biomarkers. For predicting MSI status, the model achieved an accuracy of 0.98, a sensitivity of 0.95, and a specificity of 1.00 on an external validation cohort. For predicting HRD status, the model achieved an accuracy of 0.80, a sensitivity of 0.75, and a specificity of 0.86. Furthermore, the deep learning approach significantly outperformed traditional machine learning methods in both tasks (MSI accuracy, p-value = 5.11×10-18; HRD accuracy, p-value = 1.07×10-10). Using explainability techniques, we demonstrated that the model's predictions are based on biologically meaningful features, aligning with key DNA damage repair mutation signatures.

Conclusion

We demonstrate that deep learning can identify patterns in unfiltered somatic mutations without the need for manual feature extraction. This approach enhances the detection of actionable targets and paves the way for developing NGS-based biomarkers using minimally processed data.

Self-Supervised Representation Distribution Learning for Reliable Data Augmentation in Histopathology WSI Classification

Multiple instance learning (MIL) based whole slide image (WSI) classification is often carried out on the representations of patches extracted from WSI with a pre-trained patch encoder. The performance of classification relies on both patch-level representation learning and MIL classifier training. Most MIL methods utilize a frozen model pre-trained on ImageNet or a model trained with self-supervised learning on histopathology image dataset to extract patch image representations and then fix these representations in the training of the MIL classifiers for efficiency consideration. However, the invariance of representations cannot meet the diversity requirement for training a robust MIL classifier, which has significantly limited the performance of the WSI classification. In this paper, we propose a Self-Supervised Representation Distribution Learning framework (SSRDL) for patch-level representation learning with an online representation sampling strategy (ORS) for both patch feature extraction and WSI-level data augmentation. The proposed method was evaluated on three datasets under three MIL frameworks. The experimental results have demonstrated that the proposed method achieves the best performance in histopathology image representation learning and data augmentation and outperforms state-of-the-art methods under different WSI classification frameworks. The code is available at https://github.com/lazytkm/SSRDL.