3D multi-scale, multi-task, and multi-label deep learning for prediction of lymph node metastasis in T1 lung adenocarcinoma patients' CT images

CT-based radiomics predictive model for spread through air space of IA stage lung adenocarcinoma

BackgroundSpread through air spaces (STAS) in lung adenocarcinoma means different treatment and worse prognosis.PurposeTo construct a radiomics model based on CT scans to predict the presence of STAS in stage IA lung adenocarcinoma, compared with the traditional clinical model.Material and MethodsThe study included 317 patients (median age = 57.21 years; age range = 45.84-68.61 years) with pathologically confirmed stage IA lung adenocarcinoma. In total, 122 (38.5%) patients were diagnosed with STAS by pathology after the operation. Two experienced radiologists independently segmented the lesions using MITK software and extracted 1791 radiomics features using Python. Single-factor t-test or Mann-Whitney U-test and LASSO were used to screen for radiomics signatures related to STAS. This study constructed a radiomics model, a clinical model, and a combined model, combining radiomics and clinical features. Model performance was evaluated using the area under the curve (AUC).ResultsBy single-factor analysis, four clinical features and 13 radiomics features were significantly associated with STAS. The three models (the clinical, radiomics, and combine models) achieved predictive efficacy, with an AUC of 0.849, 0.867, and 0.939, respectively, in the training set and 0.808, 0.848, and 0.876, respectively, in the testing set.ConclusionThe combined model based on the radiomics and clinical features of preoperative chest CT could be used to preoperatively diagnose the presence of STAS in stage IA lung adenocarcinoma and has an excellent diagnostic performance.

Navigating the Multiverse: a Hitchhiker's guide to selecting harmonization methods for multimodal biomedical data

The application of machine learning (ML) techniques in predictive modelling has greatly advanced our comprehension of biological systems. There is a notable shift in the trend towards integration methods that specifically target the simultaneous analysis of multiple modes or types of data, showcasing superior results compared to individual analyses. Despite the availability of diverse ML architectures for researchers interested in embracing a multimodal approach, the current literature lacks a comprehensive taxonomy that includes the pros and cons of these methods to guide the entire process. Closing this gap is imperative, necessitating the creation of a robust framework. This framework should not only categorize the diverse ML architectures suitable for multimodal analysis but also offer insights into their respective advantages and limitations. Additionally, such a framework can serve as a valuable guide for selecting an appropriate workflow for multimodal analysis. This comprehensive taxonomy would provide a clear guidance and support informed decision-making within the progressively intricate landscape of biomedical and clinical data analysis. This is an essential step towards advancing personalized medicine. The aims of the work are to comprehensively study and describe the harmonization processes that are performed and reported in the literature and present a working guide that would enable planning and selecting an appropriate integrative model. We present harmonization as a dual process of representation and integration, each with multiple methods and categories. The taxonomy of the various representation and integration methods are classified into six broad categories and detailed with the advantages, disadvantages and examples. A guide flowchart describing the step-by-step processes that are needed to adopt a multimodal approach is also presented along with examples and references. This review provides a thorough taxonomy of methods for harmonizing multimodal data and introduces a foundational 10-step guide for newcomers to implement a multimodal workflow.

Predicting lymph node metastasis of clinical T1 non-small cell lung cancer: a brief review of possible methodologies and controversies

Non-small cell lung cancer (NSCLC) is a major subtype of lung cancer and poses a serious threat to human health. Due to the advances in lung cancer screening, more and more clinical T1 NSCLC defined as a tumor with a maximum diameter of 3cm surrounded by lung tissue or visceral pleura have been detected and have achieved favorable treatment outcomes, greatly improving the prognosis of NSCLC patients. However, the preoperative lymph node staging and intraoperative lymph node dissection patterns of operable clinical T1 NSCLC are still subject to much disagreement, as well as the heterogeneity between primary tumors and metastatic lymph nodes poses a challenge in designing effective treatment strategies. This article comprehensively describes the clinical risk factors of clinical T1 NSCLC lymph node metastasis, and its invasive and non-invasive prediction, focusing on the genetic heterogeneity between the primary tumor and the metastatic lymph nodes, which is significant for a thoroughly understanding of the biological behavior of early-stage NSCLC.

Research Trends of Artificial Intelligence in Lung Cancer: A Combined Approach of Analysis With Latent Dirichlet Allocation and HJ-Biplot Statistical Methods

Lung cancer (LC) remains one of the leading causes of cancer-related mortality worldwide. With recent technological advances, artificial intelligence (AI) has begun to play a crucial role in improving diagnostic and treatment methods. It is crucial to understand how AI has integrated into LC research and to identify the main areas of focus. The aim of the study was to provide an updated insight into the role of AI in LC research, analyzing evolving topics, geographical distribution, and contributions to journals. The study explores research trends in AI applied to LC through a novel approach combining latent Dirichlet allocation (LDA) topic modeling with the HJ-Biplot statistical technique. A growing interest in AI applications in LC oncology was observed, reflected in a significant increase in publications, especially after 2017, coinciding with the availability of computing resources. Frontiers in Oncology leads in publishing AI-related LC research, reflecting rigorous investigation in the field. Geographically, China and the United States lead in contributions, attributed to significant investment in R&D and corporate sector involvement. LDA analysis highlights key research areas such as pulmonary nodule detection, patient prognosis prediction, and clinical decision support systems, demonstrating the impact of AI in improving LC outcomes. DL and AI emerge as prominent trends, focusing on radiomics and feature selection, promising better decision-making in LC care. The increase in AI-driven research covers various topics, including data analysis methodologies, tumor characterization, and predictive methods, indicating a concerted effort to advance LC research. HJ-Biplot visualization reveals thematic clustering, illustrating temporal and geographical associations and highlighting the influence of high-impact journals and countries with advanced research capabilities. This multivariate approach offers insights into global collaboration dynamics and specialization, emphasizing the evolving role of AI in LC research and diagnosis.

Ranking attention multiple instance learning for lymph node metastasis prediction on multicenter cervical cancer MRI

PURPOSE

In the current clinical diagnostic process, the gold standard for lymph node metastasis (LNM) diagnosis is histopathological examination following surgical lymphadenectomy. Developing a non-invasive and preoperative method for predicting LNM is necessary and holds significant clinical importance.

METHODS

We develop a ranking attention multiple instance learning (RA-MIL) model that integrates convolutional neural networks (CNNs) and ranking attention pooling to diagnose LNM from T2 MRI. Our RA-MIL model applies the CNNs to derive imaging features from 2D MRI slices and employs ranking attention pooling to create patient-level feature representation for diagnostic classification. Based on the MIL and attention theory, informative regions of top-ranking MRI slices from LNM-positive patients are visualized to enhance the interpretability of automatic LNM prediction. This retrospective study collected 300 female patients with cervical cancer who underwent T2-weighted magnetic resonance imaging (MRI) scanning and histopathological diagnosis from one hospital (289 patients) and one open-source dataset (11 patients).

RESULTS

Our RA-MIL model delivers promising LNM prediction performance, achieving the area under the receiver operating characteristic curve (AUC) of 0.809 on the internal test set and 0.833 on the public dataset. Experiments show significant improvements in LNM status prediction using the proposed RA-MIL model compared with other state-of-the-art (SOTA) comparative deep learning models.

CONCLUSIONS

The developed RA-MIL model has the potential to serve as a non-invasive auxiliary tool for preoperative LNM prediction, offering visual interpretability regarding informative MRI slices and regions in LNM-positive patients.

Dual ensemble system for polyp segmentation with submodels adaptive selection ensemble

Colonoscopy is one of the main methods to detect colon polyps, and its detection is widely used to prevent and diagnose colon cancer. With the rapid development of computer vision, deep learning-based semantic segmentation methods for colon polyps have been widely researched. However, the accuracy and stability of some methods in colon polyp segmentation tasks show potential for further improvement. In addition, the issue of selecting appropriate sub-models in ensemble learning for the colon polyp segmentation task still needs to be explored. In order to solve the above problems, we first implement the utilization of multi-complementary high-level semantic features through the Multi-Head Control Ensemble. Then, to solve the sub-model selection problem in training, we propose SDBH-PSO Ensemble for sub-model selection and optimization of ensemble weights for different datasets. The experiments were conducted on the public datasets CVC-ClinicDB, Kvasir, CVC-ColonDB, ETIS-LaribPolypDB and PolypGen. The results show that the DET-Former, constructed based on the Multi-Head Control Ensemble and the SDBH-PSO Ensemble, consistently provides improved accuracy across different datasets. Among them, the Multi-Head Control Ensemble demonstrated superior feature fusion capability in the experiments, and the SDBH-PSO Ensemble demonstrated excellent sub-model selection capability. The sub-model selection capabilities of the SDBH-PSO Ensemble will continue to have significant reference value and practical utility as deep learning networks evolve.

The self-distillation trained multitask dense-attention network for diagnosing lung cancers based on CT scans

BACKGROUND

The latest international multidisciplinary histopathological classification of lung cancer indicates that a deeper study of the lung adenocarcinoma requires a comprehensive multidisciplinary platform. However, in the traditional pathological examination or previous computer-vision-based research, the entire lung is not considered in a comprehensive manner.

PURPOSE

The study aims to develop a deep learning model proposed for diagnosing the lung adenocarcinoma histopathologically based on CT scans. Instead of just classifying the lung adenocarcinoma, the pathological report should be inferred based on both the invasiveness and growth pattern of the tumors.

METHODS

A self-distillation trained multitask dense-attention network (SD-MdaNet) is proposed and validated based on 2412 labeled CT scans from 476 patients and 845 unlabeled scans. Inferring the pathological report is divided into two tasks, predicting the invasiveness of the lung tumor and inferring growth patterns of tumor cells in a comprehensive histopathological subtyping manner with excellent accuracy. In the proposed method, the dense-attention module is introduced to better extract features from a small dataset in the main branch of the MdaNet. Next, task-specific attention modules are utilized in different branches and finally integrated as a multitask model. The second task is a blend of classification and regression tasks. Thus, a specialized loss function is developed. In the proposed knowledge distillation (KD) process, the MdaNet as well as its main branch trained for solving two single tasks, respectively, are treated as multiple teachers to produce a student model. A novel KD loss function is developed to take the advantage of all the models as well as data with labels and without labels.

RESULTS

SD-MdaNet achieves an AUC of98.7 ± 0.4 % $98.7\pm 0.4\%$ on invasiveness prediction, and91.6 ± 1.0 % $91.6\pm 1.0\%$ on predominant growth pattern prediction on our dataset. Moreover, the average mean squared error in inferring growth pattern proportion reaches0.0217 ± 0.0019 $0.0217\pm 0.0019$ , and the AUC for predominant growth pattern proportion reaches91.6 ± 1.0 % $91.6\pm 1.0\%$ . The proposed SD-MdaNet is significantly better than all other benchmarking methods (F D R < 0.05 $FDR<0.05$ ).

CONCLUSIONS

Experimental results demonstrate that the proposed SD-MdaNet can significantly improve the performance of the lung adenocarcinoma pathological diagnosis using only CT scans. Analyses and discussions are conducted to interpret the advantages of the SD-MdaNet.

AI/ML advances in non-small cell lung cancer biomarker discovery

Lung cancer is the leading cause of cancer deaths among both men and women, representing approximately 25% of cancer fatalities each year. The treatment landscape for non-small cell lung cancer (NSCLC) is rapidly evolving due to the progress made in biomarker-driven targeted therapies. While advancements in targeted treatments have improved survival rates for NSCLC patients with actionable biomarkers, long-term survival remains low, with an overall 5-year relative survival rate below 20%. Artificial intelligence/machine learning (AI/ML) algorithms have shown promise in biomarker discovery, yet NSCLC-specific studies capturing the clinical challenges targeted and emerging patterns identified using AI/ML approaches are lacking. Here, we employed a text-mining approach and identified 215 studies that reported potential biomarkers of NSCLC using AI/ML algorithms. We catalogued these studies with respect to BEST (Biomarkers, EndpointS, and other Tools) biomarker sub-types and summarized emerging patterns and trends in AI/ML-driven NSCLC biomarker discovery. We anticipate that our comprehensive review will contribute to the current understanding of AI/ML advances in NSCLC biomarker research and provide an important catalogue that may facilitate clinical adoption of AI/ML-derived biomarkers.

A radio-pathologic integrated model for prediction of lymph node metastasis stage in patients with gastric cancer

BACKGROUND

Accurate prediction of lymph node metastasis stage (LNMs) facilitates precision therapy for gastric cancer. We aimed to develop and validate a deep learning-based radio-pathologic model to predict the LNM stage in patients with gastric cancer by integrating CT images and histopathological whole-slide images (WSIs).

METHODS

A total of 252 patients were enrolled and randomly divided into a training set (n = 202) and a testing set (n = 50). Both pretreatment contrast-enhanced abdominal CT and WSI of biopsy specimens were collected for each patient. The deep radiologic and pathologic features were extracted from CT and WSI using ResNet-50 and Vision Transformer (ViT) network, respectively. By fusing both radiologic and pathologic features, a radio-pathologic integrated model was constructed to predict the five LNM stages. For comparison, four single-modality models using CT images or WSIs were also constructed, respectively. All models were trained on the training set and validated on the testing set.

RESULTS

The radio-pathologic integrated mode achieved an overall accuracy of 84.0% and a kappa coefficient of 0.795 on the testing set. The areas under the curves (AUCs) of the integrated model in predicting the five LNM stages were 0.978 (95% Confidence Interval (CI 0.917-1.000), 0.946 (95% CI 0.867-1.000), 0.890 (95% CI 0.718-1.000), 0.971 (95% CI 0.920-1.000), and 0.982 (95% CI 0.911-1.000), respectively. Moreover, the integrated model achieved an AUC of 0.978 (95% CI 0.912-1.000) in predicting the binary status of nodal metastasis.

CONCLUSION

Our study suggests that radio-pathologic integrated model that combined both macroscale radiologic image and microscale pathologic image can better predict lymph node metastasis stage in patients with gastric cancer.

An anatomy-aware framework for automatic segmentation of parotid tumor from multimodal MRI

Magnetic Resonance Imaging (MRI) plays an important role in diagnosing the parotid tumor, where accurate segmentation of tumors is highly desired for determining appropriate treatment plans and avoiding unnecessary surgery. However, the task remains nontrivial and challenging due to ambiguous boundaries and various sizes of the tumor, as well as the presence of a large number of anatomical structures around the parotid gland that are similar to the tumor. To overcome these problems, we propose a novel anatomy-aware framework for automatic segmentation of parotid tumors from multimodal MRI. First, a Transformer-based multimodal fusion network PT-Net is proposed in this paper. The encoder of PT-Net extracts and fuses contextual information from three modalities of MRI from coarse to fine, to obtain cross-modality and multi-scale tumor information. The decoder stacks the feature maps of different modalities and calibrates the multimodal information using the channel attention mechanism. Second, considering that the segmentation model is prone to be disturbed by similar anatomical structures and make wrong predictions, we design anatomy-aware loss. By calculating the distance between the activation regions of the prediction segmentation and the ground truth, our loss function forces the model to distinguish similar anatomical structures with the tumor and make correct predictions. Extensive experiments with MRI scans of the parotid tumor showed that our PT-Net achieved higher segmentation accuracy than existing networks. The anatomy-aware loss outperformed state-of-the-art loss functions for parotid tumor segmentation. Our framework can potentially improve the quality of preoperative diagnosis and surgery planning of parotid tumors.

A Multi-Modal Heterogeneous Graph Forest to Predict Lymph Node Metastasis of Non-Small Cell Lung Cancer

Lymph node metastasis (LNM) is critical for treatment decision-making for cancer patients, but it is difficult to diagnose accurately before surgery. Machine learning can learn nontrivial knowledge from multi-modal data to support accurate diagnosis. In this paper, we proposed a Multi-modal Heterogeneous Graph Forest (MHGF) approach to extract the deep representations of LNM from multi-modal data. Specifically, we first extracted the deep image features from CT images to represent the pathological anatomic extent of the primary tumor (pathological T stage) using a ResNet-Trans network. And then, a heterogeneous graph with six vertices and seven bi-directional relations was defined by medical experts to describe the possible relations between the clinical and image features. After that, we proposed a graph forest approach to construct the sub-graphs by removing each vertex in the complete graph iteratively. Finally, we used graph neural networks to learn the representations of each sub-graph in the forest to predict LNM and averaged all the prediction results as final results. We conducted experiments on 681 patients' multi-modal data. The proposed MHGF achieves the best performances with a 0.806 AUC value and 0.513 AP value compared with state-of-art machine learning and deep learning methods. The results indicate that the graph method can explore the relations between different types of features to learn effective deep representations for LNM prediction. Moreover, we found that the deep image features about the pathological anatomic extent of the primary tumor are useful for LNM prediction. And the graph forest approach can further improve the generalization ability and stability of the LNM prediction model.

Artificial Intelligence and the diagnosis of lung cancer in early stage: scoping review. (Preprint).. [DOI: 10.2196/preprints.38773] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

BACKGROUND

Lung cancer is considered to be the most fatal out of all diagnoseable cancers. This is, in part, due to the difficulty in detecting lung cancer at an early stage. Moreover, approximately one in five individuals who will develop lung cancer will pass away due to a misdiagnosis. Fortunately, Machine Learning (ML) and Deep Learning (DL) is considered to be a promising solution for detection of lung cancer through developments in radiology.

OBJECTIVE

The purpose of this paper is to is to review how AI can assist identifying and diagnosing of lung cancer in an early stage.

METHODS

PRISMA was utilized and were retrieved from 4 databases: Google Scholar, PubMed, EMBASE, and Institute of Electrical and Electronics Engineers (IEEE). In addition, two phases of screening were implemented in order to determine relevant literature. The first phase was reading the title and abstract, and the second stage was reading the full text. These two steps were independently conducted by three reviewers. Finally, the three authors use a narrative synthesis to present the data.

RESULTS

Overall, 543 potential studies were extracted from four databases. After screening, 26 articles that met the inclusion criteria were included in this scoping review. Several articles utilized privet data including patients’ data and other public sources. 15 articles used data from UCI repository dataset (58%). However, CT scan images was utilized on 9 studies (normal CT was mentioned in 5 articles (19%), two studies used CT scan with PET (7.7%), and two articles used FDG with CT (7.7%). While two articles used demographic data such as age, sex, and educational background (7.7%).

CONCLUSIONS

This scoping review illustrates recent studies that utilize AI models to diagnose lung cancer. The literature currently relies on private and public databases and compare models with physicians or other machine learning technology. Additional studies should be conducted to explore the efficacy of these technologies in clinical settings.