Determining the invasiveness of ground-glass nodules using a 3D multi-task network

New Perspectives on Lung Cancer Screening and Artificial Intelligence

Lung cancer is the leading cause of cancer-related death worldwide, with 1.8 million deaths annually. Early detection is vital for improving patient outcomes; however, survival rates remain low due to late-stage diagnoses. Accumulating data supports the idea that screening methods are useful for improving early diagnosis in high-risk patients. However, several barriers limit the application of lung cancer screening in real-world settings. The widespread diffusion of artificial intelligence (AI), radiomics, and machine learning has dramatically changed the current diagnostic landscape. This review explores the potential of AI and biomarker-driven methods, particularly liquid biopsy, in enhancing early lung cancer detection. We report the findings of major randomized controlled trials, cohort studies, and research on AI algorithms that use multi-modal imaging (e.g., CT and PET scans) and liquid biopsy to identify early molecular alterations. AI algorithms enhance diagnostic accuracy by automating image analysis and reducing inter-reader variability. Biomarker-driven methods identify molecular alterations in patients before imaging signs of cancer are evident. Both AI and liquid biopsy show the potential to improve sensitivity and specificity, enabling the detection of early-stage cancers that traditional methods, like low-dose CT (LDCT) scans, might miss. Integrating AI and biomarker-driven methods offers significant promise for transforming lung cancer screening. These technologies could enable earlier, more accurate detection, ultimately improving survival outcomes. AI-driven lung cancer screening can achieve over 90% sensitivity, compared to 70-80% with traditional methods, and can reduce false positives by up to 30%. AI also boosts specificity to 85-90%, with faster processing times (a few minutes vs. 30-60 min for radiologists). However, challenges remain in standardizing these approaches and integrating them into clinical practice. Ongoing research is essential to fully realize their clinical benefits and enhance timely interventions.

Predicting malignant risk of ground-glass nodules using convolutional neural networks based on dual-time-point 18F-FDG PET/CT

BACKGROUND

Accurately predicting the malignant risk of ground-glass nodules (GGOs) is crucial for precise treatment planning. This study aims to utilize convolutional neural networks based on dual-time-point 18F-FDG PET/CT to predict the malignant risk of GGOs.

METHODS

Retrospectively analyzing 311 patients with 397 GGOs, this study identified 118 low-risk GGOs and 279 high-risk GGOs through pathology and follow-up according to the new WHO classification. The dataset was randomly divided into a training set comprising 239 patients (318 lesions) and a testing set comprising 72 patients (79 lesions), we employed a self-configuring 3D nnU-net convolutional neural network with majority voting method to segment GGOs and predict malignant risk of GGOs. Three independent segmentation prediction models were developed based on thin-section lung CT, early-phase 18F-FDG PET/CT, and dual-time-point 18F-FDG PET/CT, respectively. Simultaneously, the results of the dual-time-point 18F-FDG PET/CT model on the testing set were compared with the diagnostic of nuclear medicine physicians.

RESULTS

The dual-time-point 18F-FDG PET/CT model achieving a Dice coefficient of 0.84 ± 0.02 for GGOs segmentation and demonstrating high accuracy (84.81%), specificity (84.62%), sensitivity (84.91%), and AUC (0.85) in predicting malignant risk. The accuracy of the thin-section CT model is 73.42%, and the accuracy of the early-phase 18F-FDG PET/CT model is 78.48%, both of which are lower than the accuracy of the dual-time-point 18F-FDG PET/CT model. The diagnostic accuracy for resident, junior and expert physicians were 67.09%, 74.68%, and 78.48%, respectively. The accuracy (84.81%) of the dual-time-point 18F-FDG PET/CT model was significantly higher than that of nuclear medicine physicians.

CONCLUSIONS

Based on dual-time-point 18F-FDG PET/CT images, the 3D nnU-net with a majority voting method, demonstrates excellent performance in predicting the malignant risk of GGOs. This methodology serves as a valuable adjunct for physicians in the risk prediction and assessment of GGOs.

Determining the status of tertiary lymphoid structures in invasive pulmonary adenocarcinoma based on chest CT radiomic features

OBJECTIVES

The aim of this study was to determine the status of tertiary lymphoid structures (TLSs) using radiomic features in patients with invasive pulmonary adenocarcinoma (IA).

METHODS

In this retrospective study, patients with IA from November 2015 to March 2024 were recruited from two independent centers (center 1, training and internal test data set; center 2, external test data set). TLS was divided into two groups according to hematoxylin-eosin staining. Radiomic features were extracted, and support vector machine (SVM) were implemented to predict the status of TLSs. Receiver operating characteristic (ROC) curves were used to analyze diagnostic performance. Furthermore, visual assessments of the test set were also conducted by two thoracic radiologists and compared with the radiomics results.

RESULTS

A total of 456 patients were included (training data set, n = 278; internal test data set, n = 115; external test data set, n = 63). The area under the curve (AUC) of the radiomics model on the validation set, the internal test set, and the external test set were 0.781 (95% confidence interval (CI): 0.659-0.905;), 0.804 (95% CI: 0.723-0.884;) and 0.747 (95% CI: 0.621-0.874;), respectively. In the visual assessments, the mean CT value and air bronchogram were important indicators of TLS, the AUC was 0.683. In the external test set, the AUC of the clinical model was 0.632.

CONCLUSIONS

The radiomics model has a higher AUC than the clinical model and effectively discriminates TLSs in patients with IA.

CRITICAL RELEVANCE STATEMENT

This study demonstrates that the radiomics-based model can differentiate TLSs in patients with IA. As a non-invasive biomarker, it enhances our understanding of tumor prognosis and management.

KEY POINTS

TLSs are closely related to favorable clinical outcomes in non-small cell lung cancer. Radiomics from Chest CT predicted TLSs in patients with IA. This study supports individualized clinical decision-making for patients with IA.

Res-TransNet: A Hybrid deep Learning Network for Predicting Pathological Subtypes of lung Adenocarcinoma in CT Images

This study aims to develop a CT-based hybrid deep learning network to predict pathological subtypes of early-stage lung adenocarcinoma by integrating residual network (ResNet) with Vision Transformer (ViT). A total of 1411 pathologically confirmed ground-glass nodules (GGNs) retrospectively collected from two centers were used as internal and external validation sets for model development. 3D ResNet and ViT were applied to investigate two deep learning frameworks to classify three subtypes of lung adenocarcinoma namely invasive adenocarcinoma (IAC), minimally invasive adenocarcinoma and adenocarcinoma in situ, respectively. To further improve the model performance, four Res-TransNet based models were proposed by integrating ResNet and ViT with different ensemble learning strategies. Two classification tasks involving predicting IAC from Non-IAC (Task1) and classifying three subtypes (Task2) were designed and conducted in this study. For Task 1, the optimal Res-TransNet model yielded area under the receiver operating characteristic curve (AUC) values of 0.986 and 0.933 on internal and external validation sets, which were significantly higher than that of ResNet and ViT models (p < 0.05). For Task 2, the optimal fusion model generated the accuracy and weighted F1 score of 68.3% and 66.1% on the external validation set. The experimental results demonstrate that Res-TransNet can significantly increase the classification performance compared with the two basic models and have the potential to assist radiologists in precision diagnosis.

Evaluation of the invasiveness of pure ground-glass nodules based on dual-head ResNet technique

OBJECTIVE

To intelligently evaluate the invasiveness of pure ground-glass nodules with multiple classifications using deep learning.

METHODS

pGGNs in 1136 patients were pathologically confirmed as lung precursor lesions [atypical adenomatous hyperplasia (AAH) and adenocarcinoma in situ (AIS)], minimally invasive adenocarcinoma (MIA), or invasive adenocarcinoma (IAC). Four different models [EfficientNet-b0 2D, dual-head ResNet_3D, a 3D model combining three features (3D_3F), and a 3D model combining 19 features (3D_19F)] were constructed to evaluate the invasiveness of pGGNs using the EfficientNet and ResNet networks. The Obuchowski index was used to evaluate the differences in diagnostic efficiency among the four models.

RESULTS

The patients with pGGNs (360 men, 776 women; mean age, 54.63 ± 12.36 years) included 235 cases of AAH + AIS, 332 cases of MIA, and 569 cases of IAC. In the validation group, the areas under the curve in detecting the invasiveness of pGGNs as a three-category classification (AAH + AIS, MIA, IAC) were 0.8008, 0.8090, 0.8165, and 0.8158 for EfficientNet-b0 2D, dual-head ResNet_3D, 3D_3F, and 3D_19F, respectively, whereas the accuracies were 0.6422, 0.6158, 0.651, and 0.6364, respectively. The Obuchowski index revealed no significant differences in the diagnostic performance of the four models.

CONCLUSIONS

The dual-head ResNet_3D_3F model had the highest diagnostic efficiency for evaluating the invasiveness of pGGNs in the four models.

Predicting Invasiveness of Lung Adenocarcinoma at Chest CT with Deep Learning Ternary Classification Models

Background Preoperative discrimination of preinvasive, minimally invasive, and invasive adenocarcinoma at CT informs clinical management decisions but may be challenging for classifying pure ground-glass nodules (pGGNs). Deep learning (DL) may improve ternary classification. Purpose To determine whether a strategy that includes an adjudication approach can enhance the performance of DL ternary classification models in predicting the invasiveness of adenocarcinoma at chest CT and maintain performance in classifying pGGNs. Materials and Methods In this retrospective study, six ternary models for classifying preinvasive, minimally invasive, and invasive adenocarcinoma were developed using a multicenter data set of lung nodules. The DL-based models were progressively modified through framework optimization, joint learning, and an adjudication strategy (simulating a multireader approach to resolving discordant nodule classifications), integrating two binary classification models with a ternary classification model to resolve discordant classifications sequentially. The six ternary models were then tested on an external data set of pGGNs imaged between December 2019 and January 2021. Diagnostic performance including accuracy, specificity, and sensitivity was assessed. The χ2 test was used to compare model performance in different subgroups stratified by clinical confounders. Results A total of 4929 nodules from 4483 patients (mean age, 50.1 years ± 9.5 [SD]; 2806 female) were divided into training (n = 3384), validation (n = 579), and internal (n = 966) test sets. A total of 361 pGGNs from 281 patients (mean age, 55.2 years ± 11.1 [SD]; 186 female) formed the external test set. The proposed strategy improved DL model performance in external testing (P < .001). For classifying minimally invasive adenocarcinoma, the accuracy was 85% and 79%, sensitivity was 75% and 63%, and specificity was 89% and 85% for the model with adjudication (model 6) and the model without (model 3), respectively. Model 6 showed a relatively narrow range (maximum minus minimum) across diagnostic indexes (accuracy, 1.7%; sensitivity, 7.3%; specificity, 0.9%) compared with the other models (accuracy, 0.6%-10.8%; sensitivity, 14%-39.1%; specificity, 5.5%-17.9%). Conclusion Combining framework optimization, joint learning, and an adjudication approach improved DL classification of adenocarcinoma invasiveness at chest CT. Published under a CC BY 4.0 license. Supplemental material is available for this article. See also the editorial by Sohn and Fields in this issue.

An ensemble deep learning model for risk stratification of invasive lung adenocarcinoma using thin-slice CT

Lung cancer screening using computed tomography (CT) has increased the detection rate of small pulmonary nodules and early-stage lung adenocarcinoma. It would be clinically meaningful to accurate assessment of the nodule histology by CT scans with advanced deep learning algorithms. However, recent studies mainly focus on predicting benign and malignant nodules, lacking of model for the risk stratification of invasive adenocarcinoma. We propose an ensemble multi-view 3D convolutional neural network (EMV-3D-CNN) model to study the risk stratification of lung adenocarcinoma. We include 1075 lung nodules (≤30 mm and ≥4 mm) with preoperative thin-section CT scans and definite pathology confirmed by surgery. Our model achieves a state-of-art performance of 91.3% and 92.9% AUC for diagnosis of benign/malignant and pre-invasive/invasive nodules, respectively. Importantly, our model outperforms senior doctors in risk stratification of invasive adenocarcinoma with 77.6% accuracy [i.e., Grades 1, 2, 3]). It provides detailed predictive histological information for the surgical management of pulmonary nodules. Finally, for user-friendly access, the proposed model is implemented as a web-based system ( https://seeyourlung.com.cn ).

Multi-task deep learning for medical image computing and analysis: A review

The renaissance of deep learning has provided promising solutions to various tasks. While conventional deep learning models are constructed for a single specific task, multi-task deep learning (MTDL) that is capable to simultaneously accomplish at least two tasks has attracted research attention. MTDL is a joint learning paradigm that harnesses the inherent correlation of multiple related tasks to achieve reciprocal benefits in improving performance, enhancing generalizability, and reducing the overall computational cost. This review focuses on the advanced applications of MTDL for medical image computing and analysis. We first summarize four popular MTDL network architectures (i.e., cascaded, parallel, interacted, and hybrid). Then, we review the representative MTDL-based networks for eight application areas, including the brain, eye, chest, cardiac, abdomen, musculoskeletal, pathology, and other human body regions. While MTDL-based medical image processing has been flourishing and demonstrating outstanding performance in many tasks, in the meanwhile, there are performance gaps in some tasks, and accordingly we perceive the open challenges and the perspective trends. For instance, in the 2018 Ischemic Stroke Lesion Segmentation challenge, the reported top dice score of 0.51 and top recall of 0.55 achieved by the cascaded MTDL model indicate further research efforts in high demand to escalate the performance of current models.

A deep learning approach for anterior cruciate ligament rupture localization on knee MR images

Purpose: To develop and evaluate a deep learning-based method to localize and classify anterior cruciate ligament (ACL) ruptures on knee MR images by using arthroscopy as the reference standard.

Methods: We proposed a fully automated ACL rupture localization system to localize and classify ACL ruptures. The classification of ACL ruptures was based on the projection coordinates of the ACL rupture point on the line connecting the center coordinates of the femoral and tibial footprints. The line was divided into three equal parts and the position of the projection coordinates indicated the classification of the ACL ruptures (femoral side, middle and tibial side). In total, 85 patients (mean age: 27; male: 56) who underwent ACL reconstruction surgery under arthroscopy were included. Three clinical readers evaluated the datasets separately and their diagnostic performances were compared with those of the model. The performance metrics included the accuracy, error rate, sensitivity, specificity, precision, and F1-score. A one-way ANOVA was used to evaluate the performance of the convolutional neural networks (CNNs) and clinical readers. Intraclass correlation coefficients (ICC) were used to assess interobserver agreement between the clinical readers.

Results: The accuracy of ACL localization was 3.77 ± 2.74 and 4.68 ± 3.92 (mm) for three-dimensional (3D) and two-dimensional (2D) CNNs, respectively. There was no significant difference in the ACL rupture location performance between the 3D and 2D CNNs or among the clinical readers (Accuracy, p < 0.01). The 3D CNNs performed best among the five evaluators in classifying the femoral side (sensitivity of 0.86 and specificity of 0.79), middle side (sensitivity of 0.71 and specificity of 0.84) and tibial side ACL rupture (sensitivity of 0.71 and specificity of 0.99), and the overall accuracy for sides classifying of ACL rupture achieved 0.79.

Conclusion: The proposed deep learning-based model achieved high diagnostic performances in locating and classifying ACL fractures on knee MR images.

Clinical impact of a deep learning system for automated detection of missed pulmonary nodules on routine body computed tomography including the chest region

OBJECTIVES

To evaluate the clinical impact of a deep learning system (DLS) for automated detection of pulmonary nodules on computed tomography (CT) images as a second reader.

METHODS

This single-centre retrospective study screened 21,150 consecutive body CT studies from September 2018 to February 2019. Pulmonary nodules detected by the DLS on axial CT images but not mentioned in initial radiology reports were flagged. Flagged images were scored by four board-certificated radiologists each with at least 5 years of experience. Nodules with scores of 2 (understandable miss) or 3 (should not be missed) were then categorised as unlikely to be clinically significant (2a or 3a) or likely to be clinically significant (2b or 3b) according to the 2017 Fleischner guidelines for pulmonary nodules. The miss rate was defined as the total number of studies receiving scores of 2 or 3 divided by total screened studies.

RESULTS

Among 172 nodules flagged by the DLS, 60 (35%) missed nodules were confirmed by the radiologists. The nodules were further categorised as 2a, 2b, 3a, and 3b in 24, 14, 10, and 12 studies, respectively, with an overall positive predictive value of 35%. Missed pulmonary nodules were identified in 0.3% of all CT images, and one-third of these lesions were considered clinically significant.

CONCLUSIONS

Use of DLS-assisted automated detection as a second reader can identify missed pulmonary nodules, some of which may be clinically significant.

CLINICAL RELEVANCE/APPLICATION

Use of DLS to help radiologists detect pulmonary lesions may improve patient care.

KEY POINTS

• DLS-assisted automated detection as a second reader is feasible in a large consecutive cohort. • Performance of combined radiologists and DLS was better than DLS or radiologists alone. • Pulmonary nodules were missed more frequently in abdomino-pelvis CT than the thoracic CT.

[Clinical Study of Artificial Intelligence-assisted Diagnosis System in Predicting the Invasive Subtypes of Early-stage Lung Adenocarcinoma Appearing as Pulmonary Nodules]

BACKGROUND

Lung cancer is the cancer with the highest mortality at home and abroad at present. The detection of lung nodules is a key step to reducing the mortality of lung cancer. Artificial intelligence-assisted diagnosis system presents as the state of the art in the area of nodule detection, differentiation between benign and malignant and diagnosis of invasive subtypes, however, a validation with clinical data is necessary for further application. Therefore, the aim of this study is to evaluate the effectiveness of artificial intelligence-assisted diagnosis system in predicting the invasive subtypes of early‑stage lung adenocarcinoma appearing as pulmonary nodules.

METHODS

Clinical data of 223 patients with early-stage lung adenocarcinoma appearing as pulmonary nodules admitted to the Lanzhou University Second Hospital from January 1st, 2016 to December 31th, 2021 were retrospectively analyzed, which were divided into invasive adenocarcinoma group (n=170) and non-invasive adenocarcinoma group (n=53), and the non-invasive adenocarcinoma group was subdivided into minimally invasive adenocarcinoma group (n=31) and preinvasive lesions group (n=22). The malignant probability and imaging characteristics of each group were compared to analyze their predictive ability for the invasive subtypes of early-stage lung adenocarcinoma. The concordance between qualitative diagnostic results of artificial intelligence-assisted diagnosis of the invasive subtypes of early-stage lung adenocarcinoma and postoperative pathology was then analyzed.

RESULTS

In different invasive subtypes of early-stage lung adenocarcinoma, the mean CT value of pulmonary nodules (P<0.001), diameter (P<0.001), volume (P<0.001), malignant probability (P<0.001), pleural retraction sign (P<0.001), lobulation (P<0.001), spiculation (P<0.001) were significantly different. At the same time, it was also found that with the increased invasiveness of different invasive subtypes of early-stage lung adenocarcinoma, the proportion of dominant signs of each group gradually increased. On the issue of binary classification, the sensitivity, specificity, and area under the curve (AUC) values of the artificial intelligence-assisted diagnosis system for the qualitative diagnosis of invasive subtypes of early-stage lung adenocarcinoma were 81.76%, 92.45% and 0.871 respectively. On the issue of three classification, the accuracy, recall rate, F1 score, and AUC values of the artificial intelligence-assisted diagnosis system for the qualitative diagnosis of invasive subtypes of early-stage lung adenocarcinoma were 83.86%, 85.03%, 76.46% and 0.879 respectively.

CONCLUSIONS

Artificial intelligence-assisted diagnosis system could predict the invasive subtypes of early‑stage lung adenocarcinoma appearing as pulmonary nodules, and has a certain predictive value. With the optimization of algorithms and the improvement of data, it may provide guidance for individualized treatment of patients.

[Chinese Experts Consensus on Artificial Intelligence Assisted Management for Pulmonary Nodule (2022 Version)]

Low-dose computed tomography (CT) for lung cancer screening has been proven to reduce lung cancer deaths in the screening group compared with the control group. The increasing number of pulmonary nodules being detected by CT scans significantly increase the workload of the radiologists for scan interpretation. Artificial intelligence (AI) has the potential to increase the efficiency of pulmonary nodule discrimination and has been tested in preliminary studies for nodule management. As more and more artificial AI products are commercialized, the consensus statement has been organized in a collaborative effort by Thoracic Surgery Committee, Department of Simulated Medicine, Wu Jieping Medical Foundation to aid clinicians in the application of AI-assisted management for pulmonary nodules.
.

IMAL-Net: Interpretable multi-task attention learning network for invasive lung adenocarcinoma screening in CT images

PURPOSE

Feature maps created from deep convolutional neural networks (DCNNs) have been widely used for visual explanation of DCNN-based classification tasks. However, many clinical applications such as benign-malignant classification of lung nodules normally require quantitative and objective interpretability, rather than just visualization. In this paper, we propose a novel interpretable multi-task attention learning network named IMAL-Net for early invasive adenocarcinoma screening in chest computed tomography images, which takes advantage of segmentation prior to assist interpretable classification.

METHODS

Two sub-ResNets are firstly integrated together via a prior-attention mechanism for simultaneous nodule segmentation and invasiveness classification. Then, numerous radiomic features from the segmentation results are concatenated with high-level semantic features from the classification subnetwork by FC layers to achieve superior performance. Meanwhile, an end-to-end feature selection mechanism (named FSM) is designed to quantify crucial radiomic features greatly affecting the prediction of each sample, and thus it can provide clinically applicable interpretability to the prediction result.

RESULTS

Nodule samples from a total of 1626 patients were collected from two grade-A hospitals for large-scale verification. Five-fold cross validation demonstrated that the proposed IMAL-Net can achieve an AUC score of 93.8% ± 1.1% and a recall score of 93.8% ± 2.8% for identification of invasive lung adenocarcinoma.

CONCLUSIONS

It can be concluded that fusing semantic features and radiomic features can achieve obvious improvements in the invasiveness classification task. Moreover, by learning more fine-grained semantic features and highlighting the most important radiomics features, the proposed attention and FSM mechanisms not only can further improve the performance but also can be used for both visual explanations and objective analysis of the classification results.

Consecutive Serial Non-Contrast CT Scan-Based Deep Learning Model Facilitates the Prediction of Tumor Invasiveness of Ground-Glass Nodules

Introduction

Tumors are continuously evolving biological systems which can be monitored by medical imaging. Previous studies only focus on single timepoint images, whether the performance could be further improved by using serial noncontrast CT imaging obtained during nodule follow-up management remains unclear. In this study, we evaluated DL model for predicting tumor invasiveness of GGNs through analyzing time series CT images.

Methods

A total of 168 pathologically confirmed GGN cases (48 noninvasive lesions and 120 invasive lesions) were retrospectively collected and randomly assigned to the development dataset (n = 123) and independent testing dataset (n = 45). All patients underwent consecutive noncontrast CT examinations, and the baseline CT and 3-month follow-up CT images were collected. The gross region of interest (ROI) patches containing only tumor region and the full ROI patches including both tumor and peritumor regions were cropped from CT images. A baseline model was built on the image features and demographic features. Four DL models were proposed: two single-DL model using gross ROI (model 1) or full ROI patches (model 3) from baseline CT images, and two serial-DL models using gross ROI (model 2) or full ROI patches (model 4) from consecutive CT images (baseline scan and 3-month follow-up scan). In addition, a combined model integrating serial full ROI patches and clinical information was also constructed. The performance of these predictive models was assessed with respect to discrimination and clinical usefulness.

Results

The area under the curve (AUC) of the baseline model, models 1, 2, 3, and 4 were 0.562 [(95% confidence interval (C)], 0.406~0.710), 0.693 (95% CI, 0.538-0.822), 0.787 (95% CI, 0.639-0.895), 0.727 (95% CI, 0.573-0.849), and 0.811 (95% CI, 0.667-0.912) in the independent testing dataset, respectively. The results indicated that the peritumor region had potential to contribute to tumor invasiveness prediction, and the model performance was further improved by integrating imaging scans at multiple timepoints. Furthermore, the combined model showed best discrimination ability, with AUC, sensitivity, specificity, and accuracy achieving 0.831 (95% CI, 0.690-0.926), 86.7%, 73.3%, and 82.2%, respectively.

Conclusion

The DL model integrating full ROIs from serial CT images shows improved predictive performance in differentiating noninvasive from invasive GGNs than the model using only baseline CT images, which could benefit the clinical management of GGNs.