Liver lesion changes analysis in longitudinal CECT scans by simultaneous deep learning voxel classification with SimU-Net

Metastatic Lung Lesion Changes in Follow-up Chest CT: The Advantage of Deep Learning Simultaneous Analysis of Prior and Current Scans With SimU-Net

PURPOSE

Radiological follow-up of oncology patients requires the detection of metastatic lung lesions and the quantitative analysis of their changes in longitudinal imaging studies. Our aim was to evaluate SimU-Net, a novel deep learning method for the automatic analysis of metastatic lung lesions and their temporal changes in pairs of chest CT scans.

MATERIALS AND METHODS

SimU-Net is a simultaneous multichannel 3D U-Net model trained on pairs of registered prior and current scans of a patient. It is part of a fully automatic pipeline for the detection, segmentation, matching, and classification of metastatic lung lesions in longitudinal chest CT scans. A data set of 5040 metastatic lung lesions in 344 pairs of 208 prior and current chest CT scans from 79 patients was used for training/validation (173 scans, 65 patients) and testing (35 scans, 14 patients) of a standalone 3D U-Net models and 3 simultaneous SimU-Net models. Outcome measures were the lesion detection and segmentation precision, recall, Dice score, average symmetric surface distance (ASSD), lesion matching, and classification of lesion changes from computed versus manual ground-truth annotations by an expert radiologist.

RESULTS

SimU-Net achieved a mean lesion detection recall and precision of 0.93±0.13 and 0.79±0.24 and a mean lesion segmentation Dice and ASSD of 0.84±0.09 and 0.33±0.22 mm. These results outperformed the standalone 3D U-Net model by 9.4% in the recall, 2.4% in Dice, and 15.4% in ASSD, with a minor 3.6% decrease in precision. The SimU-Net pipeline achieved perfect precision and recall (1.0±0.0) for lesion matching and classification of lesion changes.

CONCLUSIONS

Simultaneous deep learning analysis of metastatic lung lesions in prior and current chest CT scans with SimU-Net yields superior accuracy compared with individual analysis of each scan. Implementation of SimU-Net in the radiological workflow may enhance efficiency by automatically computing key metrics used to evaluate metastatic lung lesions and their temporal changes.

A graph-theoretic approach for the analysis of lesion changes and lesions detection review in longitudinal oncological imaging

Radiological follow-up of oncology patients requires the detection of lesions and the quantitative analysis of lesion changes in longitudinal imaging studies of patients, which is time-consuming and requires expertise. We present here a new method and workflow for the analysis and review of lesions and volumetric lesion changes in longitudinal scans of a patient. The generic graph-based method consists of lesion matching, classification of changes in individual lesions, and detection of patterns of lesion changes computed from the properties of the graph and its connected components. The workflow guides clinicians in the detection of missed lesions and wrongly identified lesions in manual and computed lesion annotations using the analysis of lesion changes. It serves as a heuristic method for the automatic revision of ground truth lesion annotations in longitudinal scans. The methods were evaluated on longitudinal studies of patients with three or more examinations of metastatic lesions in the lung (19 patients, 83 CT scans, 1178 lesions), the liver (18 patients, 77 CECT scans, 800 lesions) and the brain (30 patients, 102 T1W-Gad MRI scans, 317 lesions) with ground-truth lesion annotations. Lesion matching yielded a precision of 0.92-1.0 and recall of 0.91-0.99. The classification of changes in individual lesions yielded an accuracy of 0.87-0.97. The classification of patterns of lesion changes yielded an accuracy of 0.80-0.94. The lesion detection review workflow applied to manual and computed lesion annotations yielded 120 and 55 missed lesions and 20 and 164 wrongly identified lesions for all longitudinal studies of patients, respectively. The automatic analysis of lesion changes and review of lesion detection in longitudinal studies of oncological patients helps detect missed lesions and wrongly identified lesions. This method may help improve the accuracy of radiological interpretation and the disease status evaluation.

Three scans are better than two for follow-up: An automatic method for finding missed and misidentified lesions in cross-sectional follow-up of oncology patients

PURPOSE

Missed and misidentified neoplastic lesions in longitudinal studies of oncology patients are pervasive and may affect the evaluation of the disease status. Two newly identified patterns of lesion changes, lone lesions and non-consecutive lesion changes, may help radiologists to detect these lesions. This study evaluated a new interpretation revision workflow of lesion annotations in three or more consecutive scans based on these suspicious patterns.

METHODS

The interpretation revision workflow was evaluated on manual and computed lesion annotations in longitudinal oncology patient studies. For the manual revision, a senior radiologist and a senior neurosurgeon (the readers) manually annotated the lesions in each scan and later revised their annotations to identify missed and misidentified lesions with the workflow using the automatically detected patterns. For the computerized revision, lesion annotations were first computed with a previously trained nnU-Net and were then automatically revised with an AI-based method that automates the workflow readers' decisions. The evaluation included 67 patient studies with 2295 metastatic lesions in lung (19 patients, 83 CT scans, 1178 lesions), liver (18 patients, 77 CECT scans, 800 lesions) and brain (30 patients, 102 T1W-Gad MRI scans, 317 lesions).

RESULTS

Revision of the manual lesion annotations revealed 120 missed lesions and 20 misidentified lesions in 31 out of 67 (46%) studies. The automatic revision reduced the number of computed missed lesions by 55 and computed misidentified lesions by 164 in 51 out of 67 (76%) studies.

CONCLUSION

Automatic analysis of three or more consecutive volumetric scans helps find missed and misidentified lesions and may improve the evaluation of temporal changes of oncological lesions.

An automated methodology for whole-body, multimodality tracking of individual cancer lesions

Objective. Manual analysis of individual cancer lesions to assess disease response is clinically impractical and requires automated lesion tracking methodologies. However, no methodology has been developed for whole-body individual lesion tracking, across an arbitrary number of scans, and acquired with various imaging modalities.Approach. This study introduces a lesion tracking methodology and benchmarked it using 2368Ga-DOTATATE PET/CT and PET/MR images of eight neuroendocrine tumor patients. The methodology consists of six steps: (1) alignment of multiple scans via image registration, (2) body-part labeling, (3) automatic lesion-wise dilation, (4) clustering of lesions based on local lesion shape metrics, (5) assignment of lesion tracks, and (6) output of a lesion graph. Registration performance was evaluated via landmark distance, lesion matching accuracy was evaluated between each image pair, and lesion tracking accuracy was evaluated via identical track ratio. Sensitivity studies were performed to evaluate the impact of lesion dilation (fixed versus automatic dilation), anatomic location, image modalities (inter- versus intra-modality), registration mode (direct versus indirect registration), and track size (number of time-points and lesions) on lesion matching and tracking performance.Main results. Manual contouring yielded 956 lesions, 1570 lesion-matching decisions, and 493 lesion tracks. The median residual registration error was 2.5 mm. The automatic lesion dilation led to 0.90 overall lesion matching accuracy, and an 88% identical track ratio. The methodology is robust regarding anatomic locations, image modalities, and registration modes. The number of scans had a moderate negative impact on the identical track ratio (94% for 2 scans, 91% for 3 scans, and 81% for 4 scans). The number of lesions substantially impacted the identical track ratio (93% for 2 nodes versus 54% for ≥5 nodes).Significance. The developed methodology resulted in high lesion-matching accuracy and enables automated lesion tracking in PET/CT and PET/MR.

Graph-based automatic detection and classification of lesion changes in pairs of CT studies for oncology follow-up

PURPOSE

Radiological follow-up of oncology patients requires the quantitative analysis of lesion changes in longitudinal imaging studies, which is time-consuming, requires expertise, and is subject to variability. This paper presents a comprehensive graph-based method for the automatic detection and classification of lesion changes in current and prior CT scans.

METHODS

The inputs are the current and prior CT scans and their organ and lesion segmentations. Classification of lesion changes is formalized as bipartite graph matching where lesion pairings are computed by adaptive overlap-based lesion matching. Six types of lesion changes are computed by connected components analysis. The method was evaluated on 208 pairs of lung and liver CT scans from 57 patients with 4600 lesions, 1713 lesion matchings and 2887 lesion changes. Ground-truth lesion segmentations, lesion matchings and lesion changes were created by an expert radiologist.

RESULTS

Our method yields a lesion matching rate accuracy of 99.7% (394/395) and 95.0% (1252/1318) for the lung and liver datasets. Precision and recall are > 0.99 and 0.94 and 0.95 (respectively) for the detection of lesion changes. The analysis of lesion changes helped the radiologist detect 48 missed lesions and 8 spurious lesions in the input ground-truth lesion datasets.

CONCLUSION

The classification of lesion classification provides the clinician with a readily accessible and intuitive identification and classification of the lesion changes and their patterns in support of clinical decision making. Comprehensive automatic computer-aided lesion matching and analysis of lesion changes may improve quantitative follow-up and evaluation of disease status, assessment of treatment efficacy and response to therapy.

Two is better than one: longitudinal detection and volumetric evaluation of brain metastases after Stereotactic Radiosurgery with a deep learning pipeline

PURPOSE

Close MRI surveillance of patients with brain metastases following Stereotactic Radiosurgery (SRS) treatment is essential for assessing treatment response and the current disease status in the brain. This follow-up necessitates the comparison of target lesion sizes in pre- (prior) and post-SRS treatment (current) T1W-Gad MRI scans. Our aim was to evaluate SimU-Net, a novel deep-learning model for the detection and volumetric analysis of brain metastases and their temporal changes in paired prior and current scans.

METHODS

SimU-Net is a simultaneous multi-channel 3D U-Net model trained on pairs of registered prior and current scans of a patient. We evaluated its performance on 271 pairs of T1W-Gad MRI scans from 226 patients who underwent SRS. An expert oncological neurosurgeon manually delineated 1,889 brain metastases in all the MRI scans (1,368 with diameters > 5 mm, 834 > 10 mm). The SimU-Net model was trained/validated on 205 pairs from 169 patients (1,360 metastases) and tested on 66 pairs from 57 patients (529 metastases). The results were then compared to the ground truth delineations.

RESULTS

SimU-Net yielded a mean (std) detection precision and recall of 1.00±0.00 and 0.99±0.06 for metastases > 10 mm, 0.90±0.22 and 0.97±0.12 for metastases > 5 mm of, and 0.76±0.27 and 0.94±0.16 for metastases of all sizes. It improves lesion detection precision by 8% for all metastases sizes and by 12.5% for metastases < 10 mm with respect to standalone 3D U-Net. The segmentation Dice scores were 0.90±0.10, 0.89±0.10 and 0.89±0.10 for the above metastases sizes, all above the observer variability of 0.80±0.13.

CONCLUSION

Automated detection and volumetric quantification of brain metastases following SRS have the potential to enhance the assessment of treatment response and alleviate the clinician workload.

Follow-up of liver metastases: a comparison of deep learning and RECIST 1.1

OBJECTIVES

To compare liver metastases changes in CT assessed by radiologists using RECIST 1.1 and with aided simultaneous deep learning-based volumetric lesion changes analysis.

METHODS

A total of 86 abdominal CT studies from 43 patients (prior and current scans) of abdominal CT scans of patients with 1041 liver metastases (mean = 12.1, std = 11.9, range 1-49) were analyzed. Two radiologists performed readings of all pairs; conventional with RECIST 1.1 and with computer-aided assessment. For computer-aided reading, we used a novel simultaneous multi-channel 3D R2U-Net classifier trained and validated on other scans. The reference was established by having an expert radiologist validate the computed lesion detection and segmentation. The results were then verified and modified as needed by another independent radiologist. The primary outcome measure was the disease status assessment with the conventional and the computer-aided readings with respect to the reference.

RESULTS

For conventional and computer-aided reading, there was a difference in disease status classification in 40 out of 86 (46.51%) and 10 out of 86 (27.9%) CT studies with respect to the reference, respectively. Computer-aided reading improved conventional reading in 30 CT studies by 34.5% for two readers (23.2% and 46.51%) with respect to the reference standard. The main reason for the difference between the two readings was lesion volume differences (p = 0.01).

CONCLUSIONS

AI-based computer-aided analysis of liver metastases may improve the accuracy of the evaluation of neoplastic liver disease status.

CLINICAL RELEVANCE STATEMENT

AI may aid radiologists to improve the accuracy of evaluating changes over time in metastasis of the liver.

KEY POINTS

• Classification of liver metastasis changes improved significantly in one-third of the cases with an automatically generated comprehensive lesion and lesion changes report. • Simultaneous deep learning changes detection and volumetric assessment may improve the evaluation of liver metastases temporal changes potentially improving disease management.

Performance of an automated registration-based method for longitudinal lesion matching and comparison to inter-reader variability

Objective.Patients with metastatic disease are followed throughout treatment with medical imaging, and accurately assessing changes of individual lesions is critical to properly inform clinical decisions. The goal of this work was to assess the performance of an automated lesion-matching algorithm in comparison to inter-reader variability (IRV) of matching lesions between scans of metastatic cancer patients.Approach.Forty pairs of longitudinal PET/CT and CT scans were collected and organized into four cohorts: lung cancers, head and neck cancers, lymphomas, and advanced cancers. Cases were also divided by cancer burden: low-burden (<10 lesions), intermediate-burden (10-29), and high-burden (30+). Two nuclear medicine physicians conducted independent reviews of each scan-pair and manually matched lesions. Matching differences between readers were assessed to quantify the IRV of lesion matching. The two readers met to form a consensus, which was considered a gold standard and compared against the output of an automated lesion-matching algorithm. IRV and performance of the automated method were quantified using precision, recall, F1-score, and the number of differences.Main results.The performance of the automated method did not differ significantly from IRV for any metric in any cohort (p> 0.05, Wilcoxon paired test). In high-burden cases, the F1-score (median [range]) was 0.89 [0.63, 1.00] between the automated method and reader consensus and 0.93 [0.72, 1.00] between readers. In low-burden cases, F1-scores were 1.00 [0.40, 1.00] and 1.00 [0.40, 1.00], for the automated method and IRV, respectively. Automated matching was significantly more efficient than either reader (p< 0.001). In high-burden cases, median matching time for the readers was 60 and 30 min, respectively, while automated matching took a median of 3.9 minSignificance.The automated lesion-matching algorithm was successful in performing lesion matching, meeting the benchmark of IRV. Automated lesion matching can significantly expedite and improve the consistency of longitudinal lesion-matching.