Towards interactive deep-learning for tumour segmentation in head and neck cancer radiotherapy

Gradient Map-Assisted Head and Neck Tumor Segmentation: A Pre-RT to Mid-RT Approach in MRI-Guided Radiotherapy

Radiation therapy (RT) is a vital part of treatment for head and neck cancer, where accurate segmentation of gross tumor volume (GTV) is essential for effective treatment planning. This study investigates the use of pre-RT tumor regions and local gradient maps to enhance mid-RT tumor segmentation for head and neck cancer in MRI-guided adaptive radiotherapy. By leveraging pre-RT images and their segmentations as prior knowledge, we address the challenge of tumor localization in mid-RT segmentation. A gradient map of the tumor region from the pre-RT image is computed and applied to mid-RT images to improve tumor boundary delineation. Our approach demonstrated improved segmentation accuracy for both primary GTV (GTVp) and nodal GTV (GTVn), though performance was limited by data constraints. The final DSC agg scores from the challenge's test set evaluation were 0.534 for GTVp, 0.867 for GTVn, and a mean score of 0.70. This method shows potential for enhancing segmentation and treatment planning in adaptive radiotherapy. Team: DCPT-Stine's group.

UMamba Adjustment: Advancing GTV Segmentation for Head and Neck Cancer in MRI-Guided RT with UMamba and NnU-Net ResEnc Planner

Magnetic Resonance Imaging (MRI) plays a crucial role in MRI-guided adaptive radiotherapy for head and neck cancer (HNC) due to its superior soft-tissue contrast. However, accurately segmenting the gross tumor volume (GTV), which includes both the primary tumor (GTVp) and lymph nodes (GTVn), remains challenging. Recently, two deep learning segmentation innovations have shown great promise: UMamba, which effectively captures long-range dependencies, and the nnU-Net Residual Encoder (ResEnc), which enhances feature extraction through multistage residual blocks. In this study, we integrate these strengths into a novel approach, termed 'UMambaAdj'. Our proposed method was evaluated on the HNTS-MRG 2024 challenge test set using pre-RT T2-weighted MRI images, achieving an aggregated Dice Similarity Coefficient ( D S C a g g ) of 0.751 for GTVp and 0.842 for GTVn, with a mean D S C a g g of 0.796. This approach demonstrates potential for more precise tumor delineation in MRI-guided adaptive radiotherapy, ultimately improving treatment outcomes for HNC patients. Team: DCPT-Stine's group.

Deep Interactive Segmentation of Medical Images: A Systematic Review and Taxonomy

Interactive segmentation is a crucial research area in medical image analysis aiming to boost the efficiency of costly annotations by incorporating human feedback. This feedback takes the form of clicks, scribbles, or masks and allows for iterative refinement of the model output so as to efficiently guide the system towards the desired behavior. In recent years, deep learning-based approaches have propelled results to a new level causing a rapid growth in the field with 121 methods proposed in the medical imaging domain alone. In this review, we provide a structured overview of this emerging field featuring a comprehensive taxonomy, a systematic review of existing methods, and an in-depth analysis of current practices. Based on these contributions, we discuss the challenges and opportunities in the field. For instance, we find that there is a severe lack of comparison across methods which needs to be tackled by standardized baselines and benchmarks.

Deep learning-assisted interactive contouring of lung cancer: Impact on contouring time and consistency

BACKGROUND AND PURPOSE

To evaluate the impact of a deep learning (DL)-assisted interactive contouring tool on inter-observer variability and the time taken to complete tumour contouring.

MATERIALS AND METHODS

Nine clinicians contoured the gross tumour volume (GTV) using the PET-CT scans of 10 non-small cell lung cancer (NSCLC) patients, either using DL-assisted or manual contouring tools. After contouring a case using one contouring method, the same case was contoured one week later using the other method. The contours and time taken were compared.

RESULTS

Use of the DL-assisted tool led to a statistically significant decrease in active contouring time of 23 % relative to the standard manual segmentation method (p < 0.01). The mean observation time for all clinicians and cases made up nearly 60 % of interaction time for both contouring approaches. On average the time spent contouring per case was reduced from 22 min to 19 min when using the DL-assisted tool. Additionally, the DL-assisted tool reduced contour variability in the parts of tumour where clinicians tended to disagree the most, while the consensus contour was similar whichever of the two contouring approaches was used.

CONCLUSIONS

A DL-assisted interactive contouring approach decreased active contouring time and local inter-observer variability when used to delineate lung cancer GTVs compared to a standard manual method. Integration of this tool into the clinical workflow could assist clinicians in contouring tasks and improve contouring efficiency.

Enhancing the reliability of deep learning-based head and neck tumour segmentation using uncertainty estimation with multi-modal images

Objective.Deep learning shows promise in autosegmentation of head and neck cancer (HNC) primary tumours (GTV-T) and nodal metastases (GTV-N). However, errors such as including non-tumour regions or missing nodal metastases still occur. Conventional methods often make overconfident predictions, compromising reliability. Incorporating uncertainty estimation, which provides calibrated confidence intervals can address this issue. Our aim was to investigate the efficacy of various uncertainty estimation methods in improving segmentation reliability. We evaluated their confidence levels in voxel predictions and ability to reveal potential segmentation errors.Approach.We retrospectively collected data from 567 HNC patients with diverse cancer sites and multi-modality images (CT, PET, T1-, and T2-weighted MRI) along with their clinical GTV-T/N delineations. Using the nnUNet 3D segmentation pipeline, we compared seven uncertainty estimation methods, evaluating them based on segmentation accuracy (Dice similarity coefficient, DSC), confidence calibration (Expected Calibration Error, ECE), and their ability to reveal segmentation errors (Uncertainty-Error overlap using DSC, UE-DSC).Main results.Evaluated on the hold-out test dataset (n= 97), the median DSC scores for GTV-T and GTV-N segmentation across all uncertainty estimation methods had a narrow range, from 0.73 to 0.76 and 0.78 to 0.80, respectively. In contrast, the median ECE exhibited a wider range, from 0.30 to 0.12 for GTV-T and 0.25 to 0.09 for GTV-N. Similarly, the median UE-DSC also ranged broadly, from 0.21 to 0.38 for GTV-T and 0.22 to 0.36 for GTV-N. A probabilistic network-PhiSeg method consistently demonstrated the best performance in terms of ECE and UE-DSC.Significance.Our study highlights the importance of uncertainty estimation in enhancing the reliability of deep learning for autosegmentation of HNC GTV. The results show that while segmentation accuracy can be similar across methods, their reliability, measured by calibration error and uncertainty-error overlap, varies significantly. Used with visualisation maps, these methods may effectively pinpoint uncertainties and potential errors at the voxel level.

Accurate object localization facilitates automatic esophagus segmentation in deep learning

BACKGROUND

Currently, automatic esophagus segmentation remains a challenging task due to its small size, low contrast, and large shape variation. We aimed to improve the performance of esophagus segmentation in deep learning by applying a strategy that involves locating the object first and then performing the segmentation task.

METHODS

A total of 100 cases with thoracic computed tomography scans from two publicly available datasets were used in this study. A modified CenterNet, an object location network, was employed to locate the center of the esophagus for each slice. Subsequently, the 3D U-net and 2D U-net_coarse models were trained to segment the esophagus based on the predicted object center. A 2D U-net_fine model was trained based on the updated object center according to the 3D U-net model. The dice similarity coefficient and the 95% Hausdorff distance were used as quantitative evaluation indexes for the delineation performance. The characteristics of the automatically delineated esophageal contours by the 2D U-net and 3D U-net models were summarized. Additionally, the impact of the accuracy of object localization on the delineation performance was analyzed. Finally, the delineation performance in different segments of the esophagus was also summarized.

RESULTS

The mean dice coefficient of the 3D U-net, 2D U-net_coarse, and 2D U-net_fine models were 0.77, 0.81, and 0.82, respectively. The 95% Hausdorff distance for the above models was 6.55, 3.57, and 3.76, respectively. Compared with the 2D U-net, the 3D U-net has a lower incidence of delineating wrong objects and a higher incidence of missing objects. After using the fine object center, the average dice coefficient was improved by 5.5% in the cases with a dice coefficient less than 0.75, while that value was only 0.3% in the cases with a dice coefficient greater than 0.75. The dice coefficients were lower for the esophagus between the orifice of the inferior and the pulmonary bifurcation compared with the other regions.

CONCLUSION

The 3D U-net model tended to delineate fewer incorrect objects but also miss more objects. Two-stage strategy with accurate object location could enhance the robustness of the segmentation model and significantly improve the esophageal delineation performance, especially for cases with poor delineation results.

Clinical acceptability of automatically generated lymph node levels and structures of deglutition and mastication for head and neck radiation therapy

Background and Purpose

Auto-contouring of complex anatomy in computed tomography (CT) scans is a highly anticipated solution to many problems in radiotherapy. In this study, artificial intelligence (AI)-based auto-contouring models were clinically validated for lymph node levels and structures of swallowing and chewing in the head and neck.

Materials and Methods

CT scans of 145 head and neck radiotherapy patients were retrospectively curated. One cohort (n = 47) was used to analyze seven lymph node levels and the other (n = 98) used to analyze 17 swallowing and chewing structures. Separate nnUnet models were trained and validated using the separate cohorts. For the lymph node levels, preference and clinical acceptability of AI vs human contours were scored. For the swallowing and chewing structures, clinical acceptability was scored. Quantitative analyses of the test sets were performed for AI vs human contours for all structures using overlap and distance metrics.

Results

Median Dice Similarity Coefficient ranged from 0.77 to 0.89 for lymph node levels and 0.86 to 0.96 for chewing and swallowing structures. The AI contours were superior to or equally preferred to the manual contours at rates ranging from 75% to 91%; there was not a significant difference in clinical acceptability for nodal levels I-V for manual versus AI contours. Across all AI-generated lymph node level contours, 92% were rated as usable with stylistic to no edits. Of the 340 contours in the chewing and swallowing cohort, 4% required minor edits.

Conclusions

An accurate approach was developed to auto-contour lymph node levels and chewing and swallowing structures on CT images for patients with intact nodal anatomy. Only a small portion of test set auto-contours required minor edits.

Auto-segmentation of neck nodal metastases using self-distilled masked image transformer on longitudinal MR images

Objectives

Auto-segmentation promises greater speed and lower inter-reader variability than manual segmentations in radiation oncology clinical practice. This study aims to implement and evaluate the accuracy of the auto-segmentation algorithm, "Masked Image modeling using the vision Transformers (SMIT)," for neck nodal metastases on longitudinal T2-weighted (T2w) MR images in oropharyngeal squamous cell carcinoma (OPSCC) patients.

Methods

This prospective clinical trial study included 123 human papillomaviruses (HPV-positive [+]) related OSPCC patients who received concurrent chemoradiotherapy. T2w MR images were acquired on 3 T at pre-treatment (Tx), week 0, and intra-Tx weeks (1-3). Manual delineations of metastatic neck nodes from 123 OPSCC patients were used for the SMIT auto-segmentation, and total tumor volumes were calculated. Standard statistical analyses compared contour volumes from SMIT vs manual segmentation (Wilcoxon signed-rank test [WSRT]), and Spearman's rank correlation coefficients (ρ) were computed. Segmentation accuracy was evaluated on the test data set using the dice similarity coefficient (DSC) metric value. P-values <0.05 were considered significant.

Results

No significant difference in manual and SMIT delineated tumor volume at pre-Tx (8.68 ± 7.15 vs 8.38 ± 7.01 cm3, P = 0.26 [WSRT]), and the Bland-Altman method established the limits of agreement as -1.71 to 2.31 cm3, with a mean difference of 0.30 cm3. SMIT model and manually delineated tumor volume estimates were highly correlated (ρ = 0.84-0.96, P < 0.001). The mean DSC metric values were 0.86, 0.85, 0.77, and 0.79 at the pre-Tx and intra-Tx weeks (1-3), respectively.

Conclusions

The SMIT algorithm provides sufficient segmentation accuracy for oncological applications in HPV+ OPSCC.

Advances in knowledge

First evaluation of auto-segmentation with SMIT using longitudinal T2w MRI in HPV+ OPSCC.

Consistency in contouring of organs at risk by artificial intelligence vs oncologists in head and neck cancer patients

BACKGROUND

In the Danish Head and Neck Cancer Group (DAHANCA) 35 trial, patients are selected for proton treatment based on simulated reductions of Normal Tissue Complication Probability (NTCP) for proton compared to photon treatment at the referring departments. After inclusion in the trial, immobilization, scanning, contouring and planning are repeated at the national proton centre. The new contours could result in reduced expected NTCP gain of the proton plan, resulting in a loss of validity in the selection process. The present study evaluates if contour consistency can be improved by having access to AI (Artificial Intelligence) based contours.

MATERIALS AND METHODS

The 63 patients in the DAHANCA 35 pilot trial had a CT from the local DAHANCA centre and one from the proton centre. A nationally validated convolutional neural network, based on nnU-Net, was used to contour OARs on both scans for each patient. Using deformable image registration, local AI and oncologist contours were transferred to the proton centre scans for comparison. Consistency was calculated with the Dice Similarity Coefficient (DSC) and Mean Surface Distance (MSD), comparing contours from AI to AI and oncologist to oncologist, respectively. Two NTCP models were applied to calculate NTCP for xerostomia and dysphagia.

RESULTS

The AI contours showed significantly better consistency than the contours by oncologists. The median and interquartile range of DSC was 0.85 [0.78 - 0.90] and 0.68 [0.51 - 0.80] for AI and oncologist contours, respectively. The median and interquartile range of MSD was 0.9 mm [0.7 - 1.1] mm and 1.9 mm [1.5 - 2.6] mm for AI and oncologist contours, respectively. There was no significant difference in Δ NTCP.

CONCLUSIONS

The study showed that OAR contours made by the AI algorithm were more consistent than those made by oncologists. No significant impact on the Δ NTCP calculations could be discerned.

Extensive clinical testing of Deep Learning Segmentation models for thorax and breast cancer radiotherapy planning

BACKGROUND

The performance of deep learning segmentation (DLS) models for automatic organ extraction from CT images in the thorax and breast regions was investigated. Furthermore, the readiness and feasibility of integrating DLS into clinical practice were addressed by measuring the potential time savings and dosimetric impact.

MATERIAL AND METHODS

Thirty patients referred to radiotherapy for breast cancer were prospectively included. A total of 23 clinically relevant left- and right-sided organs were contoured manually on CT images according to ESTRO guidelines. Next, auto-segmentation was executed, and the geometric agreement between the auto-segmented and manually contoured organs was qualitatively assessed applying a scale in the range [0-not acceptable, 3-no corrections]. A quantitative validation was carried out by calculating Dice coefficients (DSC) and the 95% percentile of Hausdorff distances (HD95). The dosimetric impact of optimizing the treatment plans on the uncorrected DLS contours, was investigated from a dose coverage analysis using DVH values of the manually delineated contours as references.

RESULTS

The qualitative analysis showed that 93% of the DLS generated OAR contours did not need corrections, except for the heart where 67% of the contours needed corrections. The majority of DLS generated CTVs needed corrections, whereas a minority were deemed not acceptable. Still, using the DLS-model for CTV and heart delineation is on average 14 minutes faster. An average DSC=0.91 and H95=9.8 mm were found for the left and right breasts, respectively. Likewise, and average DSC in the range [0.66, 0.76]mm and HD95 in the range [7.04, 12.05]mm were found for the lymph nodes.

CONCLUSION

The validation showed that the DLS generated OAR contours can be used clinically. Corrections were required to most of the DLS generated CTVs, and therefore warrants more attention before possibly implementing the DLS models clinically.