Dose-volume-based evaluation of convolutional neural network-based auto-segmentation of thoracic organs at risk

Smart contours: deep learning-driven internal gross tumor volume delineation in non-small cell lung cancer using 4D CT maximum and average intensity projections

BACKGROUND

Delineating the internal gross tumor volume (IGTV) is crucial for the treatment of non-small cell lung cancer (NSCLC). Deep learning (DL) enables the automation of this process; however, current studies focus mainly on multiple phases of four-dimensional (4D) computed tomography (CT), which leads to indirect results. This study proposed a DL-based method for automatic IGTV delineation using maximum and average intensity projections (MIP and AIP, respectively) from 4D CT.

METHODS

We retrospectively enrolled 124 patients with NSCLC and divided them into training (70%, n = 87) and validation (30%, n = 37) cohorts. Four-dimensional CT images were acquired, and the corresponding MIP and AIP images were generated. The IGTVs were contoured on 4D CT and used as the ground truth (GT). The MIP or AIP images, along with the corresponding IGTVs (IGTVMIP-manu and IGTVAIP-manu, respectively), were fed into the DL models for training and validation. We assessed the performance of three segmentation models-U-net, attention U-net, and V-net-using the Dice similarity coefficient (DSC) and the 95th percentile of the Hausdorff distance (HD95) as the primary metrics.

RESULTS

The attention U-net model trained on AIP images presented a mean DSC of 0.871 ± 0.048 and mean HD95 of 2.958 ± 2.266 mm, whereas the model trained on MIP images achieved a mean DSC of 0.852 ± 0.053 and mean HD95 of 3.209 ± 2.136 mm. Among the models, attention U-net and U-net achieved similar results, considerably surpassing V-net.

CONCLUSIONS

DL models can automate IGTV delineation using MIP and AIP images, streamline contouring, and enhance the accuracy and consistency of lung cancer radiotherapy planning to improve patient outcomes.

Geometric and dosimetric evaluation of a commercial AI auto-contouring tool on multiple anatomical sites in CT scans

Current radiotherapy practices rely on manual contouring of CT scans, which is time-consuming, prone to variability, and requires highly trained experts. There is a need for more efficient and consistent contouring methods. This study evaluated the performance of the Varian Ethos AI auto-contouring tool to assess its potential integration into clinical workflows. This retrospective study included 223 patients with treatment sites in the pelvis, abdomen, thorax, and head and neck regions. The Ethos AI tool generated auto-contours on each patients' pre-treatment planning CT, and 45 unique structures were included across the study cohort. Multiple measures of geometric similarity were computed, including surface Dice Similarity Coefficient (sDSC) and mean distance to agreement (MDA). Dosimetric concordance was evaluated by comparing mean dose and maximum 2 cm3 dose (D2 cc) between manual and AI contours. Ethos AI demonstrated high geometric accuracy for well-defined structures like the bladder, lungs, and femoral heads. Smaller structures and those with less defined boundaries, such as optic nerves and duodenum, showed lower agreement. Over 70% of auto-contours demonstrated a sDSC > 0.8, and 74% had MDA < 2.5 mm. Geometric accuracy generally correlated with dosimetric concordance, however differences in contour definitions did result in some structures exhibiting dose deviations. The Ethos AI auto-contouring tool offers promising accuracy and reliability for many anatomical structures, supporting its use in planning workflows. Auto-contouring errors, although rare, highlight the importance of ongoing QA and expert manual oversight.

Evaluating deep learning auto-contouring for lung radiation therapy: A review of accuracy, variability, efficiency and dose, in target volumes and organs at risk

Background and purpose

Delineation of target volumes (TVs) and organs at risk (OARs) is a resource intensive process in lung radiation therapy and, despite the introduction of some auto-contouring, inter-observer variability remains a challenge. Deep learning algorithms may prove an efficient alternative and this review aims to map the evidence base on the use of deep learning algorithms for TV and OAR delineation in the radiation therapy planning process for lung cancer patients.

Materials and methods

A literature search identified studies relating to deep learning. Manual contouring and deep learning auto-contours were evaluated against one another for accuracy, inter-observer variability, contouring time and dose-volume effects. A total of 40 studies were included for review.

Results

Thirty nine out of 40 studies investigated the accuracy of deep learning auto-contours and determined that they were of a comparable accuracy to manual contours. Inter-observer variability outcomes were heterogeneous in the seven relevant studies identified. Twenty-four studies analysed the time saving associated with deep learning auto-contours and reported a significant time reduction in comparison to manual contours. The eight studies that conducted a dose-volume metric evaluation on deep learning auto-contours identified negligible effect on treatment plans.

Conclusion

The accuracy and time-saving capacity of deep learning auto-contours in comparison to manual contours has been extensively studied. However, additional research is required in the areas of inter-observer variability and dose-volume metric evaluation to further substantiate its clinical use.

Predicting ergonomic risk among laboratory technicians using a Cheetah Optimizer-Integrated Deep Convolutional Neural Network

Medical laboratory technicians play a significant role in clinical units by conducting diagnostic tests and analyses. However, their job nature involving repetitive motions, prolonged standing or sitting, etc., leads to potential ergonomic risks. This research proposed a novel hybrid strategy by integrating the Cheetah Optimizer into the Deep Convolutional Neural Network (CHObDCNN) for predicting ergonomic risks in medical laboratory technicians. The presented framework commences with collecting images containing different postures and motions of laboratory technicians working in clinical units. The collected database was pre-processed to eliminate noises and other unwanted features. The DCNN component in the proposed framework performs the ergonomic risk prediction task by examining the patterns and interconnection with the image data, while the CHO component optimizes the DCNN training by tuning its parameters to its optimal range. Thus, the combined methodology offers improved classification results by iteratively updating its parameters. The presented framework was implemented in MATLAB, and the experimental outcomes manifest that the proposed method acquired improved accuracy of 98.74 %, greater precision of 98.56 %, and reduced computational time of 2.45 ms. Finally, the comparative study with the existing techniques validates its effectiveness in ergonomic risk prediction.

Accurate object localization facilitates automatic esophagus segmentation in deep learning

BACKGROUND

Currently, automatic esophagus segmentation remains a challenging task due to its small size, low contrast, and large shape variation. We aimed to improve the performance of esophagus segmentation in deep learning by applying a strategy that involves locating the object first and then performing the segmentation task.

METHODS

A total of 100 cases with thoracic computed tomography scans from two publicly available datasets were used in this study. A modified CenterNet, an object location network, was employed to locate the center of the esophagus for each slice. Subsequently, the 3D U-net and 2D U-net_coarse models were trained to segment the esophagus based on the predicted object center. A 2D U-net_fine model was trained based on the updated object center according to the 3D U-net model. The dice similarity coefficient and the 95% Hausdorff distance were used as quantitative evaluation indexes for the delineation performance. The characteristics of the automatically delineated esophageal contours by the 2D U-net and 3D U-net models were summarized. Additionally, the impact of the accuracy of object localization on the delineation performance was analyzed. Finally, the delineation performance in different segments of the esophagus was also summarized.

RESULTS

The mean dice coefficient of the 3D U-net, 2D U-net_coarse, and 2D U-net_fine models were 0.77, 0.81, and 0.82, respectively. The 95% Hausdorff distance for the above models was 6.55, 3.57, and 3.76, respectively. Compared with the 2D U-net, the 3D U-net has a lower incidence of delineating wrong objects and a higher incidence of missing objects. After using the fine object center, the average dice coefficient was improved by 5.5% in the cases with a dice coefficient less than 0.75, while that value was only 0.3% in the cases with a dice coefficient greater than 0.75. The dice coefficients were lower for the esophagus between the orifice of the inferior and the pulmonary bifurcation compared with the other regions.

CONCLUSION

The 3D U-net model tended to delineate fewer incorrect objects but also miss more objects. Two-stage strategy with accurate object location could enhance the robustness of the segmentation model and significantly improve the esophageal delineation performance, especially for cases with poor delineation results.

Dosimetric impact of contour editing on CT and MRI deep-learning autosegmentation for brain OARs

PURPOSE

To establish the clinical applicability of deep-learning organ-at-risk autocontouring models (DL-AC) for brain radiotherapy. The dosimetric impact of contour editing, prior to model training, on performance was evaluated for both CT and MRI-based models. The correlation between geometric and dosimetric measures was also investigated to establish whether dosimetric assessment is required for clinical validation.

METHOD

CT and MRI-based deep learning autosegmentation models were trained using edited and unedited clinical contours. Autosegmentations were dosimetrically compared to gold standard contours for a test cohort. D1%, D5%, D50%, and maximum dose were used as clinically relevant dosimetric measures. The statistical significance of dosimetric differences between the gold standard and autocontours was established using paired Student's t-tests. Clinically significant cases were identified via dosimetric headroom to the OAR tolerance. Pearson's Correlations were used to investigate the relationship between geometric measures and absolute percentage dose changes for each autosegmentation model.

RESULTS

Except for the right orbit, when delineated using MRI models, the dosimetric statistical analysis revealed no superior model in terms of the dosimetric accuracy between the CT DL-AC models or between the MRI DL-AC for any investigated brain OARs. The number of patients where the clinical significance threshold was exceeded was higher for the optic chiasm D1% than other OARs, for all autosegmentation models. A weak correlation was consistently observed between the outcomes of dosimetric and geometric evaluations.

CONCLUSIONS

Editing contours before training the DL-AC model had no significant impact on dosimetry. The geometric test metrics were inadequate to estimate the impact of contour inaccuracies on dose. Accordingly, dosimetric analysis is needed to evaluate the clinical applicability of DL-AC models in the brain.

A geometry and dose-volume based performance monitoring of artificial intelligence models in radiotherapy treatment planning for prostate cancer

Background and Purpose

Clinical Artificial Intelligence (AI) implementations lack ground-truth when applied on real-world data. This study investigated how combined geometrical and dose-volume metrics can be used as performance monitoring tools to detect clinically relevant candidates for model retraining.

Materials and Methods

Fifty patients were analyzed for both AI-segmentation and planning. For AI-segmentation, geometrical (Standard Surface Dice 3 mm and Local Surface Dice 3 mm) and dose-volume based parameters were calculated for two organs (bladder and anorectum) to compare AI output against the clinically corrected structure. A Local Surface Dice was introduced to detect geometrical changes in the vicinity of the target volumes, while an Absolute Dose Difference (ADD) evaluation increased focus on dose-volume related changes. AI-planning performance was evaluated using clinical goal analysis in combination with volume and target overlap metrics.

Results

The Local Surface Dice reported equal or lower values compared to the Standard Surface Dice (anorectum: (0.93 ± 0.11) vs (0.98 ± 0.04); bladder: (0.97 ± 0.06) vs (0.98 ± 0.04)). The ADD metric showed a difference of (0.9 ± 0.8)Gy for the anorectum D 1 cm 3 . The bladder D 5cm 3 reported a difference of (0.7 ± 1.5)Gy. Mandatory clinical goals were fulfilled in 90 % of the DLP plans.

Conclusions

Combining dose-volume and geometrical metrics allowed detection of clinically relevant changes, applied to both auto-segmentation and auto-planning output and the Local Surface Dice was more sensitive to local changes compared to the Standard Surface Dice. This monitoring is able to evaluate AI behavior in clinical practice and allows candidate selection for active learning.

Clinical Implementation and Evaluation of Auto-Segmentation Tools for Multi-Site Contouring in Radiotherapy

Background and purpose

Tools for auto-segmentation in radiotherapy are widely available, but guidelines for clinical implementation are missing. The goal was to develop a workflow for performance evaluation of three commercial auto-segmentation tools to select one candidate for clinical implementation.

Materials and Methods

One hundred patients with six treatment sites (brain, head-and-neck, thorax, abdomen, and pelvis) were included. Three sets of AI-based contours for organs-at-risk (OAR) generated by three software tools and manually drawn expert contours were blindly rated for contouring accuracy. The dice similarity coefficient (DSC), the Hausdorff distance, and a dose/volume evaluation based on the recalculation of the original treatment plan were assessed. Statistically significant differences were tested using the Kruskal-Wallis test and the post-hoc Dunn Test with Bonferroni correction.

Results

The mean DSC scores compared to expert contours for all OARs combined were 0.80 ± 0.10, 0.75 ± 0.10, and 0.74 ± 0.11 for the three software tools. Physicians' rating identified equivalent or superior performance of some AI-based contours in head (eye, lens, optic nerve, brain, chiasm), thorax (e.g., heart and lungs), and pelvis and abdomen (e.g., kidney, femoral head) compared to manual contours. For some OARs, the AI models provided results requiring only minor corrections. Bowel-bag and stomach were not fit for direct use. During the interdisciplinary discussion, the physicians' rating was considered the most relevant.

Conclusion

A comprehensive method for evaluation and clinical implementation of commercially available auto-segmentation software was developed. The in-depth analysis yielded clear instructions for clinical use within the radiotherapy department.

Assessment of Optimizers and their Performance in Autosegmenting Lung Tumors

Purpose

Optimizers are widely utilized across various domains to enhance desired outcomes by either maximizing or minimizing objective functions. In the context of deep learning, they help to minimize the loss function and improve model's performance. This study aims to evaluate the accuracy of different optimizers employed for autosegmentation of non-small cell lung cancer (NSCLC) target volumes on thoracic computed tomography images utilized in oncology.

Materials and Methods

The study utilized 112 patients, comprising 92 patients from "The Cancer Imaging Archive" (TCIA) and 20 of our local clinical patients, to evaluate the efficacy of various optimizers. The gross tumor volume was selected as the foreground mask for training and testing the models. Of the 92 TCIA patients, 57 were used for training and validation, and the remaining 35 for testing using nnU-Net. The performance of the final model was further evaluated on the 20 local clinical patient datasets. Six different optimizers, namely AdaDelta, AdaGrad, Adam, NAdam, RMSprop, and stochastic gradient descent (SGD), were investigated. To assess the agreement between the predicted volume and the ground truth, several metrics including Dice similarity coefficient (DSC), Jaccard index, sensitivity, precision, Hausdorff distance (HD), 95th percentile Hausdorff distance (HD95), and average symmetric surface distance (ASSD) were utilized.

Results

The DSC values for AdaDelta, AdaGrad, Adam, NAdam, RMSprop, and SGD were 0.75, 0.84, 0.85, 0.84, 0.83, and 0.81, respectively, for the TCIA test data. However, when the model trained on TCIA datasets was applied to the clinical datasets, the DSC, HD, HD95, and ASSD metrics showed a statistically significant decrease in performance compared to the TCIA test datasets, indicating the presence of image and/or mask heterogeneity between the data sources.

Conclusion

The choice of optimizer in deep learning is a critical factor that can significantly impact the performance of autosegmentation models. However, it is worth noting that the behavior of optimizers may vary when applied to new clinical datasets, which can lead to changes in models' performance. Therefore, selecting the appropriate optimizer for a specific task is essential to ensure optimal performance and generalizability of the model to different datasets.

Association of cardiac calcium burden with overall survival after radiotherapy for non-small cell lung cancer

Background and purpose

Coronary calcifications are associated with coronary artery disease in patients undergoing radiotherapy (RT) for non-small cell lung cancer (NSCLC). We quantified calcifications in the coronary arteries and aorta and investigated their relationship with overall survival (OS) in patients treated with definitive RT (Def-RT) or post-operative RT (PORT).

Materials and methods

We analyzed 263 NSCLC patients treated from 2004 to 2017. Calcium burden was ascertained with a Hounsfield unit (HU) cutoff of > 130 in addition to a deep learning (DL) plaque estimator. The HU cutoff volumes were defined for coronary arteries (Plaque_Coro) and coronary arteries and aorta combined (Plaque_Coro+Ao), while the DL estimator ranged from 0 (no plaque) to 3 (high plaque). Patient and treatment characteristics were explored for association with OS.

Results

The median Plaque_Coro and Plaque_Coro+Ao was 0.75 cm³ and 0.87 cm³ in the Def-RT group and 0.03 cm³ and 0.52 cm³ in the PORT group. The median DL estimator was 2 in both cohorts. In Def-RT, large Plaque_Coro (HR:1.11 (95%CI:1.04-1.19); p = 0.008), and Plaque_Coro+Ao (HR:1.06 (95%CI:1.02-1.11); p = 0.03), and poor Karnofsky Performance Status (HR: 0.97 (95%CI: 0.94-0.99); p = 0.03) were associated with worse OS. No relationship was identified between the plaque volumes and OS in PORT, or between the DL plaque estimator and OS in either Def-RT or PORT.

Conclusions

Coronary artery calcification assessed from RT planning CT scans was significantly associated with OS in patients who underwent Def-RT for NSCLC. This HU thresholding method can be straightforwardly implemented such that the role of calcifications can be further explored.