Comparison of atlas-based auto-segmentation accuracy for radiotherapy in prostate cancer

Comparison of commercial atlas-based automatic segmentation software for prostate radiotherapy treatment planning

This study aims to assess the accuracy of automatic atlas-based contours for various key anatomical structures in prostate radiotherapy treatment planning. The evaluated structures include the bladder, rectum, prostate, seminal vesicles, femoral heads and penile bulb. CT images from 20 patients who underwent intensity-modulated radiotherapy were randomly chosen to create an atlas library. Atlas contours of the seven anatomical structures were generated using four software packages: ABAS, Eclipse, MIM, and RayStation. These contours were then compared to manual delineations performed by oncologists, which served as the ground truth. Evaluation metrics such as dice similarity coefficient (DSC), mean distance to agreement (MDA), and volume ratio (VR) were calculated to assess the accuracy of the contours. Additionally, the time taken by each software to generate the atlas contour was recorded. The mean DSC values for the bladder exhibited strong agreement (>0.8) with manual delineations for all software except for Eclipse and RayStation. Similarly, the femoral heads showed significant similarity between the atlas contours and ground truth across all software, with mean DSC values exceeding 0.9 and MDA values close to zero. On the other hand, the penile bulb displayed only moderate agreement with the ground truth, with mean DSC values ranging from 0.5 to 0.7 for all software. A similar trend was observed in the prostate atlas contours, except for MIM, which achieved a mean DSC of over 0.8. For the rectum, both ABAS and MIM atlases demonstrated strong agreement with the ground truth, resulting in mean DSC values of more than 0.8. Overall, MIM and ABAS outperformed Eclipse and RayStation in both DSC and MDA. These results indicate that the atlas-based segmentation employed in this study produces acceptable contours for the anatomical structures of interest in prostate radiotherapy treatment planning.

Artificial Intelligence-Based Autosegmentation: Advantages in Delineation, Absorbed Dose-Distribution, and Logistics

Purpose

The study's purpose was to compare the performance of artificial intelligence (AI) in auto-contouring compared with a human practitioner in terms of precision, differences in dose distribution, and time consumption.

Methods and Materials

Datasets of previously irradiated patients in 3 different segments (head and neck, breast, and prostate cancer) were retrospectively collected. An experienced radiation oncologist (MD) performed organs-at-risk (OARs) and standard clinical target volume delineations as baseline structures for comparison. AI-based autocontours were generated in 2 additional CT copies; therefore, 3 groups were assessed: MD alone, AI alone, and AI plus MD corrections (AI+C). Differences in Dice similarity coefficient (DSC) and person-hour burden were assessed. Furthermore, changes in clinically relevant dose-volume parameters were evaluated and compared.

Results

Seventy-five previously treated cases were collected (25 per segment) for the analysis. Compared with MD contours, the mean DSC scores were higher than 0.7 for 74% and 80% of AI and AI+C, respectively. After corrections, 17.1% structures presented DSC score deviations higher than 0.1 and 10.4% dose-volume parameters significantly changed in AI-contoured structures. The time consumption assessment yielded mean person-hour reductions of 68%, 51%, and 71% for breast, prostate, and head and neck cancer, respectively.

Conclusions

In great extent, AI yielded clinically acceptable OARs and certain clinical target volumes in the explored anatomic segments. Sparse correction and assessment requirements place AI+C as a standard workflow. Minimal clinically relevant differences in OAR exposure were identified. A substantial amount of person-hours could be repurposed with this technology.

Real-world validation of Artificial Intelligence-based Computed Tomography auto-contouring for prostate cancer radiotherapy planning

Background and purpose

Artificial Intelligence (AI)-based auto-contouring for treatment planning in radiotherapy needs extensive clinical validation, including the impact of editing after automatic segmentation. The aims of this study were to assess the performance of a commercial system for Clinical Target Volumes (CTVs) (prostate/seminal vesicles) and selected Organs at Risk (OARs) (rectum/bladder/femoral heads + femurs), evaluating also inter-observer variability (manual vs automatic + editing) and the reduction of contouring time.

Materials and methods

Two expert observers contoured CTVs/OARs of 20 patients in our Treatment Planning System (TPS). Computed Tomography (CT) images were sent to the automatic contouring workstation: automatic contours were generated and sent back to TPS, where observers could edit them if necessary. Inter- and intra-observer consistency was estimated using Dice Similarity Coefficients (DSC). Radiation oncologists were also asked to score the quality of automatic contours, ranging from 1 (complete re-contouring) to 5 (no editing). Contouring times (manual vs automatic + edit) were compared.

Results

DSCs (manual vs automatic only) were consistent with inter-observer variability (between 0.65 for seminal vesicles and 0.94 for bladder); editing further improved performances (range: 0.76-0.94). The median clinical score was 4 (little editing) and it was <4 in 3/2 patients for the two observers respectively. Inter-observer variability of automatic + editing contours improved significantly, being lower than manual contouring (e.g.: seminal vesicles: 0.83vs0.73; prostate: 0.86vs0.83; rectum: 0.96vs0.81). Oncologist contouring time reduced from 17 to 24 min of manual contouring time to 3-7 min of editing time for the two observers (p < 0.01).

Conclusion

Automatic contouring with a commercial AI-based system followed by editing can replace manual contouring, resulting in significantly reduced time for segmentation and better consistency between operators.

A Review of the Metrics Used to Assess Auto-Contouring Systems in Radiotherapy

Auto-contouring could revolutionise future planning of radiotherapy treatment. The lack of consensus on how to assess and validate auto-contouring systems currently limits clinical use. This review formally quantifies the assessment metrics used in studies published during one calendar year and assesses the need for standardised practice. A PubMed literature search was undertaken for papers evaluating radiotherapy auto-contouring published during 2021. Papers were assessed for types of metric and the methodology used to generate ground-truth comparators. Our PubMed search identified 212 studies, of which 117 met the criteria for clinical review. Geometric assessment metrics were used in 116 of 117 studies (99.1%). This includes the Dice Similarity Coefficient used in 113 (96.6%) studies. Clinically relevant metrics, such as qualitative, dosimetric and time-saving metrics, were less frequently used in 22 (18.8%), 27 (23.1%) and 18 (15.4%) of 117 studies, respectively. There was heterogeneity within each category of metric. Over 90 different names for geometric measures were used. Methods for qualitative assessment were different in all but two papers. Variation existed in the methods used to generate radiotherapy plans for dosimetric assessment. Consideration of editing time was only given in 11 (9.4%) papers. A single manual contour as a ground-truth comparator was used in 65 (55.6%) studies. Only 31 (26.5%) studies compared auto-contours to usual inter- and/or intra-observer variation. In conclusion, significant variation exists in how research papers currently assess the accuracy of automatically generated contours. Geometric measures are the most popular, however their clinical utility is unknown. There is heterogeneity in the methods used to perform clinical assessment. Considering the different stages of system implementation may provide a framework to decide the most appropriate metrics. This analysis supports the need for a consensus on the clinical implementation of auto-contouring.

A deep learning-based self-adapting ensemble method for segmentation in gynecological brachytherapy

Purpose

Fast and accurate outlining of the organs at risk (OARs) and high-risk clinical tumor volume (HRCTV) is especially important in high-dose-rate brachytherapy due to the highly time-intensive online treatment planning process and the high dose gradient around the HRCTV. This study aims to apply a self-configured ensemble method for fast and reproducible auto-segmentation of OARs and HRCTVs in gynecological cancer.

Materials and methods

We applied nnU-Net (no new U-Net), an automatically adapted deep convolutional neural network based on U-Net, to segment the bladder, rectum and HRCTV on CT images in gynecological cancer. In nnU-Net, three architectures, including 2D U-Net, 3D U-Net and 3D-Cascade U-Net, were trained and finally ensembled. 207 cases were randomly chosen for training, and 30 for testing. Quantitative evaluation used well-established image segmentation metrics, including dice similarity coefficient (DSC), 95% Hausdorff distance (HD95%), and average surface distance (ASD). Qualitative analysis of automated segmentation results was performed visually by two radiation oncologists. The dosimetric evaluation was performed by comparing the dose-volume parameters of both predicted segmentation and human contouring.

Results

nnU-Net obtained high qualitative and quantitative segmentation accuracy on the test dataset and performed better than previously reported methods in bladder and rectum segmentation. In quantitative evaluation, 3D-Cascade achieved the best performance in the bladder (DSC: 0.936 ± 0.051, HD95%: 3.503 ± 1.956, ASD: 0.944 ± 0.503), rectum (DSC: 0.831 ± 0.074, HD95%: 7.579 ± 5.857, ASD: 3.6 ± 3.485), and HRCTV (DSC: 0.836 ± 0.07, HD95%: 7.42 ± 5.023, ASD: 2.094 ± 1.311). According to the qualitative evaluation, over 76% of the test data set had no or minor visually detectable errors in segmentation.

Conclusion

This work showed nnU-Net’s superiority in segmenting OARs and HRCTV in gynecological brachytherapy cases in our center, among which 3D-Cascade shows the highest accuracy in segmentation across different applicators and patient anatomy.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13014-022-02121-3.