A national study on the inter-observer variability in the delineation of organs at risk in the brain

Contouring aid tools in radiotherapy. Smoothing: the false friend

OBJECTIVE

Contouring accuracy is critical in modern radiotherapy. Several tools are available to assist clinicians in this task. This study aims to evaluate the performance of the smoothing tool in the ARIA system to obtain more consistent volumes.

METHODS

Eleven different geometric shapes were delineated in ARIA v15.6 (Sphere, Cube, Square Prism, Six-Pointed Star Prism, Arrow Prism, And Cylinder and the respective volumes at 45° of axis deviation (_45)) in 1, 3, 5, 7, and 10 cm side or diameter each. Post-processing drawing tools to smooth those first-generated volumes were applied in different options (2D-ALL vs 3D) and grades (1, 3, 5, 10, 15, and 20). These volumetric transformations were analyzed by comparing different parameters: volume changes, center of mass, and DICE similarity coefficient index. Then we studied how smoothing affected two different volumes in a head and neck cancer patient: a single rounded node and the volume delineating cervical nodal areas.

RESULTS

No changes in data were found between 2D-ALL or 3D smoothing. Minimum deviations were found (range from 0 to 0.45 cm) in the center of mass. Volumes and the DICE index decreased as the degree of smoothing increased. Some discrepancies were found, especially in figures with cleft and spikes that behave differently. In the clinical case, smoothing should be applied only once throughout the target delineation process, preferably in the largest volume (PTV) to minimize errors.

CONCLUSION

Smoothing is a good tool to reduce artifacts due to the manual delineation of radiotherapy volumes. The resulting volumes must be always carefully reviewed.

Consistency in contouring of organs at risk by artificial intelligence vs oncologists in head and neck cancer patients

BACKGROUND

In the Danish Head and Neck Cancer Group (DAHANCA) 35 trial, patients are selected for proton treatment based on simulated reductions of Normal Tissue Complication Probability (NTCP) for proton compared to photon treatment at the referring departments. After inclusion in the trial, immobilization, scanning, contouring and planning are repeated at the national proton centre. The new contours could result in reduced expected NTCP gain of the proton plan, resulting in a loss of validity in the selection process. The present study evaluates if contour consistency can be improved by having access to AI (Artificial Intelligence) based contours.

MATERIALS AND METHODS

The 63 patients in the DAHANCA 35 pilot trial had a CT from the local DAHANCA centre and one from the proton centre. A nationally validated convolutional neural network, based on nnU-Net, was used to contour OARs on both scans for each patient. Using deformable image registration, local AI and oncologist contours were transferred to the proton centre scans for comparison. Consistency was calculated with the Dice Similarity Coefficient (DSC) and Mean Surface Distance (MSD), comparing contours from AI to AI and oncologist to oncologist, respectively. Two NTCP models were applied to calculate NTCP for xerostomia and dysphagia.

RESULTS

The AI contours showed significantly better consistency than the contours by oncologists. The median and interquartile range of DSC was 0.85 [0.78 - 0.90] and 0.68 [0.51 - 0.80] for AI and oncologist contours, respectively. The median and interquartile range of MSD was 0.9 mm [0.7 - 1.1] mm and 1.9 mm [1.5 - 2.6] mm for AI and oncologist contours, respectively. There was no significant difference in Δ NTCP.

CONCLUSIONS

The study showed that OAR contours made by the AI algorithm were more consistent than those made by oncologists. No significant impact on the Δ NTCP calculations could be discerned.

Multi-organ segmentation of CT via convolutional neural network: impact of training setting and scanner manufacturer

Objective. Automated organ segmentation on CT images can enable the clinical use of advanced quantitative software devices, but model performance sensitivities must be understood before widespread adoption can occur. The goal of this study was to investigate performance differences between Convolutional Neural Networks (CNNs) trained to segment one (single-class) versus multiple (multi-class) organs, and between CNNs trained on scans from a single manufacturer versus multiple manufacturers.Methods. The multi-class CNN was trained on CT images obtained from 455 whole-body PET/CT scans (413 for training, 42 for testing) taken with Siemens, GE, and Phillips PET/CT scanners where 16 organs were segmented. The multi-class CNN was compared to 16 smaller single-class CNNs trained using the same data, but with segmentations of only one organ per model. In addition, CNNs trained on Siemens-only (N = 186) and GE-only (N = 219) scans (manufacturer-specific) were compared with CNNs trained on data from both Siemens and GE scanners (manufacturer-mixed). Segmentation performance was quantified using five performance metrics, including the Dice Similarity Coefficient (DSC).Results. The multi-class CNN performed well compared to previous studies, even in organs usually considered difficult auto-segmentation targets (e.g., pancreas, bowel). Segmentations from the multi-class CNN were significantly superior to those from smaller single-class CNNs in most organs, and the 16 single-class models took, on average, six times longer to segment all 16 organs compared to the single multi-class model. The manufacturer-mixed approach achieved minimally higher performance over the manufacturer-specific approach.Significance. A CNN trained on contours of multiple organs and CT data from multiple manufacturers yielded high-quality segmentations. Such a model is an essential enabler of image processing in a software device that quantifies and analyzes such data to determine a patient's treatment response. To date, this activity of whole organ segmentation has not been adopted due to the intense manual workload and time required.

A national repository of complete radiotherapy plans: design, Results, and experiences

BACKGROUND

Previously, many radiotherapy (RT) trials were based on a few selected dose measures. Many research questions, however, rely on access to the complete dose information. To support such access, a national RT plan database was created. The system focuses on data security, ease of use, and re-use of data. This article reports on the development and structure, and the functionality and experience of this national database.

METHODS AND MATERIALS

A system based on the DICOM-RT standard, DcmCollab, was implemented with direct connections to all Danish RT centres. Data is segregated into any number of collaboration projects. User access to the system is provided through a web interface. The database has a finely defined access permission model to support legal requirements.

RESULTS

Currently, data for more than 14,000 patients have been submitted to the system, and more than 50 research projects are registered. The system is used for data collection, trial quality assurance, and audit data set generation.Users reported that the process of submitting data, waiting for it to be processed, and then manually attaching it to a project was resource intensive. This was accommodated with the introduction of triggering features, eliminating much of the need for users to manage data manually. Many other features, including structure name mapping, RT plan viewer, and the Audit Tool were developed based on user input.

CONCLUSION

The DcmCollab system has provided an efficient means to collect and access complete datasets for multi-centre RT research. This stands in contrast with previous methods of collecting RT data in multi-centre settings, where only singular data points were manually reported. To accommodate the evolving legal environment, DcmCollab has been defined as a 'data processor', meaning that it is a tool for other research projects to use rather than a research project in and of itself.

Deep-learning magnetic resonance imaging-based automatic segmentation for organs-at-risk in the brain: Accuracy and impact on dose distribution

Background and purpose

Normal tissue sparing in radiotherapy relies on proper delineation. While manual contouring is time consuming and subject to inter-observer variability, auto-contouring could optimize workflows and harmonize practice. We assessed the accuracy of a commercial, deep-learning, MRI-based tool for brain organs-at-risk delineation.

Materials and methods

Thirty adult brain tumor patients were retrospectively manually recontoured. Two additional structure sets were obtained: AI (artificial intelligence) and AIedit (manually corrected auto-contours). For 15 selected cases, identical plans were optimized for each structure set. We used Dice Similarity Coefficient (DSC) and mean surface-distance (MSD) for geometric comparison and gamma analysis and dose-volume-histogram comparison for dose metrics evaluation. Wilcoxon signed-ranks test was used for paired data, Spearman coefficient(ρ) for correlations and Bland-Altman plots to assess level of agreement.

Results

Auto-contouring was significantly faster than manual (1.1/20 min, p < 0.01). Median DSC and MSD were 0.7/0.9 mm for AI and 0.8/0.5 mm for AIedit. DSC was significantly correlated with structure size (ρ = 0.76, p < 0.01), with higher DSC for large structures. Median gamma pass rate was 74% (71-81%) for Plan_AI and 82% (75-86%) for Plan_AIedit, with no correlation with DSC or MSD. Differences between Dmean_AI and Dmean_Ref were ≤ 0.2 Gy (p < 0.05). The dose difference was moderately correlated with DSC. Bland Altman plot showed minimal discrepancy (0.1/0) between AI and reference Dmean/Dmax.

Conclusions

The AI-model showed good accuracy for large structures, but developments are required for smaller ones. Auto-segmentation was significantly faster, with minor differences in dose distribution caused by geometric variations.

Practical and technical key challenges in head and neck adaptive radiotherapy: The GORTEC point of view

Anatomical variations occur during head and neck (H&N) radiotherapy (RT) treatment. These variations may result in underdosage to the target volume or overdosage to the organ at risk. Replanning during the treatment course can be triggered to overcome this issue. Due to technological, methodological and clinical evolutions, tools for adaptive RT (ART) are becoming increasingly sophisticated. The aim of this paper is to give an overview of the key steps of an H&N ART workflow and tools from the point of view of a group of French-speaking medical physicists and physicians (from GORTEC). Focuses are made on image registration, segmentation, estimation of the delivered dose of the day, workflow and quality assurance for an implementation of H&N offline and online ART. Practical recommendations are given to assist physicians and medical physicists in a clinical workflow.

Comparing 3D, 2.5D, and 2D Approaches to Brain Image Auto-Segmentation

Deep-learning methods for auto-segmenting brain images either segment one slice of the image (2D), five consecutive slices of the image (2.5D), or an entire volume of the image (3D). Whether one approach is superior for auto-segmenting brain images is not known. We compared these three approaches (3D, 2.5D, and 2D) across three auto-segmentation models (capsule networks, UNets, and nnUNets) to segment brain structures. We used 3430 brain MRIs, acquired in a multi-institutional study, to train and test our models. We used the following performance metrics: segmentation accuracy, performance with limited training data, required computational memory, and computational speed during training and deployment. The 3D, 2.5D, and 2D approaches respectively gave the highest to lowest Dice scores across all models. 3D models maintained higher Dice scores when the training set size was decreased from 3199 MRIs down to 60 MRIs. 3D models converged 20% to 40% faster during training and were 30% to 50% faster during deployment. However, 3D models require 20 times more computational memory compared to 2.5D or 2D models. This study showed that 3D models are more accurate, maintain better performance with limited training data, and are faster to train and deploy. However, 3D models require more computational memory compared to 2.5D or 2D models.