Evaluation of the Block Matching deformable registration algorithm in the field of head-and-neck adaptive radiotherapy

Gross failure rates and failure modes for a commercial AI-based auto-segmentation algorithm in head and neck cancer patients

PURPOSE

Artificial intelligence (AI) based commercial software can be used to automatically delineate organs at risk (OAR), with potential for efficiency savings in the radiotherapy treatment planning pathway, and reduction of inter- and intra-observer variability. There has been little research investigating gross failure rates and failure modes of such systems.

METHOD

50 head and neck (H&N) patient data sets with "gold standard" contours were compared to AI-generated contours to produce expected mean and standard deviation values for the Dice Similarity Coefficient (DSC), for four common H&N OARs (brainstem, mandible, left and right parotid). An AI-based commercial system was applied to 500 H&N patients. AI-generated contours were compared to manual contours, outlined by an expert human, and a gross failure was set at three standard deviations below the expected mean DSC. Failures were inspected to assess reason for failure of the AI-based system with failures relating to suboptimal manual contouring censored. True failures were classified into 4 sub-types (setup position, anatomy, image artefacts and unknown).

RESULTS

There were 24 true failures of the AI-based commercial software, a gross failure rate of 1.2%. Fifteen failures were due to patient anatomy, four were due to dental image artefacts, three were due to patient position and two were unknown. True failure rates by OAR were 0.4% (brainstem), 2.2% (mandible), 1.4% (left parotid) and 0.8% (right parotid).

CONCLUSION

True failures of the AI-based system were predominantly associated with a non-standard element within the CT scan. It is likely that these non-standard elements were the reason for the gross failure, and suggests that patient datasets used to train the AI model did not contain sufficient heterogeneity of data. Regardless of the reasons for failure, the true failure rate for the AI-based system in the H&N region for the OARs investigated was low (∼1%).

Improved accuracy of auto-segmentation of organs at risk in radiotherapy planning for nasopharyngeal carcinoma based on fully convolutional neural network deep learning

OBJECTIVE

We examined a modified encoder-decoder architecture-based fully convolutional neural network, OrganNet, for simultaneous auto-segmentation of 24 organs at risk (OARs) in the head and neck, followed by validation tests and evaluation of clinical application.

MATERIALS AND METHODS

Computed tomography (CT) images from 310 radiotherapy plans were used as the experimental data set, of which 260 and 50 were used as the training and test sets, respectively. An improved U-Net architecture was established by introducing a batch normalization layer, residual squeeze-and-excitation layer, and unique organ-specific loss function for deep learning training. The performance of the trained network model was evaluated by comparing the manual-delineation and the STAPLE contour of 10 physicians from different centers.

RESULTS

Our model achieved good segmentation in all 24 OARs in nasopharyngeal cancer radiotherapy plan CT images, with an average Dice similarity coefficient of 83.75%. Specifically, the mean Dice coefficients in large-volume organs (brainstem, spinal cord, left/right parotid glands, left/right temporal lobes, and left/right mandibles) were 84.97% - 95.00%, and in small-volume organs (pituitary, lens, optic nerve, and optic chiasma) were 55.46% - 91.56%. respectively. Using the STAPLE contours as standard contour, the OrganNet achieved comparable or better DICE in organ segmentation then that of the manual-delineation as well.

CONCLUSION

The established OrganNet enables simultaneous automatic segmentation of multiple targets on CT images of the head and neck radiotherapy plans, effectively improves the accuracy of U-Net based segmentation for OARs, especially for small-volume organs.

Evaluation of DIR algorithm performance in real patients for radiotherapy treatments: A systematic review of operator-dependent strategies

PURPOSE

The performance of deformable medical image registration (DIR) algorithms has become a major concern.

METHODS

We aimed to obtain updated information on DIR algorithm performance quantification through a literature review of articles published between 2010 and 2022. We focused only on studies using operator-based methods to treat real patients. The PubMed, Google Scholar and Embase databases were searched following PRISMA guidelines.

RESULTS

One hundred and seven articles were identified. The mean number of patients and registrations per publication was 20 and 63, respectively. We found 23 different geometric metrics appearing at least twice, and the dosimetric impact of DIR was quantified in 32 articles. Forty-eight different at-risk organs were described, and target volumes were studied in 43 publications. Prostate, head-and-neck and thoracic locations represented more than ¾ of the studied locations. We summarized the type of DIR and the images used, and other key elements. Intra/interobserver variability, threshold values and the correlation between metrics were also discussed.

CONCLUSIONS

This literature review covers the past decade and should facilitate the implementation of DIR algorithms in clinical practice by providing practical and pertinent information to quantify their performance on real patients.

Estimating the accumulative dose uncertainty for intracavitary and interstitial brachytherapy

BACKGROUND

Image-guided adaptive brachytherapy shows the ability to deliver high doses to tumors while sparing normal tissues. However, interfraction dose delivery introduces uncertainties to high dose estimation, which relates to normal tissue toxicity. The purpose of this study was to investigate the high-dose regions of two applicator approaches in brachytherapy.

METHOD

For 32 cervical cancer patients, the CT images from each fraction were wrapped to a reference image, and the displacement vector field (DVF) was calculated with a hybrid intensity-based deformable registration algorithm. The fractional dose was then accumulated to calculate the position and the overlap of high dose (D2cc) during multiple fractions.

RESULT

The overall Dice similarity coefficient (DSC) of the deformation algorithm for the bladder and the rectum was (0.97 and 0.91). No significant difference was observed between the two applicators. However, the location of the intracavitary brachytherapy (ICBT) high-dose region was relatively concentrated. The overlap volume of bladder and rectum D2cc was 0.42 and 0.71, respectively, which was higher than that of interstitial brachytherapy (ISBT) (0.26 and 0.31). The cumulative dose was overestimated in ISBT cases when using the GEC-recommended method. The ratio of bladder and rectum D2cc to the GEC method was 0.99 and 1, respectively, which was higher than that of the ISBT method (0.96 and 0.94).

CONCLUSION

High-dose regions for brachytherapy based on different applicator types were different. The 3D-printed ICBT has better high-dose region consistency than freehand ISBT and hence is more predictable.

Comprehensive evaluation of ten deformable image registration algorithms for contour propagation between CT and cone-beam CT images in adaptive head & neck radiotherapy

Deformable image registration (DIR) is a critical technic in adaptive radiotherapy (ART) for propagating contours between planning computerized tomography (CT) images and treatment CT/cone-beam CT (CBCT) images to account for organ deformation for treatment re-planning. To validate the ability and accuracy of DIR algorithms in organ at risk (OAR) contour mapping, ten intensity-based DIR strategies, which were classified into four categories—optical flow-based, demons-based, level-set-based and spline-based—were tested on planning CT and fractional CBCT images acquired from twenty-one head & neck (H&N) cancer patients who underwent 6~7-week intensity-modulated radiation therapy (IMRT). Three similarity metrics, i.e., the Dice similarity coefficient (DSC), the percentage error (PE) and the Hausdorff distance (HD), were employed to measure the agreement between the propagated contours and the physician-delineated ground truths of four OARs, including the vertebra (VTB), the vertebral foramen (VF), the parotid gland (PG) and the submandibular gland (SMG). It was found that the evaluated DIRs in this work did not necessarily outperform rigid registration. DIR performed better for bony structures than soft-tissue organs, and the DIR performance tended to vary for different ROIs with different degrees of deformation as the treatment proceeded. Generally, the optical flow-based DIR performed best, while the demons-based DIR usually ranked last except for a modified demons-based DISC used for CT-CBCT DIR. These experimental results suggest that the choice of a specific DIR algorithm depends on the image modality, anatomic site, magnitude of deformation and application. Therefore, careful examinations and modifications are required before accepting the auto-propagated contours, especially for automatic re-planning ART systems.

Methodology for analysis and reporting patterns of failure in the Era of IMRT: head and neck cancer applications

Background

The aim of this study is to develop a methodology to standardize the analysis and reporting of the patterns of loco-regional failure after IMRT of head and neck cancer.

Material and Methods

Twenty-one patients with evidence of local and/or regional failure following IMRT for head-and-neck cancer were retrospectively reviewed under approved IRB protocol. Manually delineated recurrent gross disease (rGTV) on the diagnostic CT documenting recurrence (rCT) was co-registered with the original planning CT (pCT) using both deformable (DIR) and rigid (RIR) image registration software. Subsequently, mapped rGTVs were compared relative to original planning target volumes (TVs) and dose using a centroid-based approaches. Failures were then classified into five types based on combined spatial and dosimetric criteria; A (central high dose), B (peripheral high dose), C (central elective dose), D (peripheral elective dose), and E (extraneous dose).

Results

A total of 26 recurrences were identified. Using DIR, recurrences were assigned to more central TVs compared to RIR as detected using the spatial centroid-based method (p = 0.0002). rGTVs mapped using DIR had statistically significant higher mean doses when compared to rGTVs mapped rigidly (mean dose 70 vs. 69 Gy, p = 0.03). According to the proposed classification 22 out of 26 failures were of type A (central high dose) as assessed by DIR method compared to 18 out of 26 for the RIR because of the tendencey of RIR to assign failures more peripherally.

Conclusions

RIR tends to assigns failures more peripherally. DIR-based methods showed that the vast majority of failures originated in the high dose target volumes and received full prescribed doses suggesting biological rather than technology-related causes of failure. Validated DIR-based registration is recommended for accurate failure characterization and a novel typology-indicative taxonomy is recommended for failure reporting in the IMRT era.

Electronic supplementary material

The online version of this article (doi:10.1186/s13014-016-0678-7) contains supplementary material, which is available to authorized users.

The suitability of common metrics for assessing parotid and larynx autosegmentation accuracy

Contouring structures in the head and neck is time‐consuming, and automatic segmentation is an important part of an adaptive radiotherapy workflow. Geometric accuracy of automatic segmentation algorithms has been widely reported, but there is no consensus as to which metrics provide clinically meaningful results. This study investigated whether geometric accuracy (as quantified by several commonly used metrics) was associated with dosimetric differences for the parotid and larynx, comparing automatically generated contours against manually drawn ground truth contours. This enabled the suitability of different commonly used metrics to be assessed for measuring automatic segmentation accuracy of the parotid and larynx. Parotid and larynx structures for 10 head and neck patients were outlined by five clinicians to create ground truth structures. An automatic segmentation algorithm was used to create automatically generated normal structures, which were then used to create volumetric‐modulated arc therapy plans. The mean doses to the automatically generated structures were compared with those of the corresponding ground truth structures, and the relative difference in mean dose was calculated for each structure. It was found that this difference did not correlate with the geometric accuracy provided by several metrics, notably the Dice similarity coefficient, which is a commonly used measure of spatial overlap. Surface‐based metrics provided stronger correlation and are, therefore, more suitable for assessing automatic segmentation of the parotid and larynx.

PACS number(s): 87.57.nm, 87.55.D, 87.55.Qr

A support vector machine tool for adaptive tomotherapy treatments: Prediction of head and neck patients criticalities

PURPOSE

Adaptive radiation therapy (ART) is an advanced field of radiation oncology. Image-guided radiation therapy (IGRT) methods can support daily setup and assess anatomical variations during therapy, which could prevent incorrect dose distribution and unexpected toxicities. A re-planning to correct these anatomical variations should be done daily/weekly, but to be applicable to a large number of patients, still require time consumption and resources. Using unsupervised machine learning on retrospective data, we have developed a predictive network, to identify patients that would benefit of a re-planning.

METHODS

1200 MVCT of 40 head and neck (H&N) cases were re-contoured, automatically, using deformable hybrid registration and structures mapping. Deformable algorithm and MATLAB(®) homemade machine learning process, developed, allow prediction of criticalities for Tomotherapy treatments.

RESULTS

Using retrospective analysis of H&N treatments, we have investigated and predicted tumor shrinkage and organ at risk (OAR) deformations. Support vector machine (SVM) and cluster analysis have identified cases or treatment sessions with potential criticalities, based on dose and volume discrepancies between fractions. During 1st weeks of treatment, 84% of patients shown an output comparable to average standard radiation treatment behavior. Starting from the 4th week, significant morpho-dosimetric changes affect 77% of patients, suggesting need for re-planning. The comparison of treatment delivered and ART simulation was carried out with receiver operating characteristic (ROC) curves, showing monotonous increase of ROC area.

CONCLUSIONS

Warping methods, supported by daily image analysis and predictive tools, can improve personalization and monitoring of each treatment, thereby minimizing anatomic and dosimetric divergences from initial constraints.

Quality assurance assessment of diagnostic and radiation therapy-simulation CT image registration for head and neck radiation therapy: anatomic region of interest-based comparison of rigid and deformable algorithms

PURPOSE

To develop a quality assurance (QA) workflow by using a robust, curated, manually segmented anatomic region-of-interest (ROI) library as a benchmark for quantitative assessment of different image registration techniques used for head and neck radiation therapy-simulation computed tomography (CT) with diagnostic CT coregistration.

MATERIALS AND METHODS

Radiation therapy-simulation CT images and diagnostic CT images in 20 patients with head and neck squamous cell carcinoma treated with curative-intent intensity-modulated radiation therapy between August 2011 and May 2012 were retrospectively retrieved with institutional review board approval. Sixty-eight reference anatomic ROIs with gross tumor and nodal targets were then manually contoured on images from each examination. Diagnostic CT images were registered with simulation CT images rigidly and by using four deformable image registration (DIR) algorithms: atlas based, B-spline, demons, and optical flow. The resultant deformed ROIs were compared with manually contoured reference ROIs by using similarity coefficient metrics (ie, Dice similarity coefficient) and surface distance metrics (ie, 95% maximum Hausdorff distance). The nonparametric Steel test with control was used to compare different DIR algorithms with rigid image registration (RIR) by using the post hoc Wilcoxon signed-rank test for stratified metric comparison.

RESULTS

A total of 2720 anatomic and 50 tumor and nodal ROIs were delineated. All DIR algorithms showed improved performance over RIR for anatomic and target ROI conformance, as shown for most comparison metrics (Steel test, P < .008 after Bonferroni correction). The performance of different algorithms varied substantially with stratification by specific anatomic structures or category and simulation CT section thickness.

CONCLUSION

Development of a formal ROI-based QA workflow for registration assessment demonstrated improved performance with DIR techniques over RIR. After QA, DIR implementation should be the standard for head and neck diagnostic CT and simulation CT allineation, especially for target delineation.