Deep learning model for automatic contouring of cardiovascular substructures on radiotherapy planning CT images: Dosimetric validation and reader study based clinical acceptability testing

Dosimetric Impact of Delineation and Motion Uncertainties on the Heart and Substructures in Lung Cancer Radiotherapy

AIMS

Delineation variations and organ motion produce difficult-to-quantify uncertainties in planned radiation doses to targets and organs at risk. Similar to manual contouring, most automatic segmentation tools generate single delineations per structure; however, this does not indicate the range of clinically acceptable delineations. This study develops a method to generate a range of automatic cardiac structure segmentations, incorporating motion and delineation uncertainty, and evaluates the dosimetric impact in lung cancer.

MATERIALS AND METHODS

Eighteen cardiac structures were delineated using a locally developed auto-segmentation tool. It was applied to lung cancer planning CTs for 27 curative (planned dose ≥50 Gy) cases, and delineation variations were estimated by using ten mapping-atlases to provide separate substructure segmentations. Motion-related cardiac segmentation variations were estimated by auto-contouring structures on ten respiratory phases for 9/27 cases that had 4D-planning CTs. Dose volume histograms (DVHs) incorporating these variations were generated for comparison.

RESULTS

Variations in mean doses (Dmean), defined as the range in values across ten feasible auto-segmentations, were calculated for each cardiac substructure. Over the study cohort the median variations for delineation uncertainty and motion were 2.20-11.09 Gy and 0.72-4.06 Gy, respectively. As relative values, variations in Dmean were between 18.7%-65.3% and 7.8%-32.5% for delineation uncertainty and motion, respectively. Doses vary depending on the individual planned dose distribution, not simply on segmentation differences, with larger dose variations to cardiac structures lying within areas of steep dose gradient.

CONCLUSION

Radiotherapy dose uncertainties from delineation variations and respiratory-related heart motion were quantified using a cardiac substructure automatic segmentation tool. This predicts the 'dose range' where doses to structures are most likely to fall, rather than single DVH curves. This enables consideration of these uncertainties in cardiotoxicity research and for future plan optimisation. The tool was designed for cardiac structures, but similar methods are potentially applicable to other OARs.

Cardiac substructure delineation in radiation therapy - A state-of-the-art review

Delineation of cardiac substructures is crucial for a better understanding of radiation-related cardiotoxicities and to facilitate accurate and precise cardiac dose calculation for developing and applying risk models. This review examines recent advancements in cardiac substructure delineation in the radiation therapy (RT) context, aiming to provide a comprehensive overview of the current level of knowledge, challenges and future directions in this evolving field. Imaging used for RT planning presents challenges in reliably visualising cardiac anatomy. Although cardiac atlases and contouring guidelines aid in standardisation and reduction of variability, significant uncertainties remain in defining cardiac anatomy. Coupled with the inherent complexity of the heart, this necessitates auto-contouring for consistent large-scale data analysis and improved efficiency in prospective applications. Auto-contouring models, developed primarily for breast and lung cancer RT, have demonstrated performance comparable to manual contouring, marking a significant milestone in the evolution of cardiac delineation practices. Nevertheless, several key concerns require further investigation. There is an unmet need for expanding cardiac auto-contouring models to encompass a broader range of cancer sites. A shift in focus is needed from ensuring accuracy to enhancing the robustness and accessibility of auto-contouring models. Addressing these challenges is paramount for the integration of cardiac substructure delineation and associated risk models into routine clinical practice, thereby improving the safety of RT for future cancer patients.

How to contour the different heart subregions for future deep-learning modeling of the heart: A practical pictorial proposal for radiation oncologists

There are currently no accurate rules for manually delineating the subregions of the heart (cavities, vessels, aortic/mitral valves, Planning organ at Risk Volumes for coronary arteries) with the perspective of deep-learning based modeling. Our objective was to present a practical pictorial view for radiation oncologists, based on the RTOG atlas and anatomical complementary considerations for the cases where the RTOG guidelines are missing.

An open source auto-segmentation algorithm for delineating heart and substructures - Development and validation within a multicenter lung cancer cohort

BACKGROUND AND PURPOSE

Irradiation of the heart in thoracic cancers raises toxicity concerns. For accurate dose estimation, automated heart and substructure segmentation is potentially useful. In this study, a hybrid automatic segmentation is developed. The accuracy of delineation and dose predictions were evaluated, testing the method's potential within heart toxicity studies.

MATERIALS AND METHODS

The hybrid segmentation method delineated the heart, four chambers, three large vessels, and the coronary arteries. The method consisted of a nnU-net heart segmentation and partly atlas- and model-based segmentation of the substructures. The nnU-net training and atlas segmentation was based on lung cancer patients and was validated against a national consensus dataset of 12 patients with breast cancer. The accuracy of dose predictions between manual and auto-segmented heart and substructures was evaluated by transferring the dose distribution of 240 previously treated lung cancer patients to the consensus data set.

RESULTS

The hybrid auto-segmentation method performed well with a heart dice similarity coefficient (DSC) of 0.95, with no statistically significant difference between the automatic and manual delineations. The DSC for the chambers varied from 0.78-0.86 for the automatic segmentation and was comparable with the inter-observer variability. Most importantly, the automatic segmentation was as precise as the clinical experts in predicting the dose distribution to the heart and all substructures.

CONCLUSION

The hybrid segmentation method performed well in delineating the heart and substructures. The prediction of dose by the automatic segmentation was aligned with the manual delineations, enabling measurement of heart and substructure dose in large cohorts. The delineation algorithm will be available for download.

Estimating how contouring differences affect normal tissue complication probability modelling

Background and purpose

Normal tissue complication probability (NTCP) models are developed from large retrospective datasets where automatic contouring is often used to contour the organs at risk. This study proposes a methodology to estimate how discrepancies between two sets of contours are reflected on NTCP model performance. We apply this methodology to heart contours within a dataset of non-small cell lung cancer (NSCLC) patients.

Materials and methods

One of the contour sets is designated the ground truth and a dosimetric parameter derived from it is used to simulate outcomes via a predefined NTCP relationship. For each simulated outcome, the selected dosimetric parameters associated with each contour set are individually used to fit a toxicity model and their performance is compared. Our dataset comprised 605 stage IIA-IIIB NSCLC patients. Manual, deep learning, and atlas-based heart contours were available.

Results

How contour differences were reflected in NTCP model performance depended on the slope of the predefined model, the dosimetric parameter utilized, and the size of the cohort. The impact of contour differences on NTCP model performance increased with steeper NTCP curves. In our dataset, parameters on the low range of the dose-volume histogram were more robust to contour differences.

Conclusions

Our methodology can be used to estimate whether a given contouring model is fit for NTCP model development. For the heart in comparable datasets, average Dice should be at least as high as between our manual and deep learning contours for shallow NTCP relationships (88.5 ± 4.5 %) and higher for steep relationships.

Attention-Gated Deep-Learning-Based Automatic Digitization of Interstitial Needles in High-Dose-Rate Brachytherapy for Cervical Cancer

Purpose

Deep learning can be used to automatically digitize interstitial needles in high-dose-rate (HDR) brachytherapy for patients with cervical cancer. The aim of this study was to design a novel attention-gated deep-learning model, which may further improve the accuracy of and better differentiate needles.

Methods and Materials

Seventeen patients with cervical cancer with 56 computed tomography-based interstitial HDR brachytherapy plans from the local hospital were retrospectively chosen with the local institutional review board's approval. Among them, 50 plans were randomly selected as the training set and the rest as the validation set. Spatial and channel attention gates (AGs) were added to 3-dimensional convolutional neural networks (CNNs) to highlight needle features and suppress irrelevant regions; this was supposed to facilitate convergence and improve accuracy of automatic needle digitization. Subsequently, the automatically digitized needles were exported to the Oncentra treatment planning system (Elekta Solutions AB, Stockholm, Sweden) for dose evaluation. The geometric and dosimetric accuracy of automatic needle digitization was compared among 3 methods: (1) clinically approved plans with manual needle digitization (ground truth); (2) the conventional deep-learning (CNN) method; and (3) the attention-added deep-learning (CNN + AG) method, in terms of the Dice similarity coefficient (DSC), tip and shaft positioning errors, dose distribution in the high-risk clinical target volume (HR-CTV), organs at risk, and so on.

Results

The attention-gated CNN model was superior to CNN without AGs, with a greater DSC (approximately 94% for CNN + AG vs 89% for CNN). The needle tip and shaft errors of the CNN + AG method (1.1 mm and 1.8 mm, respectively) were also much smaller than those of the CNN method (2.0 mm and 3.3 mm, respectively). Finally, the dose difference for the HR-CTV D90 using the CNN + AG method was much more accurate than that using CNN (0.4% and 1.7%, respectively).

Conclusions

The attention-added deep-learning model was successfully implemented for automatic needle digitization in HDR brachytherapy, with clinically acceptable geometric and dosimetric accuracy. Compared with conventional deep-learning neural networks, attention-gated deep learning may have superior performance and great clinical potential.

Assessment of heart-substructures auto-contouring accuracy for application in heart-sparing radiotherapy for lung cancer

Objectives

We validated an auto-contouring algorithm for heart substructures in lung cancer patients, aiming to establish its accuracy and reliability for radiotherapy (RT) planning. We focus on contouring an amalgamated set of subregions in the base of the heart considered to be a new organ at risk, the cardiac avoidance area (CAA), to enable maximum dose limit implementation in lung RT planning.

Methods

The study validates a deep-learning model specifically adapted for auto-contouring the CAA (which includes the right atrium, aortic valve root, and proximal segments of the left and right coronary arteries). Geometric, dosimetric, quantitative, and qualitative validation measures are reported. Comparison with manual contours, including assessment of interobserver variability, and robustness testing over 198 cases are also conducted.

Results

Geometric validation shows that auto-contouring performance lies within the expected range of manual observer variability despite being slightly poorer than the average of manual observers (mean surface distance for CAA of 1.6 vs 1.2 mm, dice similarity coefficient of 0.86 vs 0.88). Dosimetric validation demonstrates consistency between plans optimized using auto-contours and manual contours. Robustness testing confirms acceptable contours in all cases, with 80% rated as "Good" and the remaining 20% as "Useful."

Conclusions

The auto-contouring algorithm for heart substructures in lung cancer patients demonstrates acceptable and comparable performance to human observers.

Advances in knowledge

Accurate and reliable auto-contouring results for the CAA facilitate the implementation of a maximum dose limit to this region in lung RT planning, which has now been introduced in the routine setting at our institution.

A deep-learning model using enhanced chest CT images to predict PD-L1 expression in non-small-cell lung cancer patients

AIM

To develop a deep-learning model using contrast-enhanced chest computed tomography (CT) images to predict programmed death-ligand 1 (PD-L1) expression in patients with non-small-cell lung cancer (NSCLC).

MATERIALS AND METHODS

Preoperative enhanced chest CT images and immunohistochemistry results for PD-L1 expression (<1% and ≥1% were defined as negative and positive, respectively) were collected retrospectively from 125 NSCLC patients to train and validate a deep-learning radiomics model (DLRM) for the prediction of PD-L1 expression in tumours. The DLRM was developed by combining the deep-learning signature (DLS) obtained from a convolutional neural network and clinicopathological factors. The indexes of the area under the curve (AUC), integrated discrimination improvement (IDI), and decision curve analysis (DCA) were used to evaluate the efficiency of the DLRM.

RESULTS

DLS and tumour stage were identified as independent predictors of PD-L1 expression by the DLRM. The AUCs of the DLRM were 0.804 (95% confidence interval: 0.697-0.911) and 0.804 (95% confidence interval: 0.679-0.929) in the training and validation cohorts, respectively. IDI analysis showed the DLRM had better diagnostic accuracy than DLS (0.0028 [p<0.05]) in the validation cohort. Additionally, DCA revealed that the DLRM had more net benefit than the DLS for clinical utility.

CONCLUSION

The proposed DLRM using enhanced chest CT images could function as a non-invasive diagnostic tool to differentiate PD-L1 expression in NSCLC patients.

An investigation into the risk of population bias in deep learning autocontouring

BACKGROUND AND PURPOSE

To date, data used in the development of Deep Learning-based automatic contouring (DLC) algorithms have been largely sourced from single geographic populations. This study aimed to evaluate the risk of population-based bias by determining whether the performance of an autocontouring system is impacted by geographic population.

MATERIALS AND METHODS

80 Head Neck CT deidentified scans were collected from four clinics in Europe (n = 2) and Asia (n = 2). A single observer manually delineated 16 organs-at-risk in each. Subsequently, the data was contoured using a DLC solution, and trained using single institution (European) data. Autocontours were compared to manual delineations using quantitative measures. A Kruskal-Wallis test was used to test for any difference between populations. Clinical acceptability of automatic and manual contours to observers from each participating institution was assessed using a blinded subjective evaluation.

RESULTS

Seven organs showed a significant difference in volume between groups. Four organs showed statistical differences in quantitative similarity measures. The qualitative test showed greater variation in acceptance of contouring between observers than between data from different origins, with greater acceptance by the South Korean observers.

CONCLUSION

Much of the statistical difference in quantitative performance could be explained by the difference in organ volume impacting the contour similarity measures and the small sample size. However, the qualitative assessment suggests that observer perception bias has a greater impact on the apparent clinical acceptability than quantitatively observed differences. This investigation of potential geographic bias should extend to more patients, populations, and anatomical regions in the future.

Radiation therapy in the thoracic region: Radio-induced cardiovascular disease, cardiac delineation and sparing, cardiac dose constraints, and cardiac implantable electronic devices

Radiation therapy in the thoracic region may deliver incidental ionizing radiation to the surrounding healthy structures, including the heart. Radio-induced heart toxicity has long been a concern in breast cancer and Hodgkin's lymphoma and was deemed a long-term event. However, recent data highlight the need to limit the dose to the heart in less favorable thoracic cancers too, such as lung and esophageal cancers in which incidental irradiation led to increased mortality. This article will summarize available cardiac dose constraints in various clinical settings and the types of radio-induced cardiovascular diseases encountered as well as delineation of cardiac subheadings and management of cardiac devices. Although still not completely deciphered, heart dose constraints remain intensively investigated and the mean dose to the heart is no longer the only dosimetric parameter to consider since the left anterior descending artery as well as the left ventricle should also be part of dosimetry constraints.

Validation of a Fully Automated Hybrid Deep Learning Cardiac Substructure Segmentation Tool for Contouring and Dose Evaluation in Lung Cancer Radiotherapy

BACKGROUND AND PURPOSE

Accurate and consistent delineation of cardiac substructures is challenging. The aim of this work was to validate a novel segmentation tool for automatic delineation of cardiac structures and subsequent dose evaluation, with potential application in clinical settings and large-scale radiation-related cardiotoxicity studies.

MATERIALS AND METHODS

A recently developed hybrid method for automatic segmentation of 18 cardiac structures, combining deep learning, multi-atlas mapping and geometric segmentation of small challenging substructures, was independently validated on 30 lung cancer cases. These included anatomical and imaging variations, such as tumour abutting heart, lung collapse and metal artefacts. Automatic segmentations were compared with manual contours of the 18 structures using quantitative metrics, including Dice similarity coefficient (DSC), mean distance to agreement (MDA) and dose comparisons.

RESULTS

A comparison of manual and automatic contours across all cases showed a median DSC of 0.75-0.93 and a median MDA of 2.09-3.34 mm for whole heart and chambers. The median MDA for great vessels, coronary arteries, cardiac valves, sinoatrial and atrioventricular conduction nodes was 3.01-8.54 mm. For the 27 cases treated with curative intent (planned target volume dose ≥50 Gy), the median dose difference was -1.12 to 0.57 Gy (absolute difference of 1.13-3.25%) for the mean dose to heart and chambers; and -2.25 to 4.45 Gy (absolute difference of 0.94-6.79%) for the mean dose to substructures.

CONCLUSION

The novel hybrid automatic segmentation tool reported high accuracy and consistency over a validation set with challenging anatomical and imaging variations. This has promising applications in substructure dose calculations of large-scale datasets and for future studies on long-term cardiac toxicity.

Open-source, fully-automated hybrid cardiac substructure segmentation: development and optimisation

Radiotherapy for thoracic and breast tumours is associated with a range of cardiotoxicities. Emerging evidence suggests cardiac substructure doses may be more predictive of specific outcomes, however, quantitative data necessary to develop clinical planning constraints is lacking. Retrospective analysis of patient data is required, which relies on accurate segmentation of cardiac substructures. In this study, a novel model was designed to deliver reliable, accurate, and anatomically consistent segmentation of 18 cardiac substructures on computed tomography (CT) scans. Thirty manually contoured CT scans were included. The proposed multi-stage method leverages deep learning (DL), multi-atlas mapping, and geometric modelling to automatically segment the whole heart, cardiac chambers, great vessels, heart valves, coronary arteries, and conduction nodes. Segmentation performance was evaluated using the Dice similarity coefficient (DSC), mean distance to agreement (MDA), Hausdorff distance (HD), and volume ratio. Performance was reliable, with no errors observed and acceptable variation in accuracy between cases, including in challenging cases with imaging artefacts and atypical patient anatomy. The median DSC range was 0.81-0.93 for whole heart and cardiac chambers, 0.43-0.76 for great vessels and conduction nodes, and 0.22-0.53 for heart valves. For all structures the median MDA was below 6 mm, median HD ranged 7.7-19.7 mm, and median volume ratio was close to one (0.95-1.49) for all structures except the left main coronary artery (2.07). The fully automatic algorithm takes between 9 and 23 min per case. The proposed fully-automatic method accurately delineates cardiac substructures on radiotherapy planning CT scans. Robust and anatomically consistent segmentations, particularly for smaller structures, represents a major advantage of the proposed segmentation approach. The open-source software will facilitate more precise evaluation of cardiac doses and risks from available clinical datasets.

A Review of the Metrics Used to Assess Auto-Contouring Systems in Radiotherapy

Auto-contouring could revolutionise future planning of radiotherapy treatment. The lack of consensus on how to assess and validate auto-contouring systems currently limits clinical use. This review formally quantifies the assessment metrics used in studies published during one calendar year and assesses the need for standardised practice. A PubMed literature search was undertaken for papers evaluating radiotherapy auto-contouring published during 2021. Papers were assessed for types of metric and the methodology used to generate ground-truth comparators. Our PubMed search identified 212 studies, of which 117 met the criteria for clinical review. Geometric assessment metrics were used in 116 of 117 studies (99.1%). This includes the Dice Similarity Coefficient used in 113 (96.6%) studies. Clinically relevant metrics, such as qualitative, dosimetric and time-saving metrics, were less frequently used in 22 (18.8%), 27 (23.1%) and 18 (15.4%) of 117 studies, respectively. There was heterogeneity within each category of metric. Over 90 different names for geometric measures were used. Methods for qualitative assessment were different in all but two papers. Variation existed in the methods used to generate radiotherapy plans for dosimetric assessment. Consideration of editing time was only given in 11 (9.4%) papers. A single manual contour as a ground-truth comparator was used in 65 (55.6%) studies. Only 31 (26.5%) studies compared auto-contours to usual inter- and/or intra-observer variation. In conclusion, significant variation exists in how research papers currently assess the accuracy of automatically generated contours. Geometric measures are the most popular, however their clinical utility is unknown. There is heterogeneity in the methods used to perform clinical assessment. Considering the different stages of system implementation may provide a framework to decide the most appropriate metrics. This analysis supports the need for a consensus on the clinical implementation of auto-contouring.

The impact of inter-observer variation in delineation on robustness of radiomics features in non-small cell lung cancer

Artificial intelligence and radiomics have the potential to revolutionise cancer prognostication and personalised treatment. Manual outlining of the tumour volume for extraction of radiomics features (RF) is a subjective process. This study investigates robustness of RF to inter-observer variation (IOV) in contouring in lung cancer. We utilised two public imaging datasets: ‘NSCLC-Radiomics’ and ‘NSCLC-Radiomics-Interobserver1’ (‘Interobserver’). For ‘NSCLC-Radiomics’, we created an additional set of manual contours for 92 patients, and for ‘Interobserver’, there were five manual and five semi-automated contours available for 20 patients. Dice coefficients (DC) were calculated for contours. 1113 RF were extracted including shape, first order and texture features. Intraclass correlation coefficient (ICC) was computed to assess robustness of RF to IOV. Cox regression analysis for overall survival (OS) was performed with a previously published radiomics signature. The median DC ranged from 0.81 (‘NSCLC-Radiomics’) to 0.85 (‘Interobserver’—semi-automated). The median ICC for the ‘NSCLC-Radiomics’, ‘Interobserver’ (manual) and ‘Interobserver’ (semi-automated) were 0.90, 0.88 and 0.93 respectively. The ICC varied by feature type and was lower for first order and gray level co-occurrence matrix (GLCM) features. Shape features had a lower median ICC in the ‘NSCLC-Radiomics’ dataset compared to the ‘Interobserver’ dataset. Survival analysis showed similar separation of curves for three of four RF apart from ‘original_shape_Compactness2’, a feature with low ICC (0.61). The majority of RF are robust to IOV, with first order, GLCM and shape features being the least robust. Semi-automated contouring improves feature stability. Decreased robustness of a feature is significant as it may impact upon the features’ prognostic capability.

Validation of an established deep learning auto-segmentation tool for cardiac substructures in 4D radiotherapy planning scans

•

Cardiotoxicity is a common complication of lung cancer radiotherapy.

•

Segmentation of cardiac substructures is time-consuming and challenging.

•

Deep learning segmentation tools can perform this task in 3D and 4D scans.

•

Performance is high when assessed geometrically, dosimetrically and clinically.

•

Auto-segmentation tools may accelerate clinical workflows and enable research.

Background

Emerging data suggest that dose-sparing several key cardiac regions is prognostically beneficial in lung cancer radiotherapy. The cardiac substructures are challenging to contour due to their complex geometry, poor soft tissue definition on computed tomography (CT) and cardiorespiratory motion artefact. A neural network was previously trained to generate the cardiac substructures using three-dimensional radiotherapy planning CT scans (3D-CT). In this study, the performance of that tool on the average intensity projection from four-dimensional (4D) CT scans (4D-AVE), now commonly used in lung radiotherapy, was evaluated.

Materials and Methods

The 4D-AVE of n=20 patients completing radiotherapy for lung cancer 2015–2020 underwent manual and automated cardiac substructure segmentation. Manual and automated substructures were compared geometrically and dosimetrically. Two senior clinicians also qualitatively assessed the auto-segmentation tool’s output.

Results

Geometric comparison of the automated and manual segmentations exhibited high levels of similarity across parameters, including volume difference (11.8% overall) and Dice similarity coefficient (0.85 overall), and were consistent with 3D-CT performance. Differences in mean (median 0.2 Gy, range −1.6–0.3 Gy) and maximum (median 0.4 Gy, range −2.2–0.9 Gy) doses to substructures were generally small. Nearly all structures (99.5 %) were deemed to be appropriate for clinical use without further editing.

Conclusions

Cardiac substructure auto-segmentation using a deep learning-based tool trained on a 3D-CT dataset was feasible on the 4D-AVE scan, meaning this tool is suitable for use on 4D-CT radiotherapy planning scans. Application of this tool would increase the practicality of routine clinical cardiac substructure delineation, and enable further cardiac radiation effects research.