Interobserver variability of clinical target volume delineation in supra-diaphragmatic Hodgkin's disease: a multi-institutional experience

Involved-site Radiotherapy Delineation Dilemmas in the Treatment of Adult Hodgkin Lymphoma: Turkish Society for Radiation Oncology Hematological Oncology, Pediatric Oncology and TBI Study Group Case-based Questionnaire Review (TROD 03-005)

AIMS

The International Lymphoma Radiation Oncology Group (ILROG) defined involved-site radiotherapy (ISRT) guidelines. These rules offer a certain variability that allows for autonomous decision-making in diverse clinical settings. However, this flexibility also gives rise to conflicts about the selection of treatment fields in the daily decision-making process. The aim of this study was to show the extent of interobserver variability when ILROG-ISRT recommendations were used in different clinical scenarios.

MATERIALS AND METHODS

The 10-question survey used in our study consisted of two parts (part A and part B) and was prepared by four senior radiation oncologists experienced in the haemato-oncology field. The results were presented by stratifying according to clinical experience (<10 years, ≥10 years). Binomial tests (one-sided) were conducted to assess whether answers for each group and the whole group reached a consensus.

RESULTS

Twenty-six radiation oncologists, 13 of whom had less than 10 years of experience and 13 seniors, participated in the survey. Eighty per cent of respondents thought ILROG did not bring sufficient solutions for all clinical scenarios but offered solutions in some cases. In different case-based scenarios, the consensus among the respondents decreased down to 38%. Senior radiation oncologists were found to have more doubts about the adequacy of current guidelines.

CONCLUSIONS

ILROG guidelines allow for a high degree of variability in real-life clinical scenarios and different interpretation of the recommendations may lead to increased toxicity and recurrences. Therefore, there is a need for refinement in ISRT delineation strategies. On behalf of the Turkish Society for Radiation Oncology Hematological Oncology, Pediatric Oncology and TBI Study Group, we are planning to carry out further educational contouring sessions to detect the interobserver variability in real-life contouring cases.

Clinical evaluation of atlas-based auto-segmentation in breast and nodal radiotherapy

OBJECTIVES

Accurate contouring of anatomical structures allows for high-precision radiotherapy planning, targeting the dose at treatment volumes and avoiding organs at risk. Manual contouring is time-consuming with significant user variability, whereas auto-segmentation (AS) has proven efficiency benefits but requires editing before treatment planning. This study investigated whether atlas-based AS (ABAS) accuracy improves with template atlas group size and character-specific atlas and test case selection.

METHODS AND MATERIALS

One clinician retrospectively contoured the breast, nodes, lung, heart, and brachial plexus on 100 CT scans, adhering to peer-reviewed guidelines. Atlases were clustered in group sizes, treatment positions, chest wall separations, and ASs created with Mirada software. The similarity of ASs compared to reference contours was described by the Jaccard similarity coefficient (JSC) and centroid distance variance (CDV).

RESULTS

Across group sizes, for all structures combined, the mean JSC was 0.6 (SD 0.3, p = .999). Across atlas-specific groups, 0.6 (SD 0.3, p = 1.000). The correlation between JSC and structure volume was weak in both scenarios (adjusted R2-0.007 and 0.185).Mean CDV was similar across groups but varied up to 1.2 cm for specific structures.

CONCLUSIONS

Character-specific atlas groups and test case selection did not improve accuracy outcomes. High-quality ASs were obtained from groups containing as few as ten atlases, subsequently simplifying the application of ABAS. CDV measures indicating auto-segmentation variations on the x, y, and z axes can be utilised to decide on the clinical relevance of variations and reduce AS editing.

ADVANCES IN KNOWLEDGE

High-quality ABASs can be obtained from as few as ten template atlases.Atlas and test case selection do not improve AS accuracy.Unlike well-known quantitative similarity indices, volume displacement metrics provide information on the location of segmentation variations, helping assessment of the clinical relevance of variations and reducing clinician editing. Volume displacement metrics combined with the qualitative measure of clinician assessment could reduce user variability.

Risk and Response Adapted Treatment Guidelines for Managing First Relapsed and Refractory Classical Hodgkin Lymphoma in Children and Young People. Recommendations from the EuroNet Pediatric Hodgkin Lymphoma Group

The objective of this guideline is to aid clinicians in making individual salvage treatment plans for pediatric and adolescent patients with first relapse or refractory (R/R) classical Hodgkin lymphoma (cHL). While salvage with standard dose chemotherapy followed by high dose chemotherapy and autologous stem cell transplant is often considered the standard of care in adult practice, pediatric practice adopts a more individualized risk stratified and response adapted approach to salvage treatment with greater use of non-transplant salvage. Here, we present on behalf of the EuroNet Pediatric Hodgkin Lymphoma group, evidence and consensus-based guidelines for standardized diagnostic, prognostic and response procedures to allocate children and adolescents with R/R cHL to stratified salvage treatments.

Automatic image segmentation based on synthetic tissue model for delineating organs at risk in spinal metastasis treatment planning

Purpose

One of the main goals in software solutions for treatment planning is to automatize delineation of organs at risk (OARs). In this pilot feasibility study a clinical validation was made of computed tomography (CT)-based extracranial auto-segmentation (AS) using the Brainlab Anatomical Mapping tool (AM).

Methods

The delineation of nine extracranial OARs (lungs, kidneys, trachea, heart, liver, spinal cord, esophagus) from clinical datasets of 24 treated patients was retrospectively evaluated. Manual delineation of OARs was conducted in clinical routine and compared with AS datasets using AM. The Dice similarity coefficient (DSC) and maximum Hausdorff distance (HD) were used as statistical and geometrical measurements, respectively. Additionally, all AS structures were validated using a subjective qualitative scoring system.

Results

All patient datasets investigated were successfully processed with the evaluated AS software. For the left lung (0.97 ± 0.03), right lung (0.97 ± 0.05), left kidney (0.91 ± 0.07), and trachea (0.93 ± 0.04), the DSC was high with low variability. The DSC scores of other organs (right kidney, heart, liver, spinal cord), except the esophagus, ranged between 0.7 and 0.9. The calculated HD values yielded comparable results. Qualitative assessment showed a general acceptance in more than 85% of AS OARs—except for the esophagus.

Conclusions

The Brainlab AM software is ready for clinical use in most of the OARs evaluated in the thoracic and abdominal region. The software generates highly conformal structure sets compared to manual contouring. The current study design needs revision for further research.

Magnetic resonance imaging (MRI) compared with computed tomography (CT) for interobserver agreement of gross tumor volume delineation in pancreatic cancer: a multi-institutional contouring study on behalf of the AIRO group for gastrointestinal cancers

BACKGROUND

Due to the high soft tissue resolution, magnetic resonance imaging (MRI) could improve the accuracy of pancreatic tumor delineation in radiation treatment planning. A multi-institutional study was proposed to evaluate the impact of MRI on inter-observer agreement in gross tumor volume (GTV) and duodenum delineation for pancreatic cancer compared with computer tomography (CT).

MATERIAL AND METHODS

Two clinical cases of borderline resectable (Case 1) and unresectable (Case 2) pancreatic cancer were selected. In two sequential steps, diagnostic contrast-enhanced CT scan and MRI sequences were sent to the participating centers. CT-GTVs were contoured while blinded to MRI data sets. DICE index was used to evaluate the spatial overlap accuracy.

RESULTS

Thirty-one radiation oncologists from different Institutions submitted the delineated volumes. CT- and MRI-GTV mean volumes were 21.6 ± 9.0 cm³ and 17.2 ± 6.0 cm³, respectively for Case 1, and 31.3 ± 15.6 cm³ and 33.2 ± 20.2 cm³, respectively for Case 2. Resulting MRI-GTV mean volume was significantly smaller than CT-GTV in the borderline resectable case (p < .05). A substantial agreement was shown by the median DICE index for CT- and MRI-GTV resulting as 0.74 (IQR: 0.67-0.75) and 0.61 (IQR: 0.57-0.67) for Case 1; a moderate agreement was instead reported for Case 2: 0.59 (IQR:0.52-0.66) and 0.53 (IQR:0.42-0.62) for CT- and MRI-GTV, respectively.

CONCLUSION

Diagnostic MRI resulted in smaller GTV in borderline resectable case with a substantial agreement between observers, and was comparable to CT scan in interobserver variability, in both cases. The greater variability in the unresectable case underlines the critical issues related to the outlining when vascular structures are more involved. The integration of MRI with contrast-enhancement CT, thanks to its high definition of tumor relationship with neighboring vessels, could offer a greater accuracy of target delineation.

Inter-observer variability of clinical target volume delineation in definitive radiotherapy of neck lymph node metastases from unknown primary. A cooperative study of the Italian Association of Radiotherapy and Clinical Oncology (AIRO) Head and Neck Group

BACKGROUND

This study, promoted by Italian Association of Radiotherapy and Clinical Oncology (AIRO) Head and Neck Group, aimed to assess the current national practice of target volume delineation on a case of neck lymph node metastases from unknown primary evaluating inter-observer variability, in a setting of primary radiotherapy.

MATERIALS AND METHODS

A case of metastatic neck lymph node from occult primary was proposed to 17 radiation oncologists. A national reference RT center was identified and considered as benchmark. Participants were requested to delineate target volumes. A structured questionnaire was administered. A comparison between following parameters of the CTVs was performed: centroids distances, Dice similarity index (DSI), Jaccard index and mean distance to agreement (MDA). Volume expressed in cubic centimeters and CTVs cranio-caudal extension were evaluated.

RESULTS

Sixteen of 17 radiation oncologists recommended three CTVs dose levels. (CTV HD, CTV ID and CTV LD); CTV ID was not delineated by one of the participants and by the reference center. The distance between the reference centroid and the mean centroid of CTVs HD was 1.09 cm (0.36-3.99 cm); for CTV LD, a mean centroids distance of 2.45 (0.27-4.83 cm) was found, and for CTV HD, mean DSI is 0.48 and mean Jaccard index is 0.32 and MDA was 8.89 mm. CTV LD showed a mean DSI of 0.46, mean Jaccard index of 0.31 and MDA of 14.87 when compared to the reference.

CONCLUSION

Many aspects concerning treatment optimization of cervical nodes metastases from occult primary remain unclear, and we found a notable heterogeneity of global radiotherapy management reporting discordances both in target volume delineation and volume prescription.

Short interactive workshops reduce variability in contouring treatment volumes for spine stereotactic body radiation therapy: Experience with the ESTRO FALCON programme and EduCase™ training tool

We report the results of 4, 2-h contouring workshops on target volume definition for spinal stereotactic radiotherapy. They combined traditional teaching methods with a web-based contouring/contour-analysis platform and led to a significant reduction in delineation variability. Short, interactive workshops can reduce interobserver variability in spine SBRT target volume delineation.

A dose based approach for evaluation of inter-observer variations in target delineation

BACKGROUND AND PURPOSE

Substantial inter-observer variations in target delineation have been presented previously. Target delineation for paediatric cases is difficult due to the small number of children, the variation in paediatric targets, the number of study protocols, and the individual patient's specific needs and demands. Uncertainties in target delineation might lead to under-dosage or over-dosage. The aim of this work is to apply the concept of a consensus volume and good quality treatment plans to visualise and quantify inter-observer target delineation variations in dosimetric terms in addition to conventional geometrically based volume concordance indices.

MATERIAL AND METHODS

Two paediatric cases were used to demonstrate the potential of adding dose metrics when evaluating target delineation diversity; Hodgkin's disease (case 1) and rhabdomyosarcoma of the parotid gland (case 2). The variability in target delineation (PTV delineations) between six centres was quantified using the generalised conformity index, CIgen, generated for volume overlap. The STAPLE algorithm, as implemented in CERR, was used for both cases to derive a consensus volumes. STAPLE is a probabilistic estimate of the true volume generated from all observers. Dose distributions created by each centre for the original target volumes were then applied to this consensus volume.

RESULTS

A considerable variation in target segmentation was seen in both cases. For case 1 the variation was 374-960 cm3 (average 669 cm3) and for case 2; 65-126 cm3 (average 109 cm3). CIgen were 0.53 and 0.70, respectively. The DVHs in absolute volume displayed for the delineated target volume as well as for the consensus volume adds information on both "compliant" target volumes as well as outliers which are hidden with just the use of concordance indices.

CONCLUSIONS

The DVHs in absolute volume add valuable and easily understood information to various indices for evaluating uniformity in target delineation.

Regional deep hyperthermia: impact of observer variability in CT-based manual tissue segmentation on simulated temperature distribution

The aim of this study was to systematically investigate the influence of the inter- and intra-observer segmentation variation of tumors and organs at risk on the simulated temperature coverage of the target. CT scans of six patients with tumors in the pelvic region acquired for radiotherapy treatment planning were used for hyperthermia treatment planning. To study the effect of inter-observer variation, three observers manually segmented in the CT images of each patient the following structures: fat, muscle, bone and the bladder. The gross tumor volumes (GTV) were contoured by three radiation oncology residents and used as the hyperthermia target volumes. For intra-observer variation, one of the observers of each group contoured the structures of each patient three times with a time span of one week between the segmentations. Moreover, the impact of segmentation variations in organs at risk (OARs) between the three inter-observers was investigated on simulated temperature distributions using only one GTV. The spatial overlap between individual segmentations was assessed by the Dice similarity coefficient (DSC) and the mean surface distance (MSD). Additionally, the temperatures T90/T10 delivered to 90%/10% of the GTV, respectively, were assessed for each observer combination. The results of the segmentation similarity evaluation showed that the DSC of the inter-observer variation of fat, muscle, the bladder, bone and the target was 0.68  ±  0.12, 0.88  ±  0.05, 0.73  ±  0.14, 0.91  ±  0.04 and 0.64  ±  0.11, respectively. Similar results were found for the intra-observer variation. The MSD results were similar to the DSCs for both observer variations. A statistically significant difference (p  <  0.05) was found for T90 and T10 in the predicted target temperature due to the observer variability. The conclusion is that intra- and inter-observer variations have a significant impact on the temperature coverage of the target. Furthermore, OARs, such as bone and the bladder, may essentially influence the homogeneity of the simulated target temperature distribution.

Interobserver delineation uncertainty in involved-node radiation therapy (INRT) for early-stage Hodgkin lymphoma: on behalf of the Radiotherapy Committee of the EORTC lymphoma group

BACKGROUND AND PURPOSE

In early-stage classical Hodgkin lymphoma (HL) the target volume nowadays consists of the volume of the originally involved nodes. Delineation of this volume on a post-chemotherapy CT-scan is challenging. We report on the interobserver variability in target volume definition and its impact on resulting treatment plans.

MATERIALS AND METHODS

Two representative cases were selected (1: male, stage IB, localization: left axilla; 2: female, stage IIB, localizations: mediastinum and bilateral neck). Eight experienced observers individually defined the clinical target volume (CTV) using involved-node radiotherapy (INRT) as defined by the EORTC-GELA guidelines for the H10 trial. A consensus contour was generated and the standard deviation computed. We investigated the overlap between observer and consensus contour [Sørensen-Dice coefficient (DSC)] and the magnitude of gross deviations between the surfaces of the observer and consensus contour (Hausdorff distance). 3D-conformal (3D-CRT) and intensity-modulated radiotherapy (IMRT) plans were calculated for each contour in order to investigate the impact of interobserver variability on each treatment modality. Similar target coverage was enforced for all plans.

RESULTS

The median CTV was 120 cm³ (IQR: 95-173 cm³) for Case 1, and 255 cm³ (IQR: 183-293 cm³) for Case 2. DSC values were generally high (>0.7), and Hausdorff distances were about 30 mm. The SDs between all observer contours, providing an estimate of the systematic error associated with delineation uncertainty, ranged from 1.9 to 3.8 mm (median: 3.2 mm). Variations in mean dose resulting from different observer contours were small and were not higher in IMRT plans than in 3D-CRT plans.

CONCLUSIONS

We observed considerable differences in target volume delineation, but the systematic delineation uncertainty of around 3 mm is comparable to that reported in other tumour sites. This report is a first step towards calculating an evidence-based planning target volume margin for INRT in HL.

Uncertainties in volume delineation in radiation oncology: A systematic review and recommendations for future studies

BACKGROUND AND PURPOSE

Volume delineation is a well-recognised potential source of error in radiotherapy. Whilst it is important to quantify the degree of interobserver variability (IOV) in volume delineation, the resulting impact on dosimetry and clinical outcomes is a more relevant endpoint. We performed a literature review of studies evaluating IOV in target volume and organ-at-risk (OAR) delineation in order to analyse these with respect to the metrics used, reporting of dosimetric consequences, and use of statistical tests.

METHODS AND MATERIALS

Medline and Pubmed databases were queried for relevant articles using keywords. We included studies published in English between 2000 and 2014 with more than two observers.

RESULTS

119 studies were identified covering all major tumour sites. CTV (n=47) and GTV (n=38) were most commonly contoured. Median number of participants and data sets were 7 (3-50) and 9 (1-132) respectively. There was considerable heterogeneity in the use of metrics and methods of analysis. Statistical analysis of results was reported in 68% (n=81) and dosimetric consequences in 21% (n=25) of studies.

CONCLUSION

There is a lack of consistency in conducting and reporting analyses from IOV studies. We suggest a framework to use for future studies evaluating IOV.

Interobserver variability in gross tumor volume delineation for hepatocellular carcinoma : Results of Korean Radiation Oncology Group 1207 study

PURPOSE

There has been increasing use of external beam radiotherapy for localized treatment of hepatocellular carcinoma (HCC) with both palliative and curative intent. Quality control of target delineation in primary HCC is essential to deliver adequate doses of radiation to the primary tumor while preserving adjacent healthy organs. We analyzed interobserver variability in gross tumor volume (GTV) delineation for HCC.

PATIENTS AND METHODS

Twelve radiation oncologists specializing in liver malignancy participated in a multi-institutional contouring dummy-run study of nine HCC cases and independently delineated GTV on the same set of provided computed tomography images. Quantitative analysis was performed using an expectation maximization algorithm for simultaneous truth and performance level estimation (STAPLE) with kappa statistics calculating agreement between physicians. To quantify the interobserver variability of GTV delineations, the ratio of the actual delineated volume to the estimated consensus volume (STAPLE), the ratio of the common and encompassing volumes, and the coefficient of variation were calculated.

RESULTS

The median kappa agreement level was 0.71 (range 0.28-0.86). The ratio of the actual delineated volume to the estimated consensus volume ranged from 0.19 to 1.93 (median 0.94) for all cases. The ratio of the common and encompassing volumes ranged from 0.001 to 0.56 (median 0.25). The coefficient of variation for GTV delineation ranged from 8 to 57 % (median 26 %).

CONCLUSION

The interobserver variability in target delineation of HCC GTV in this study is noteworthy. Multi-institution studies involving radiotherapy for HCC require appropriate quality assurance programs for target delineation.

Interobserver variability in clinical target volume delineation for primary mediastinal B-cell lymphoma

PURPOSE

The purpose of this study was to evaluate interobserver variability among radiation oncologists with experience in the field of lymphoma radiation therapy in the delineation of clinical target volume (CTV) in a challenging case of primary mediastinal B-cell lymphoma.

METHODS AND MATERIALS

Ten experienced radiation oncologists were invited to a 1-day contouring session. The case of a 56-year-old man with primary mediastinal B-cell lymphoma with complete metabolic response after chemotherapy was chosen as the sample for the study. A brief presentation of his clinical history was given, together with guidelines for contouring. The 10 CTVs obtained were then compared in terms of variation in total volume and in craniocaudal, laterolateral, and anteroposterior diameters. The CTV with the best Dice similarity coefficient (DSC) between the union of all 10 CTVs and the individual CTV was considered the reference CTV, and the DSC and the Hausdorff distance (HD) for each volume compared with the reference CTV were then calculated.

RESULTS

A significant variability was found in total volume (mean, 498.3 cm(3); range, 181.8-1003 cm(3)) and craniocaudal (median, 144.7 mm; range, 80.6-159 mm), laterolateral (median, 133.5 mm; range, 83.7-149.5 mm), and anteroposterior diameters (median, 136.2 mm; range, 84-150.5 mm). Analysis of the DSC and the HD showed a mean DSC of 0.53 (range, 0.31-0.74) and a mean HD of 6.4 cm (range, 1.8-14.8 cm).

CONCLUSIONS

Results of this study strongly indicate the need to develop and share appropriate contouring guidelines among experts and suggest the promotion of specific educational activities to improve radiation therapy quality in both clinical trials and routine clinical practice.

Delineation of the larynx as organ at risk in radiotherapy: a contouring course within "Rete Oncologica Piemonte-Valle d'Aosta" network to reduce inter- and intraobserver variability

AIMS

To evaluate the usefulness of a contouring course in reducing inter- and intraobserver variability in the definition of the larynx as organ at risk (OAR).

METHODS

Within the "Rete Oncologica Piemonte-Valle d'Aosta" network, a contouring course focusing on larynx delineation was proposed. Twenty-six radiotherapist technicians (RTTs) experienced in delineating OARs were asked to contour larynx before and after the training. An expert radiation oncologist defined the reference volume for educational purpose. The contoured volumes obtained before and after the course were compared using descriptive statistics (mean value, standard deviation-SD, and coefficient of variation-COV) of volumes and maximum diameters. Conformity index (CI), dice coefficient (DC), and percentage of overlap were used to evaluate the spatial accuracy of the different volumes compared to the reference. Further analysis regarding the variation in the centre of mass (COM) displacement was performed.

RESULTS

The mean volume was 40.4 cm³ before and 65.9 cm³ after the course, approaching the reference value. Mean anteroposterior, laterolateral, and craniocaudal diameters improved, getting each closer to the reference. Moreover, the COM moved approaching reference coordinates. Mean percentage of intersection and DC strongly increased after the course, rising from 57.76 to 93.83 % and from 0.68 to 0.89, respectively. CI enhanced from 0.06 to 0.31.

CONCLUSIONS

This study shows an improvement in larynx definition after the contouring course with lower interobserver variability and major consistency compared to the reference volume. Other specific educational activities may further increase the quality of radiation therapy contouring in this setting.

Uncertainties in target volume delineation in radiotherapy - are they relevant and what can we do about them?

Background

Modern radiotherapy techniques enable delivery of high doses to the target volume without escalating dose to organs at risk, offering the possibility of better local control while preserving good quality of life. Uncertainties in target volume delineation have been demonstrated for most tumour sites, and various studies indicate that inconsistencies in target volume delineation may be larger than errors in all other steps of the treatment planning and delivery process. The aim of this paper is to summarize the degree of delineation uncertainties for different tumour sites reported in the literature and review the effect of strategies to minimize them.

Conclusions

Our review confirmed that interobserver variability in target volume contouring represents the largest uncertainty in the process for most tumour sites, potentially resulting in a systematic error in dose delivery, which could influence local control in individual patients. For most tumour sites the optimal combination of imaging modalities for target delineation still needs to be determined. Strict use of delineation guidelines and protocols is advisable both in every day clinical practice and in clinical studies to diminish interobserver variability. Continuing medical education of radiation oncologists cannot be overemphasized, intensive formal training on interpretation of sectional imaging should be included in the program for radiation oncology residents.

Radiotherapy of Hodgkin and Non-Hodgkin Lymphoma

Purpose:

To improve the contouring of clinical target volume for the radiotherapy of neck Hodgkin/non-Hodgkin lymphoma by localizing the prechemotherapy gross target volume onto the simulation computed tomography using [18F]-fluorodeoxyglucose positron emission tomography/computed tomography.

Material and Methods:

The gross target volume delineated on prechemotherapy [18F]-fluorodeoxyglucose positron emission tomography/computed tomography images was warped onto simulation computed tomography using deformable image registration. Fifteen patients with neck Hodgkin/non-Hodgkin lymphoma were analyzed. Quality of image registration was measured by computing the Dice similarity coefficient on warped organs at risk. Five radiation oncologists visually scored the localization of automatic gross target volume, ranking it from 1 (wrong) to 5 (excellent). Deformable registration was compared to rigid registration by computing the overlap index between the automatic gross target volume and the planned clinical target volume and quantifying the V95 coverage.

Results:

The Dice similarity coefficient was 0.80 ± 0.07 (median ± quartiles). The physicians’ survey had a median score equal to 4 (good). By comparing the rigid versus deformable registration, the overlap index increased from a factor of about 4 and the V95 (percentage of volume receiving the 95% of the prescribed dose) went from 0.84 ± 0.38 to 0.99 ± 0.10 (median ± quartiles).

Conclusion:

This study demonstrates the impact of using deformable registration between prechemotherapy [18F]-fluorodeoxyglucose positron emission tomography/computed tomography and simulation computed tomography, in order to automatically localize the gross target volume for radiotherapy treatment of patients with Hodgkin/non-Hodgkin lymphoma.

Inter-observer variability of clinical target volume delineation in radiotherapy treatment of pancreatic cancer: a multi-institutional contouring experience

BACKGROUND

An observational multi-institutional study has been conducted aimed to evaluate the inter-observer variability in clinical target volume (CTV) delineation among different radiation oncologists in radiotherapy treatment of pancreatic cancer.

METHODS

A multi-institutional contouring dummy-run of two different cases of pancreatic cancer treated by postoperative and preoperative radiotherapy (RT) was performed. Clinical history, diagnostics, and planning CT imaging were available on AIRO website (http://www.radioterapiaitalia.it). Participants were requested to delineate CTVs according to their skills and knowledge. Aiming to quantify interobserver variability of CTVs delineations, the total volume, craniocaudal, laterolateral, and anteroposterior diameters were calculated. Descriptive statistic was calculated. The 95% Confidence Interval (95% CI) for coefficient of variation (CV) was estimated. The Dice Similarity Index (DSI) was used to evaluate the spatial overlap accuracy of the different CTVs compared with the CTVs of a national reference Centre considered as a benchmark. The mean DSI (mDSI) was calculated and reported.

RESULTS

A total of 18 radiation oncologists from different Institutes submitted the targets. Less variability was observed for the Elective CTV rather than the Boost CTV, in both cases. The estimated CV were 28.8% (95% CI: 21.2-45.0%) and 20.0% (95% CI: 14.9-30.6%) for the Elective CTV, in adjuvant (Case 1) and neoadjuvant (Case 2) case, respectively. The mDSI value was 0.68 for the Elective CTVs in both cases (range 0.19-0.79 in postoperative vs range 0.35-0.79 in preoperative case). The mDSI was increased to 0.71 (Case 1) and 0.72 (Case 2) if the observers with a worse agreement have been excluded. On the other hand, a CV of 42.4% (95% CI: 30.1-72.4%) and 63.8% (95% CI: 43.9-119.2%) with a mDSI value of 0.44 and 0.52, were calculated for the Boost CTV in Case 1 and Case 2, respectively.

CONCLUSIONS

The CV and mDSI obtained values for Elective CTVs showed an acceptable agreement among participants either in postoperative as well in preoperative setting. Additional strategies to reduce the variability in Boost CTV delineation need to be found and promoted.

Interobserver variability of clinical target volume delineation in soft-tissue sarcomas

PURPOSE

The present observational study reports the results of a multi-institutional dummy-run designed to estimate the consistency of interobserver variability in clinical target volume delineation in two different cases of soft-tissue sarcomas in which postoperative and preoperative radiotherapy were prescribed, respectively. The purpose of this work was to quantify interobserver variability in routine clinical practice.

PATIENTS AND METHODS

Two different cases of soft-tissues sarcomas were chosen: a case of postoperative and a case of preoperative radiation therapy. Participating centres were requested to delineate clinical target volumes according to their experience in both cases. Descriptive statistic was calculated for each variable (volume, diameters) separately for two cases. Box-whiskers plots were used for presentation of clinical target volume. A Shapiro-Wilk's test was performed to evaluate the departures from normality distribution for each variable. The comparison between relative variations of diameters was evaluated using the Student's t test.

RESULTS

Several variations affecting both volumes and diameters were observed. Main variations were observed in the craniocaudal and laterolateral diameters. Each case showed similar dispersion, indicating a lack of reproducibility in volumes definition.

CONCLUSIONS

This observational study highlighted that, in the absence of specific instructions or guidelines, the interobserver variability can be significant both in postoperative and preoperative radiotherapy of soft-tissue sarcomas.

Multicenter evaluation of different target volume delineation concepts in pediatric Hodgkin's lymphoma. A case study

BACKGROUND AND PURPOSE

In pediatric Hodgkin's lymphoma (PHL) improvements in imaging and multiagent chemotherapy have allowed for a reduction in target volume. The involved-node (IN) concept is being tested in several treatment regimens for adult Hodgkin's lymphoma. So far there is no consensus on the definition of the IN. To improve the reproducibility of the IN, we tested a new involved-node-level (INL) concept, using defined anatomical boundaries as basis for target delineation. The aim was to evaluate the feasibility of IN and INL concepts for PHL in terms of interobserver variability.

PATIENTS AND METHODS

The INL concept was defined for the neck and mediastinum by the PHL Radiotherapy Group based on accepted concepts for solid tumors. Seven radiation oncologists from six European centers contoured neck and mediastinal clinical target volumes (CTVs) of 2 patients according to the IN and the new INL concepts. The median CTVs, coefficient of variation (COV), and general conformity index (CI) were assessed. The intraclass correlation coefficient (ICC) for reliability of delineations was calculated.

RESULTS

All observers agreed that INL is a feasible and practicable delineation concept resulting in stronger interobserver concordance than the IN (mediastinum CI(INL) = 0.39 vs. CI(IN) = 0.28, neck left CI(INL) = 0.33; CI(IN) = 0.18; neck right CI(INL) = 0.24, CI(IN) = 0.14). The COV showed less dispersion and the ICC indicated higher reliability of contouring for INL (ICC(INL) = 0.62, p < 0.05) as for IN (ICC(IN) = 0.40, p < 0.05).

CONCLUSION

INL is a practical and feasible alternative to IN resulting in more homogeneous target delineation, and it should be therefore considered as a future target volume concept in PHL.

Assessment of accuracy and efficiency of atlas-based autosegmentation for prostate radiotherapy in a variety of clinical conditions

BACKGROUND AND PURPOSE

The goal of the current study was to evaluate the commercially available atlas-based autosegmentation software for clinical use in prostate radiotherapy. The accuracy was benchmarked against interobserver variability.

MATERIAL AND METHODS

A total of 20 planning computed tomographs (CTs) and 10 cone-beam CTs (CBCTs) were selected for prostate, rectum, and bladder delineation. The images varied regarding to individual (age, body mass index) and setup parameters (contrast agent, rectal balloon, implanted markers). Automatically created contours with ABAS(®) and iPlan(®) were compared to an expert's delineation by calculating the Dice similarity coefficient (DSC) and conformity index.

RESULTS

Demo-atlases of both systems showed different results for bladder (DSC(ABAS) 0.86 ± 0.17, DSC(iPlan) 0.51 ± 0.30) and prostate (DSC(ABAS) 0.71 ± 0.14, DSC(iPlan) 0.57 ± 0.19). Rectum delineation (DSC(ABAS) 0.78 ± 0.11, DSC(iPlan) 0.84 ± 0.08) demonstrated differences between the systems but better correlation of the automatically drawn volumes. ABAS(®) was closest to the interobserver benchmark. Autosegmentation with iPlan(®), ABAS(®) and manual segmentation took 0.5, 4 and 15-20 min, respectively. Automatic contouring on CBCT showed high dependence on image quality (DSC bladder 0.54, rectum 0.42, prostate 0.34).

CONCLUSION

For clinical routine, efforts are still necessary to either redesign algorithms implemented in autosegmentation or to optimize image quality for CBCT to guarantee required accuracy and time savings for adaptive radiotherapy.

Critical discussion of evaluation parameters for inter-observer variability in target definition for radiation therapy

BACKGROUND AND PURPOSE

Inter-observer studies represent a valid method for the evaluation of target definition uncertainties and contouring guidelines. However, data from the literature do not yet give clear guidelines for reporting contouring variability. Thus, the purpose of this work was to compare and discuss various methods to determine variability on the basis of clinical cases and a literature review.

PATIENTS AND METHODS

In this study, 7 prostate and 8 lung cases were contoured on CT images by 8 experienced observers. Analysis of variability included descriptive statistics, calculation of overlap measures, and statistical measures of agreement. Cross tables with ratios and correlations were established for overlap parameters.

RESULTS

It was shown that the minimal set of parameters to be reported should include at least one of three volume overlap measures (i.e., generalized conformity index, Jaccard coefficient, or conformation number). High correlation between these parameters and scatter of the results was observed.

CONCLUSION

A combination of descriptive statistics, overlap measure, and statistical measure of agreement or reliability analysis is required to fully report the interrater variability in delineation.