1
|
Rhee DJ, Akinfenwa CPA, Rigaud B, Jhingran A, Cardenas CE, Zhang L, Prajapati S, Kry SF, Brock KK, Beadle BM, Shaw W, O'Reilly F, Parkes J, Burger H, Fakie N, Trauernicht C, Simonds H, Court LE. Automatic contouring QA method using a deep learning-based autocontouring system. J Appl Clin Med Phys 2022; 23:e13647. [PMID: 35580067 PMCID: PMC9359039 DOI: 10.1002/acm2.13647] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2022] [Revised: 02/27/2022] [Accepted: 04/28/2022] [Indexed: 02/04/2023] Open
Abstract
Purpose To determine the most accurate similarity metric when using an independent system to verify automatically generated contours. Methods A reference autocontouring system (primary system to create clinical contours) and a verification autocontouring system (secondary system to test the primary contours) were used to generate a pair of 6 female pelvic structures (UteroCervix [uterus + cervix], CTVn [nodal clinical target volume (CTV)], PAN [para‐aortic lymph nodes], bladder, rectum, and kidneys) on 49 CT scans from our institution and 38 from other institutions. Additionally, clinically acceptable and unacceptable contours were manually generated using the 49 internal CT scans. Eleven similarity metrics (volumetric Dice similarity coefficient (DSC), Hausdorff distance, 95% Hausdorff distance, mean surface distance, and surface DSC with tolerances from 1 to 10 mm) were calculated between the reference and the verification autocontours, and between the manually generated and the verification autocontours. A support vector machine (SVM) was used to determine the threshold that separates clinically acceptable and unacceptable contours for each structure. The 11 metrics were investigated individually and in certain combinations. Linear, radial basis function, sigmoid, and polynomial kernels were tested using the combinations of metrics as inputs for the SVM. Results The highest contouring error detection accuracies were 0.91 for the UteroCervix, 0.90 for the CTVn, 0.89 for the PAN, 0.92 for the bladder, 0.95 for the rectum, and 0.97 for the kidneys and were achieved using surface DSCs with a thickness of 1, 2, or 3 mm. The linear kernel was the most accurate and consistent when a combination of metrics was used as an input for the SVM. However, the best model accuracy from the combinations of metrics was not better than the best model accuracy from a surface DSC as an input. Conclusions We distinguished clinically acceptable contours from clinically unacceptable contours with an accuracy higher than 0.9 for the targets and critical structures in patients with cervical cancer; the most accurate similarity metric was surface DSC with a thickness of 1, 2, or 3 mm.
Collapse
Affiliation(s)
- Dong Joo Rhee
- The University of Texas Graduate School of Biomedical Sciences at Houston, Houston, Texas, USA.,Department of Radiation Physics, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | | | - Bastien Rigaud
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Anuja Jhingran
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Carlos E Cardenas
- Department of Radiation Physics, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Lifei Zhang
- Department of Radiation Physics, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Surendra Prajapati
- Department of Radiation Physics, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Stephen F Kry
- Department of Radiation Physics, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Kristy K Brock
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Beth M Beadle
- Department of Radiation Oncology, Stanford University School of Medicine, Stanford, California, USA
| | - William Shaw
- Department of Medical Physics (G68), University of the Free State, Bloemfontein, South Africa
| | - Frederika O'Reilly
- Department of Medical Physics (G68), University of the Free State, Bloemfontein, South Africa
| | - Jeannette Parkes
- Division of Radiation Oncology and Medical Physics, University of Cape Town and Groote Schuur Hospital, Cape Town, South Africa
| | - Hester Burger
- Division of Radiation Oncology and Medical Physics, University of Cape Town and Groote Schuur Hospital, Cape Town, South Africa
| | - Nazia Fakie
- Division of Radiation Oncology and Medical Physics, University of Cape Town and Groote Schuur Hospital, Cape Town, South Africa
| | - Chris Trauernicht
- Division of Medical Physics, Stellenbosch University, Tygerberg Academic Hospital, Cape Town, South Africa
| | - Hannah Simonds
- Division of Radiation Oncology, Stellenbosch University, Tygerberg Academic Hospital, Cape Town, South Africa
| | - Laurence E Court
- Department of Radiation Physics, Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| |
Collapse
|
2
|
Gonzalez Y, Shen C, Jung H, Nguyen D, Jiang SB, Albuquerque K, Jia X. Semi-automatic sigmoid colon segmentation in CT for radiation therapy treatment planning via an iterative 2.5-D deep learning approach. Med Image Anal 2021; 68:101896. [PMID: 33383333 PMCID: PMC7847132 DOI: 10.1016/j.media.2020.101896] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2020] [Revised: 11/03/2020] [Accepted: 11/04/2020] [Indexed: 10/22/2022]
Abstract
Automatic sigmoid colon segmentation in CT for radiotherapy treatment planning is challenging due to complex organ shape, close distances to other organs, and large variations in size, shape, and filling status. The patient bowel is often not evacuated, and CT contrast enhancement is not used, which further increase problem difficulty. Deep learning (DL) has demonstrated its power in many segmentation problems. However, standard 2-D approaches cannot handle the sigmoid segmentation problem due to incomplete geometry information and 3-D approaches often encounters the challenge of a limited training data size. Motivated by human's behavior that segments the sigmoid slice by slice while considering connectivity between adjacent slices, we proposed an iterative 2.5-D DL approach to solve this problem. We constructed a network that took an axial CT slice, the sigmoid mask in this slice, and an adjacent CT slice to segment as input and output the predicted mask on the adjacent slice. We also considered other organ masks as prior information. We trained the iterative network with 50 patient cases using five-fold cross validation. The trained network was repeatedly applied to generate masks slice by slice. The method achieved average Dice similarity coefficients of 0.82 0.06 and 0.88 0.02 in 10 test cases without and with using prior information.
Collapse
Affiliation(s)
- Yesenia Gonzalez
- innovative Technology of Radiotherapy Computation and Hardware (iTORCH) Laboratory. Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA; Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA
| | - Chenyang Shen
- innovative Technology of Radiotherapy Computation and Hardware (iTORCH) Laboratory. Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA; Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA.
| | - Hyunuk Jung
- innovative Technology of Radiotherapy Computation and Hardware (iTORCH) Laboratory. Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA; Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA
| | - Dan Nguyen
- Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA
| | - Steve B Jiang
- Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA
| | - Kevin Albuquerque
- Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA
| | - Xun Jia
- innovative Technology of Radiotherapy Computation and Hardware (iTORCH) Laboratory. Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA; Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, TX 75390, USA.
| |
Collapse
|
3
|
Rhee DJ, Cardenas CE, Elhalawani H, McCarroll R, Zhang L, Yang J, Garden AS, Peterson CB, Beadle BM, Court LE. Automatic detection of contouring errors using convolutional neural networks. Med Phys 2019; 46:5086-5097. [PMID: 31505046 PMCID: PMC6842055 DOI: 10.1002/mp.13814] [Citation(s) in RCA: 58] [Impact Index Per Article: 11.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2019] [Revised: 08/28/2019] [Accepted: 08/30/2019] [Indexed: 12/14/2022] Open
Abstract
PURPOSE To develop a head and neck normal structures autocontouring tool that could be used to automatically detect the errors in autocontours from a clinically validated autocontouring tool. METHODS An autocontouring tool based on convolutional neural networks (CNN) was developed for 16 normal structures of the head and neck and tested to identify the contour errors from a clinically validated multiatlas-based autocontouring system (MACS). The computed tomography (CT) scans and clinical contours from 3495 patients were semiautomatically curated and used to train and validate the CNN-based autocontouring tool. The final accuracy of the tool was evaluated by calculating the Sørensen-Dice similarity coefficients (DSC) and Hausdorff distances between the automatically generated contours and physician-drawn contours on 174 internal and 24 external CT scans. Lastly, the CNN-based tool was evaluated on 60 patients' CT scans to investigate the possibility to detect contouring failures. The contouring failures on these patients were classified as either minor or major errors. The criteria to detect contouring errors were determined by analyzing the DSC between the CNN- and MACS-based contours under two independent scenarios: (a) contours with minor errors are clinically acceptable and (b) contours with minor errors are clinically unacceptable. RESULTS The average DSC and Hausdorff distance of our CNN-based tool was 98.4%/1.23 cm for brain, 89.1%/0.42 cm for eyes, 86.8%/1.28 cm for mandible, 86.4%/0.88 cm for brainstem, 83.4%/0.71 cm for spinal cord, 82.7%/1.37 cm for parotids, 80.7%/1.08 cm for esophagus, 71.7%/0.39 cm for lenses, 68.6%/0.72 for optic nerves, 66.4%/0.46 cm for cochleas, and 40.7%/0.96 cm for optic chiasm. With the error detection tool, the proportions of the clinically unacceptable MACS contours that were correctly detected were 0.99/0.80 on average except for the optic chiasm, when contours with minor errors are clinically acceptable/unacceptable, respectively. The proportions of the clinically acceptable MACS contours that were correctly detected were 0.81/0.60 on average except for the optic chiasm, when contours with minor errors are clinically acceptable/unacceptable, respectively. CONCLUSION Our CNN-based autocontouring tool performed well on both the publically available and the internal datasets. Furthermore, our results show that CNN-based algorithms are able to identify ill-defined contours from a clinically validated and used multiatlas-based autocontouring tool. Therefore, our CNN-based tool can effectively perform automatic verification of MACS contours.
Collapse
Affiliation(s)
- Dong Joo Rhee
- The University of Texas Graduate School of Biomedical Sciences at HoustonHoustonTX77030USA
- Department of Radiation PhysicsDivision of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
| | - Carlos E. Cardenas
- Department of Radiation PhysicsDivision of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
| | - Hesham Elhalawani
- Department of Radiation OncologyDivision of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
| | - Rachel McCarroll
- Department of Radiation OncologyThe University of Maryland Medical SystemBaltimoreMD21201USA
| | - Lifei Zhang
- Department of Radiation PhysicsDivision of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
| | - Jinzhong Yang
- Department of Radiation PhysicsDivision of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
| | - Adam S. Garden
- Department of Radiation OncologyDivision of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
| | - Christine B. Peterson
- Department of BiostatisticsDivision of Basic SciencesThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
| | - Beth M. Beadle
- Department of Radiation OncologyStanford University School of MedicineStanfordCA94305USA
| | - Laurence E. Court
- Department of Radiation PhysicsDivision of Radiation OncologyThe University of Texas MD Anderson Cancer CenterHoustonTX77030USA
| |
Collapse
|