1
|
Rusanov B, Ebert MA, Sabet M, Rowshanfarzad P, Barry N, Kendrick J, Alkhatib Z, Gill S, Dass J, Bucknell N, Croker J, Tang C, White R, Bydder S, Taylor M, Slama L, Mukwada G. Guidance on selecting and evaluating AI auto-segmentation systems in clinical radiotherapy: insights from a six-vendor analysis. Phys Eng Sci Med 2025; 48:301-316. [PMID: 39804550 PMCID: PMC11997002 DOI: 10.1007/s13246-024-01513-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2024] [Accepted: 12/20/2024] [Indexed: 04/15/2025]
Abstract
Artificial Intelligence (AI) based auto-segmentation has demonstrated numerous benefits to clinical radiotherapy workflows. However, the rapidly changing regulatory, research, and market environment presents challenges around selecting and evaluating the most suitable solution. To support the clinical adoption of AI auto-segmentation systems, Selection Criteria recommendations were developed to enable a holistic evaluation of vendors, considering not only raw performance but associated risks uniquely related to the clinical deployment of AI. In-house experience and key bodies of work on ethics, standards, and best practices for AI in Radiation Oncology were reviewed to inform selection criteria and evaluation strategies. A retrospective analysis using the criteria was performed across six vendors, including a quantitative assessment using five metrics (Dice, Hausdorff Distance, Average Surface Distance, Surface Dice, Added Path Length) across 20 head and neck, 20 thoracic, and 19 male pelvis patients for AI models as of March 2023. A total of 47 selection criteria were identified across seven categories. A retrospective analysis showed that overall no vendor performed exceedingly well, with systematically poor performance in Data Security & Responsibility, Vendor Support Tools, and Transparency & Ethics. In terms of raw performance, vendors varied widely from excellent to poor. As new regulations come into force and the scope of AI auto-segmentation systems adapt to clinical needs, continued interest in ensuring safe, fair, and transparent AI will persist. The selection and evaluation framework provided herein aims to promote user confidence by exploring the breadth of clinically relevant factors to support informed decision-making.
Collapse
Affiliation(s)
- Branimir Rusanov
- School of Physics, Mathematics and Computing, The University of Western Australia, Crawley, WA, Australia.
- Department of Radiation Oncology, Sir Charles Gairdner Hospital, Nedlands, WA, Australia.
- Center for Advanced Technologies in Cancer Research, Perth, WA, Australia.
| | - Martin A Ebert
- School of Physics, Mathematics and Computing, The University of Western Australia, Crawley, WA, Australia
- Department of Radiation Oncology, Sir Charles Gairdner Hospital, Nedlands, WA, Australia
- Center for Advanced Technologies in Cancer Research, Perth, WA, Australia
- Australian Centre for Quantitative Imaging, The University of Western Australia, Crawley, WA, Australia
- School of Medicine and Public Health, University of Wisconsin, Madison, WI, USA
| | - Mahsheed Sabet
- School of Physics, Mathematics and Computing, The University of Western Australia, Crawley, WA, Australia
- Department of Radiation Oncology, Sir Charles Gairdner Hospital, Nedlands, WA, Australia
- Center for Advanced Technologies in Cancer Research, Perth, WA, Australia
| | - Pejman Rowshanfarzad
- School of Physics, Mathematics and Computing, The University of Western Australia, Crawley, WA, Australia
- Center for Advanced Technologies in Cancer Research, Perth, WA, Australia
| | - Nathaniel Barry
- School of Physics, Mathematics and Computing, The University of Western Australia, Crawley, WA, Australia
- Center for Advanced Technologies in Cancer Research, Perth, WA, Australia
| | - Jake Kendrick
- School of Physics, Mathematics and Computing, The University of Western Australia, Crawley, WA, Australia
- Center for Advanced Technologies in Cancer Research, Perth, WA, Australia
| | - Zaid Alkhatib
- Department of Radiation Oncology, Sir Charles Gairdner Hospital, Nedlands, WA, Australia
| | - Suki Gill
- Department of Radiation Oncology, Sir Charles Gairdner Hospital, Nedlands, WA, Australia
| | - Joshua Dass
- Department of Radiation Oncology, Sir Charles Gairdner Hospital, Nedlands, WA, Australia
| | - Nicholas Bucknell
- Department of Radiation Oncology, Sir Charles Gairdner Hospital, Nedlands, WA, Australia
| | - Jeremy Croker
- Department of Radiation Oncology, Sir Charles Gairdner Hospital, Nedlands, WA, Australia
| | - Colin Tang
- Department of Radiation Oncology, Sir Charles Gairdner Hospital, Nedlands, WA, Australia
| | - Rohen White
- Department of Radiation Oncology, Sir Charles Gairdner Hospital, Nedlands, WA, Australia
| | - Sean Bydder
- Department of Radiation Oncology, Sir Charles Gairdner Hospital, Nedlands, WA, Australia
| | - Mandy Taylor
- Department of Radiation Oncology, Sir Charles Gairdner Hospital, Nedlands, WA, Australia
| | - Luke Slama
- Department of Radiation Oncology, Sir Charles Gairdner Hospital, Nedlands, WA, Australia
| | - Godfrey Mukwada
- School of Physics, Mathematics and Computing, The University of Western Australia, Crawley, WA, Australia
- Department of Radiation Oncology, Sir Charles Gairdner Hospital, Nedlands, WA, Australia
| |
Collapse
|
2
|
Ferreira Silvério N, van den Wollenberg W, Betgen A, Wiersema L, Marijnen CAM, Peters F, van der Heide UA, Simões R, Intven MPW, van der Bijl E, Janssen T. Incorporating patient-specific prior clinical knowledge to improve clinical target volume auto-segmentation generalisability for online adaptive radiotherapy of rectal cancer: A multicenter validation. Radiother Oncol 2025; 203:110667. [PMID: 39675574 DOI: 10.1016/j.radonc.2024.110667] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2024] [Revised: 11/29/2024] [Accepted: 12/09/2024] [Indexed: 12/17/2024]
Abstract
BACKGROUND & PURPOSE Deep learning (DL) based auto-segmentation has shown to be beneficial for online adaptive radiotherapy (OART). However, auto-segmentation of clinical target volumes (CTV) is complex, as clinical interpretations are crucial in their definition. The resulting variation between clinicians and institutes hampers the generalizability of DL networks. In OART the CTV is delineated during treatment preparation which makes the clinician intent explicitly available during treatment. We studied whether multicenter generalisability improves when using this prior clinical knowledge, the pre-treatment delineation, as a patient-specific prior for DL models for online auto-segmentation of the mesorectal CTV. MATERIAL & METHODS We included intermediate risk or locally advanced rectal cancer patients from three centers. Patient-specific weight maps were created by combining the patient-specific CTV delineation on the pre-treatment scan with population-based variation of likely inter-fraction mesorectal CTV deformations. We trained two models to auto-segment the mesorectal CTV on in-house data, one with (MRI + prior) and one without (MRI-only) priors. Both models were applied to two external datasets. An external baseline model was trained without priors from scratch for one external center. Performance was evaluated on the DSC, surface Dice, 95HD and MSD. RESULTS For both external centers, the MRI + prior model outperformed the MRI-only model significantly on the segmentation metrics (p-values < 0.01). There was no significant difference between the external baseline model and the MRI + prior model. CONCLUSION Adding patient-specific weight maps makes the CTV segmentation model more robust to institutional preferences. Performance was comparable to a model trained locally from scratch. This makes this approach suitable for generalization to multiple centers.
Collapse
Affiliation(s)
- Nicole Ferreira Silvério
- Department of Radiation Oncology, The Netherlands Cancer Institute, Plesmanlaan 121 1066CX Amsterdam, the Netherlands
| | - Wouter van den Wollenberg
- Department of Radiation Oncology, The Netherlands Cancer Institute, Plesmanlaan 121 1066CX Amsterdam, the Netherlands
| | - Anja Betgen
- Department of Radiation Oncology, The Netherlands Cancer Institute, Plesmanlaan 121 1066CX Amsterdam, the Netherlands
| | - Lisa Wiersema
- Department of Radiation Oncology, The Netherlands Cancer Institute, Plesmanlaan 121 1066CX Amsterdam, the Netherlands
| | - Corrie A M Marijnen
- Department of Radiation Oncology, The Netherlands Cancer Institute, Plesmanlaan 121 1066CX Amsterdam, the Netherlands
| | - Femke Peters
- Department of Radiation Oncology, The Netherlands Cancer Institute, Plesmanlaan 121 1066CX Amsterdam, the Netherlands
| | - Uulke A van der Heide
- Department of Radiation Oncology, The Netherlands Cancer Institute, Plesmanlaan 121 1066CX Amsterdam, the Netherlands
| | - Rita Simões
- Department of Radiation Oncology, The Netherlands Cancer Institute, Plesmanlaan 121 1066CX Amsterdam, the Netherlands
| | - Martijn P W Intven
- Department of Radiation Oncology, University Medical Center Utrecht, Heidelberglaan 100 3584CX Utrecht, the Netherlands
| | - Erik van der Bijl
- Department of Radiation Oncology, Radboud University Medical Center, Geert Grooteplein Zuid 10 6525 GA Nijmegen, the Netherlands
| | - Tomas Janssen
- Department of Radiation Oncology, The Netherlands Cancer Institute, Plesmanlaan 121 1066CX Amsterdam, the Netherlands.
| |
Collapse
|
3
|
Smine Z, Poeta S, De Caluwé A, Desmet A, Garibaldi C, Brou Boni K, Levillain H, Van Gestel D, Reynaert N, Dhont J. Automated segmentation in planning-CT for breast cancer radiotherapy: A review of recent advances. Radiother Oncol 2025; 202:110615. [PMID: 39489430 DOI: 10.1016/j.radonc.2024.110615] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2024] [Revised: 10/21/2024] [Accepted: 10/28/2024] [Indexed: 11/05/2024]
Abstract
Postoperative radiotherapy (RT) has been shown to effectively reduce disease recurrence and mortality in breast cancer (BC) treatment. A critical step in the planning workflow is the accurate delineation of clinical target volumes (CTV) and organs-at-risk (OAR). This literature review evaluates recent advancements in deep-learning (DL) and atlas-based auto-contouring techniques for CTVs and OARs in BC planning-CT images for RT. It examines their performance regarding geometrical and dosimetric accuracy, inter-observer variability, and time efficiency. Our findings indicate that both DL- and atlas-based methods generally show comparable performance across OARs and CTVs, with DL methods slightly outperforming in consistency and accuracy. Auto-segmentation of breast and most OARs achieved robust results in both segmentation quality and dosimetric planning. However, lymph node levels (LNLs) presented the greatest challenge in auto-segmentation with significant impact on dosimetric planning. The translation of these findings into clinical practice is limited by the geometric performance metrics and the lack of dose evaluation studies. Additionally, auto-contouring algorithms showed diverse structure sets, while training datasets varied in size, origin, patient positioning and imaging protocols, affecting model sensitivity. Guideline inconsistencies and varying definitions of ground truth led to substantial variability, suggesting a need for a reliable consensus training dataset. Finally, our review highlights the popularity of the U-Net architecture. In conclusion, while automated contouring has proven efficient for many OARs and the breast-CTV, further improvements are necessary in LNL delineation, dosimetric analysis, and consensus building.
Collapse
Affiliation(s)
- Zineb Smine
- Radiophysics and MRI Physics Laboratory, Université Libre De Bruxelles (ULB), Brussels, Belgium; Department of Medical Physics, Institut Jules Bordet, Hôpital Universitaire de Bruxelles (H.U.B), Université Libre de Bruxelles (ULB), Brussels, Belgium.
| | - Sara Poeta
- Department of Medical Physics, Institut Jules Bordet, Hôpital Universitaire de Bruxelles (H.U.B), Université Libre de Bruxelles (ULB), Brussels, Belgium
| | - Alex De Caluwé
- Department of Radiotherapy, Institut Jules Bordet, Hôpital Universitaire de Bruxelles (H.U.B), Université Libre de Bruxelles (ULB), Brussels, Belgium
| | - Antoine Desmet
- Department of Radiotherapy, Institut Jules Bordet, Hôpital Universitaire de Bruxelles (H.U.B), Université Libre de Bruxelles (ULB), Brussels, Belgium
| | - Cristina Garibaldi
- Unit of Radiation Research, IEO, European Institute of Oncology IRCCS, Milan, Italy
| | - Kevin Brou Boni
- Radiophysics and MRI Physics Laboratory, Université Libre De Bruxelles (ULB), Brussels, Belgium; Department of Medical Physics, Institut Jules Bordet, Hôpital Universitaire de Bruxelles (H.U.B), Université Libre de Bruxelles (ULB), Brussels, Belgium
| | - Hugo Levillain
- Radiophysics and MRI Physics Laboratory, Université Libre De Bruxelles (ULB), Brussels, Belgium; Department of Medical Physics, Institut Jules Bordet, Hôpital Universitaire de Bruxelles (H.U.B), Université Libre de Bruxelles (ULB), Brussels, Belgium
| | - Dirk Van Gestel
- Department of Radiotherapy, Institut Jules Bordet, Hôpital Universitaire de Bruxelles (H.U.B), Université Libre de Bruxelles (ULB), Brussels, Belgium
| | - Nick Reynaert
- Radiophysics and MRI Physics Laboratory, Université Libre De Bruxelles (ULB), Brussels, Belgium; Department of Medical Physics, Institut Jules Bordet, Hôpital Universitaire de Bruxelles (H.U.B), Université Libre de Bruxelles (ULB), Brussels, Belgium
| | - Jennifer Dhont
- Radiophysics and MRI Physics Laboratory, Université Libre De Bruxelles (ULB), Brussels, Belgium; Department of Medical Physics, Institut Jules Bordet, Hôpital Universitaire de Bruxelles (H.U.B), Université Libre de Bruxelles (ULB), Brussels, Belgium.
| |
Collapse
|
4
|
Finnegan RN, Quinn A, Booth J, Belous G, Hardcastle N, Stewart M, Griffiths B, Carroll S, Thwaites DI. Cardiac substructure delineation in radiation therapy - A state-of-the-art review. J Med Imaging Radiat Oncol 2024; 68:914-949. [PMID: 38757728 PMCID: PMC11686467 DOI: 10.1111/1754-9485.13668] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2024] [Accepted: 04/29/2024] [Indexed: 05/18/2024]
Abstract
Delineation of cardiac substructures is crucial for a better understanding of radiation-related cardiotoxicities and to facilitate accurate and precise cardiac dose calculation for developing and applying risk models. This review examines recent advancements in cardiac substructure delineation in the radiation therapy (RT) context, aiming to provide a comprehensive overview of the current level of knowledge, challenges and future directions in this evolving field. Imaging used for RT planning presents challenges in reliably visualising cardiac anatomy. Although cardiac atlases and contouring guidelines aid in standardisation and reduction of variability, significant uncertainties remain in defining cardiac anatomy. Coupled with the inherent complexity of the heart, this necessitates auto-contouring for consistent large-scale data analysis and improved efficiency in prospective applications. Auto-contouring models, developed primarily for breast and lung cancer RT, have demonstrated performance comparable to manual contouring, marking a significant milestone in the evolution of cardiac delineation practices. Nevertheless, several key concerns require further investigation. There is an unmet need for expanding cardiac auto-contouring models to encompass a broader range of cancer sites. A shift in focus is needed from ensuring accuracy to enhancing the robustness and accessibility of auto-contouring models. Addressing these challenges is paramount for the integration of cardiac substructure delineation and associated risk models into routine clinical practice, thereby improving the safety of RT for future cancer patients.
Collapse
Affiliation(s)
- Robert N Finnegan
- Northern Sydney Cancer CentreRoyal North Shore HospitalSydneyNew South WalesAustralia
- Institute of Medical Physics, School of Physics, University of SydneySydneyNew South WalesAustralia
| | - Alexandra Quinn
- Northern Sydney Cancer CentreRoyal North Shore HospitalSydneyNew South WalesAustralia
| | - Jeremy Booth
- Northern Sydney Cancer CentreRoyal North Shore HospitalSydneyNew South WalesAustralia
- Institute of Medical Physics, School of Physics, University of SydneySydneyNew South WalesAustralia
| | - Gregg Belous
- Australian e‐Health Research CentreCommonwealth Scientific and Industrial Research OrganisationBrisbaneQueenslandAustralia
| | - Nicholas Hardcastle
- Department of Physical SciencesPeter MacCallum Cancer CentreMelbourneVictoriaAustralia
- Sir Peter MacCallum Department of OncologyUniversity of MelbourneMelbourneVictoriaAustralia
| | - Maegan Stewart
- Northern Sydney Cancer CentreRoyal North Shore HospitalSydneyNew South WalesAustralia
- School of Health Sciences, Faculty of Medicine and HealthUniversity of SydneySydneyNew South WalesAustralia
| | - Brooke Griffiths
- Northern Sydney Cancer CentreRoyal North Shore HospitalSydneyNew South WalesAustralia
| | - Susan Carroll
- Northern Sydney Cancer CentreRoyal North Shore HospitalSydneyNew South WalesAustralia
- School of Health Sciences, Faculty of Medicine and HealthUniversity of SydneySydneyNew South WalesAustralia
| | - David I Thwaites
- Institute of Medical Physics, School of Physics, University of SydneySydneyNew South WalesAustralia
- Radiotherapy Research GroupLeeds Institute of Medical Research, St James's Hospital and University of LeedsLeedsUK
| |
Collapse
|
5
|
Wahid KA, Kaffey ZY, Farris DP, Humbert-Vidan L, Moreno AC, Rasmussen M, Ren J, Naser MA, Netherton TJ, Korreman S, Balakrishnan G, Fuller CD, Fuentes D, Dohopolski MJ. Artificial intelligence uncertainty quantification in radiotherapy applications - A scoping review. Radiother Oncol 2024; 201:110542. [PMID: 39299574 PMCID: PMC11648575 DOI: 10.1016/j.radonc.2024.110542] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2024] [Revised: 08/18/2024] [Accepted: 09/09/2024] [Indexed: 09/22/2024]
Abstract
BACKGROUND/PURPOSE The use of artificial intelligence (AI) in radiotherapy (RT) is expanding rapidly. However, there exists a notable lack of clinician trust in AI models, underscoring the need for effective uncertainty quantification (UQ) methods. The purpose of this study was to scope existing literature related to UQ in RT, identify areas of improvement, and determine future directions. METHODS We followed the PRISMA-ScR scoping review reporting guidelines. We utilized the population (human cancer patients), concept (utilization of AI UQ), context (radiotherapy applications) framework to structure our search and screening process. We conducted a systematic search spanning seven databases, supplemented by manual curation, up to January 2024. Our search yielded a total of 8980 articles for initial review. Manuscript screening and data extraction was performed in Covidence. Data extraction categories included general study characteristics, RT characteristics, AI characteristics, and UQ characteristics. RESULTS We identified 56 articles published from 2015 to 2024. 10 domains of RT applications were represented; most studies evaluated auto-contouring (50 %), followed by image-synthesis (13 %), and multiple applications simultaneously (11 %). 12 disease sites were represented, with head and neck cancer being the most common disease site independent of application space (32 %). Imaging data was used in 91 % of studies, while only 13 % incorporated RT dose information. Most studies focused on failure detection as the main application of UQ (60 %), with Monte Carlo dropout being the most commonly implemented UQ method (32 %) followed by ensembling (16 %). 55 % of studies did not share code or datasets. CONCLUSION Our review revealed a lack of diversity in UQ for RT applications beyond auto-contouring. Moreover, we identified a clear need to study additional UQ methods, such as conformal prediction. Our results may incentivize the development of guidelines for reporting and implementation of UQ in RT.
Collapse
Affiliation(s)
- Kareem A Wahid
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA; Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Zaphanlene Y Kaffey
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - David P Farris
- Research Medical Library, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Laia Humbert-Vidan
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Amy C Moreno
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | | | - Jintao Ren
- Department of Oncology, Aarhus University Hospital, Denmark
| | - Mohamed A Naser
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Tucker J Netherton
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Stine Korreman
- Department of Oncology, Aarhus University Hospital, Denmark
| | | | - Clifton D Fuller
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - David Fuentes
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA.
| | - Michael J Dohopolski
- Department of Radiation Oncology, The University of Texas Southwestern Medical Center, Dallas, TX, USA.
| |
Collapse
|
6
|
Callens D, Malone C, Carver A, Fiandra C, Gooding MJ, Korreman SS, Matos Dias J, Popple RA, Rocha H, Crijns W, Cardenas CE. Is full-automation in radiotherapy treatment planning ready for take off? Radiother Oncol 2024; 201:110546. [PMID: 39326522 DOI: 10.1016/j.radonc.2024.110546] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2024] [Revised: 09/03/2024] [Accepted: 09/23/2024] [Indexed: 09/28/2024]
Abstract
Radiotherapy treatment planning is undergoing a transformation with the increasing integration of automation. This transition draws parallels with the aviation industry, which has a long-standing history of addressing challenges and opportunities introduced by automated systems. Both fields witness a shift from manual operations to systems capable of operating independently, raising questions about the risks and evolving role of humans within automated workflows. In response to this shift, a working group assembled during the ESTRO Physics Workshop 2023, reflected on parallels to draw lessons for radiotherapy. A taxonomy is proposed, leveraging insights from aviation, that outlines the observed levels of automation within the context of radiotherapy and their corresponding implications for human involvement. Among the common identified risks associated with automation integration are complacency, overreliance, attention tunneling, data overload, a lack of transparency and training. These risks require mitigation strategies. Such strategies include ensuring role complementarity, introducing checklists and safety requirements for human-automation interaction and using automation for cognitive unload and workflow management. Focusing on already automated processes, such as dose calculation and auto-contouring as examples, we have translated lessons learned from aviation. It remains crucial to strike a balance between automation and human involvement. While automation offers the potential for increased efficiency and accuracy, it must be complemented by human oversight, expertise, and critical decision-making. The irreplaceable value of human judgment remains, particularly in complex clinical situations. Learning from aviation, we identify a need for human factors engineering research in radiation oncology and a continued requirement for proactive incident learning.
Collapse
Affiliation(s)
- Dylan Callens
- Laboratory of Experimental Radiotherapy, Catholic University of Leuven, Leuven, Belgium; Department of Radiation Oncology, University Hospitals of Leuven, Leuven, Belgium.
| | - Ciaran Malone
- St.Luke's Radiation Oncology Network, Dublin, Ireland
| | - Antony Carver
- Department of Medical Physics, University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | | | - Mark J Gooding
- Inpictura Ltd, Abingdon, UK; Division of Cancer Sciences, Faculty of Biology, Medicine and Health, The University of Manchester, Manchester, UK
| | - Stine S Korreman
- Department of Clinical Medicine, Aarhus University, Aarhus, Denmark; Danish Center for Particle Therapy, Aarhus University Hospital, Aarhus, Denmark
| | - Joana Matos Dias
- Faculty of Economics and INESC Coimbra, University of Coimbra, Coimbra, Portugal
| | - Richard A Popple
- Department of Radiation Oncology, University of Alabama at Birmingham, Birmingham, AL, USA
| | - Humberto Rocha
- CeBER, Faculty of Economics, University of Coimbra, Coimbra, Portugal
| | - Wouter Crijns
- Laboratory of Experimental Radiotherapy, Catholic University of Leuven, Leuven, Belgium; Department of Radiation Oncology, University Hospitals of Leuven, Leuven, Belgium
| | - Carlos E Cardenas
- Department of Radiation Oncology, University of Alabama at Birmingham, Birmingham, AL, USA
| |
Collapse
|
7
|
Zhou YY, Li YN, Xu JF, Chen B, Li HL, Zheng YX, Pan LS, Cai LM, Wang HM. Rapid Selection of Patients Suitable for Deep Inspiration Breath-Hold Using an Automatic Delineating System and RapidPlan Model in Patients With Left Breast Cancer Undergoing Adjuvant Radiation Therapy With IMRT. Int J Radiat Oncol Biol Phys 2024; 120:1066-1075. [PMID: 38942395 DOI: 10.1016/j.ijrobp.2024.06.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2024] [Revised: 06/08/2024] [Accepted: 06/14/2024] [Indexed: 06/30/2024]
Abstract
PURPOSE This study aimed to determine whether radiation therapy plans created using an automatic delineating system and a RapidPlan (RP) module could rapidly and accurately predict heart doses and benefit from deep inspiratory breath-hold (DIBH) in patients with left breast cancer. METHODS AND MATERIALS One hundred thirty-six clinically approved free breathing (FB) plans for patients with left breast cancer were included, defined as manual delineation-manual plan (MD-MP). A total of 104 of 136 plans were selected for RP model training. A total of 32 of 136 patients were automatically delineated by software, after which the RP generated plans, defined as automatic delineation-RapidPlan (AD-RP). In addition, 40 patients who used DIBH were included to analyze differences in heart benefits from DIBH. RESULTS Two RP models were established for post-breast-conserving surgery (BCS) and post-modified radical mastectomy. There were no significant differences in most of the dosimetric parameters between the MD-MP and AD-RP. The heart doses of the 2 plans were strongly correlated in patients after BCS (0.80 ≤ r ≤ 0.88, P < .05) and moderately correlated in patients after postmodified radical mastectomy (0.46 ≤ r ≤ 0.58, P <.05). The RP model predicted the mean heart dose (MHD) within ± 59.67 cGy and ± 63.32 cGy for patients who underwent the 2 surgeries described above. The heart benefits from DIBH were significantly greater in patients with FB-MHD ≥ 4 Gy than in those with FB-MHD < 4 Gy. CONCLUSIONS The combined automatic delineation RP model allows for the rapid and accurate prediction of heart dose under FB in patients with left breast cancer. FB-MHD ≥ 4 Gy can be used as a dose threshold to select patients suitable for DIBH.
Collapse
Affiliation(s)
- Ying-Ying Zhou
- Department of Radiation Oncology, Nanfang Hospital, Southern Medical University, Guangzhou, China
| | - Yan-Ning Li
- Department of Radiation Oncology, Nanfang Hospital, Southern Medical University, Guangzhou, China
| | - Jin-Feng Xu
- Department of Radiation Oncology, Nanfang Hospital, Southern Medical University, Guangzhou, China
| | - Bo Chen
- Department of Radiation Oncology, Nanfang Hospital, Southern Medical University, Guangzhou, China
| | - Hua-Li Li
- School of Biomedical Engineering, Southern Medical University, Guangzhou, China
| | - Yue-Xin Zheng
- The First School of Clinical Medicine, Southern Medical University, Guangzhou, China
| | - Li-Sheng Pan
- Department of Radiation Oncology, Nanfang Hospital, Southern Medical University, Guangzhou, China
| | - Long-Mei Cai
- Department of Radiation Oncology, Nanfang Hospital, Southern Medical University, Guangzhou, China.
| | - Hong-Mei Wang
- Department of Radiation Oncology, Nanfang Hospital, Southern Medical University, Guangzhou, China.
| |
Collapse
|
8
|
Lee BM, Kim JS, Chang Y, Choi SH, Park JW, Byun HK, Kim YB, Lee IJ, Chang JS. Experience of Implementing Deep Learning-Based Automatic Contouring in Breast Radiation Therapy Planning: Insights From Over 2000 Cases. Int J Radiat Oncol Biol Phys 2024; 119:1579-1589. [PMID: 38431232 DOI: 10.1016/j.ijrobp.2024.02.041] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2023] [Revised: 02/12/2024] [Accepted: 02/18/2024] [Indexed: 03/05/2024]
Abstract
PURPOSE This study evaluated the impact and clinical utility of an auto-contouring system for radiation therapy treatments. METHODS AND MATERIALS The auto-contouring system was implemented in 2019. We evaluated data from 2428 patients who underwent adjuvant breast radiation therapy before and after the system's introduction. We collected the treatment's finalized contours, which were reviewed and revised by a multidisciplinary team. After implementation, the treatment contours underwent a finalization process that involved manual review and adjustment of the initial auto-contours. For the preimplementation group (n = 369), auto-contours were generated retrospectively. We compared the auto-contours and final contours using the Dice similarity coefficient (DSC) and the 95% Hausdorff distance (HD95). RESULTS We analyzed 22,215 structures from final and corresponding auto-contours. The final contours were generally larger, encompassing more slices in the superior or inferior directions. Among organs at risk (OAR), the heart, esophagus, spinal cord, and contralateral breast demonstrated significantly increased DSC and decreased HD95 postimplementation (all P < .05), except for the lungs, which presented inaccurate segmentation. Among target volumes, CTVn_L2, L3, L4, and the internal mammary node showed increased DSC and decreased HD95 postimplementation (all P < .05), although the increase was less pronounced than the OAR outcomes. The analysis also covered factors contributing to significant differences, pattern identification, and outlier detection. CONCLUSIONS In our study, the adoption of an auto-contouring system was associated with an increased reliance on automated settings, underscoring its utility and the potential risk of automation bias. Given these findings, we underscore the importance of considering the integration of stringent risk assessments and quality management strategies as a precautionary measure for the optimal use of such systems.
Collapse
Affiliation(s)
- Byung Min Lee
- Department of Radiation Oncology, Yonsei Cancer Center, Yonsei University College of Medicine, Seoul, Republic of Korea; Department of Radiation Oncology, Uijeongbu St. Mary's Hospital, Catholic University of Korea, Seoul, Republic of Korea
| | - Jin Sung Kim
- Department of Radiation Oncology, Yonsei Cancer Center, Yonsei University College of Medicine, Seoul, Republic of Korea
| | | | - Seo Hee Choi
- Department of Radiation Oncology, Yonsei Cancer Center, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Jong Won Park
- Department of Radiation Oncology, Yonsei Cancer Center, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Hwa Kyung Byun
- Department of Radiation Oncology, Yonsei Cancer Center, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Yong Bae Kim
- Department of Radiation Oncology, Yonsei Cancer Center, Yonsei University College of Medicine, Seoul, Republic of Korea
| | - Ik Jae Lee
- Department of Radiation Oncology, Yonsei Cancer Center, Yonsei University College of Medicine, Seoul, Republic of Korea.
| | - Jee Suk Chang
- Department of Radiation Oncology, Yonsei Cancer Center, Yonsei University College of Medicine, Seoul, Republic of Korea.
| |
Collapse
|
9
|
Wahid KA, Kaffey ZY, Farris DP, Humbert-Vidan L, Moreno AC, Rasmussen M, Ren J, Naser MA, Netherton TJ, Korreman S, Balakrishnan G, Fuller CD, Fuentes D, Dohopolski MJ. Artificial Intelligence Uncertainty Quantification in Radiotherapy Applications - A Scoping Review. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2024:2024.05.13.24307226. [PMID: 38798581 PMCID: PMC11118597 DOI: 10.1101/2024.05.13.24307226] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/29/2024]
Abstract
Background/purpose The use of artificial intelligence (AI) in radiotherapy (RT) is expanding rapidly. However, there exists a notable lack of clinician trust in AI models, underscoring the need for effective uncertainty quantification (UQ) methods. The purpose of this study was to scope existing literature related to UQ in RT, identify areas of improvement, and determine future directions. Methods We followed the PRISMA-ScR scoping review reporting guidelines. We utilized the population (human cancer patients), concept (utilization of AI UQ), context (radiotherapy applications) framework to structure our search and screening process. We conducted a systematic search spanning seven databases, supplemented by manual curation, up to January 2024. Our search yielded a total of 8980 articles for initial review. Manuscript screening and data extraction was performed in Covidence. Data extraction categories included general study characteristics, RT characteristics, AI characteristics, and UQ characteristics. Results We identified 56 articles published from 2015-2024. 10 domains of RT applications were represented; most studies evaluated auto-contouring (50%), followed by image-synthesis (13%), and multiple applications simultaneously (11%). 12 disease sites were represented, with head and neck cancer being the most common disease site independent of application space (32%). Imaging data was used in 91% of studies, while only 13% incorporated RT dose information. Most studies focused on failure detection as the main application of UQ (60%), with Monte Carlo dropout being the most commonly implemented UQ method (32%) followed by ensembling (16%). 55% of studies did not share code or datasets. Conclusion Our review revealed a lack of diversity in UQ for RT applications beyond auto-contouring. Moreover, there was a clear need to study additional UQ methods, such as conformal prediction. Our results may incentivize the development of guidelines for reporting and implementation of UQ in RT.
Collapse
Affiliation(s)
- Kareem A. Wahid
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Zaphanlene Y. Kaffey
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - David P. Farris
- Research Medical Library, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Laia Humbert-Vidan
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Amy C. Moreno
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | | | - Jintao Ren
- Department of Oncology, Aarhus University Hospital, Denmark
| | - Mohamed A. Naser
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Tucker J. Netherton
- Department of Radiation Physics, University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Stine Korreman
- Department of Oncology, Aarhus University Hospital, Denmark
| | | | - Clifton D. Fuller
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - David Fuentes
- Department of Imaging Physics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | - Michael J. Dohopolski
- Department of Radiation Oncology, The University of Texas Southwestern Medical Center, Dallas, Texas, USA
| |
Collapse
|
10
|
Li J, Song Y, Wu Y, Liang L, Li G, Bai S. Clinical evaluation on automatic segmentation results of convolutional neural networks in rectal cancer radiotherapy. Front Oncol 2023; 13:1158315. [PMID: 37731629 PMCID: PMC10508953 DOI: 10.3389/fonc.2023.1158315] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Accepted: 08/11/2023] [Indexed: 09/22/2023] Open
Abstract
Purpose Image segmentation can be time-consuming and lacks consistency between different oncologists, which is essential in conformal radiotherapy techniques. We aimed to evaluate automatic delineation results generated by convolutional neural networks (CNNs) from geometry and dosimetry perspectives and explore the reliability of these segmentation tools in rectal cancer. Methods Forty-seven rectal cancer cases treated from February 2018 to April 2019 were randomly collected retrospectively in our cancer center. The oncologists delineated regions of interest (ROIs) on planning CT images as the ground truth, including clinical target volume (CTV), bladder, small intestine, and femoral heads. The corresponding automatic segmentation results were generated by DeepLabv3+ and ResUNet, and we also used Atlas-Based Autosegmentation (ABAS) software for comparison. The geometry evaluation was carried out using the volumetric Dice similarity coefficient (DSC) and surface DSC, and critical dose parameters were assessed based on replanning optimized by clinically approved or automatically generated CTVs and organs at risk (OARs), i.e., the Planref and Plantest. Pearson test was used to explore the correlation between geometric metrics and dose parameters. Results In geometric evaluation, DeepLabv3+ performed better in DCS metrics for the CTV (volumetric DSC, mean = 0.96, P< 0.01; surface DSC, mean = 0.78, P< 0.01) and small intestine (volumetric DSC, mean = 0.91, P< 0.01; surface DSC, mean = 0.62, P< 0.01), ResUNet had advantages in volumetric DSC of the bladder (mean = 0.97, P< 0.05). For critical dose parameters analysis between Planref and Plantest, there was a significant difference for target volumes (P< 0.01), and no significant difference was found for the ResUNet-generated small intestine (P > 0.05). For the correlation test, a negative correlation was found between DSC metrics (volumetric, surface DSC) and dosimetric parameters (δD95, δD95, HI, CI) for target volumes (P< 0.05), and no significant correlation was found for most tests of OARs (P > 0.05). Conclusions CNNs show remarkable repeatability and time-saving in automatic segmentation, and their accuracy also has a certain potential in clinical practice. Meanwhile, clinical aspects, such as dose distribution, may need to be considered when comparing the performance of auto-segmentation methods.
Collapse
Affiliation(s)
- Jing Li
- Radiotherapy Physics & Technology Center, Cancer Center, West China Hospital, Sichuan University, Chengdu, China
| | - Ying Song
- Radiotherapy Physics & Technology Center, Cancer Center, West China Hospital, Sichuan University, Chengdu, China
- Machine Intelligence Laboratory, College of Computer Science, Chengdu, China
| | - Yongchang Wu
- Radiotherapy Physics & Technology Center, Cancer Center, West China Hospital, Sichuan University, Chengdu, China
| | - Lan Liang
- Radiotherapy Physics & Technology Center, Cancer Center, West China Hospital, Sichuan University, Chengdu, China
| | - Guangjun Li
- Radiotherapy Physics & Technology Center, Cancer Center, West China Hospital, Sichuan University, Chengdu, China
| | - Sen Bai
- Radiotherapy Physics & Technology Center, Cancer Center, West China Hospital, Sichuan University, Chengdu, China
| |
Collapse
|