1
|
Maes D, Gates EDH, Meyer J, Kang J, Nguyen BNT, Lavilla M, Melancon D, Weg ES, Tseng YD, Lim A, Bowen SR. Framework for Radiation Oncology Department-wide Evaluation and Implementation of Commercial Artificial Intelligence Autocontouring. Pract Radiat Oncol 2024; 14:e150-e158. [PMID: 37935308 DOI: 10.1016/j.prro.2023.10.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2023] [Revised: 10/19/2023] [Accepted: 10/22/2023] [Indexed: 11/09/2023]
Abstract
PURPOSE Artificial intelligence (AI)-based autocontouring in radiation oncology has potential benefits such as standardization and time savings. However, commercial AI solutions require careful evaluation before clinical integration. We developed a multidimensional evaluation method to test pretrained AI-based automated contouring solutions across a network of clinics. METHODS AND MATERIALS Curated data included 121 patient planning computed tomography (CT) scans with a total of 859 clinically approved contours used for treatment from 4 clinics. Regions of interest (ROIs) were generated with 3 commercial AI-based automated contouring software solutions (AI1, AI2, AI3) spanning the following disease sites: brain, head and neck (H&N), thorax, abdomen, and pelvis. Quantitative agreement between AI-generated and clinical contours was measured by Dice similarity coefficient (DSC) and Hausdorff distance (HD). Qualitative assessment was performed by multiple experts scoring blinded AI-contours using a Likert scale. Workflow and usability surveying was also conducted. RESULTS AI1, AI2, and AI3 contours had high quantitative agreement in 27.8%, 32.8%, and 34.1% of cases (DSC >0.9), performing well in pelvis (median DSC = 0.86/0.88/0.91) and thorax (median DSC = 0.91/0.89/0.91). All 3 solutions had low quantitative agreement in 7.4%, 8.8%, and 6.1% of cases (DSC <0.5), performing worse in brain (median DSC = 0.65/0.78/0.75) and H&N (median DSC = 0.76/0.80/0.81). Qualitatively, AI1 and AI2 contours were acceptable (rated 1-2) with at most minor edits in 70.7% and 74.6% of ROIs (2906 ratings), higher for abdomen (AI1: 79.2%) and thorax (AI2: 90.2%), and lower for H&N (29.0/35.6%). An end-user survey showed strong user preference for full automation and mixed preferences for accuracy versus total number of structures generated. CONCLUSIONS Our evaluation method provided a comprehensive analysis of both quantitative and qualitative measures of commercially available pretrained AI autocontouring algorithms. The evaluation framework served as a roadmap for clinical integration that aligned with user workflow preference.
Collapse
Affiliation(s)
- Dominic Maes
- Department of Radiation Oncology, Fred Hutchinson Cancer Center, Seattle, Washington; Department of Radiation Oncology, University of Washington, Seattle, Washington.
| | - Evan D H Gates
- Department of Radiation Oncology, University of Washington, Seattle, Washington
| | - Juergen Meyer
- Department of Radiation Oncology, Fred Hutchinson Cancer Center, Seattle, Washington; Department of Radiation Oncology, University of Washington, Seattle, Washington
| | - John Kang
- Department of Radiation Oncology, University of Washington, Seattle, Washington
| | - Bao-Ngoc Thi Nguyen
- Department of Radiation Oncology, Fred Hutchinson Cancer Center, Seattle, Washington
| | - Myra Lavilla
- Department of Radiation Oncology, Fred Hutchinson Cancer Center, Seattle, Washington
| | - Dustin Melancon
- Department of Radiation Oncology, University of Washington, Seattle, Washington
| | - Emily S Weg
- Department of Radiation Oncology, Fred Hutchinson Cancer Center, Seattle, Washington; Department of Radiation Oncology, University of Washington, Seattle, Washington
| | - Yolanda D Tseng
- Department of Radiation Oncology, University of Washington, Seattle, Washington; Clinical Research Division, Fred Hutchinson Cancer Center, Seattle, Washington
| | - Andrew Lim
- Department of Radiation Oncology, Fred Hutchinson Cancer Center, Seattle, Washington; Department of Radiation Oncology, University of Washington, Seattle, Washington; Department of Radiation Oncology, University of Southern California, Los Angeles, California
| | - Stephen R Bowen
- Department of Radiation Oncology, Fred Hutchinson Cancer Center, Seattle, Washington; Department of Radiation Oncology, University of Washington, Seattle, Washington; Department of Radiology, University of Washington, Seattle, Washington
| |
Collapse
|
2
|
Rekulapelli A, Desai RP, Narayan A, Martin LW, Hall R, Larner JM, Balkrishnan R. Racial and Treatment Center Differences on Time to Treatment Initiation for Nonsmall Cell Lung Cancer Patients Receiving Radiation Therapy As an Initial Treatment. Health Equity 2022; 6:603-609. [PMID: 36081886 PMCID: PMC9448516 DOI: 10.1089/heq.2022.0104] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/20/2022] [Indexed: 11/13/2022] Open
Abstract
Objective: Because time to treatment has been shown to be associated with increase in the risk of death for Non Small Cell Lung Cancer (NSCLC) patients, we examined the prevalence and magnitude of racial disparities in mean time to radiation therapy (TTRT) for Stage I-III non-small cell lung cancer patients across a variety of treatment facilities. Methods: Utilizing the United States National Cancer Database (NCDB), we determined differences in TTRT between different races and different treatment facilities. Results: Concordant with past research, we found that non-White patients and patients treated at academic facilities, regardless of race, have longer mean TTRT, and that racial disparities in TTRT extend across all treatment facilities (all p<0.05). Conclusions: These findings shed light on the potential presence of and impact of structural racism on patients seeking cancer treatment, and the need for further investigation behind the reasonings behind longer TTI for non-White patients. To elucidate the real-world applicability of these results, further investigation into the societal determinants that perpetuate disparity in time to radiation therapy, and potential interventions in the clinical setting to improve cultural and racial sensitivity among healthcare professionals is recommended.
Collapse
Affiliation(s)
- Akhil Rekulapelli
- University of Virginia School of Medicine, Charlottesville, Virginia, USA
| | - Raj P. Desai
- University of Virginia School of Medicine, Charlottesville, Virginia, USA
| | - Aditya Narayan
- University of Virginia School of Medicine, Charlottesville, Virginia, USA
| | - Linda W. Martin
- University of Virginia School of Medicine, Charlottesville, Virginia, USA
| | - Richard Hall
- University of Virginia School of Medicine, Charlottesville, Virginia, USA
| | - James M. Larner
- University of Virginia School of Medicine, Charlottesville, Virginia, USA
| | - Rajesh Balkrishnan
- University of Virginia School of Medicine, Charlottesville, Virginia, USA
| |
Collapse
|
3
|
Kouka M, Engelhardt M, Wittig A, Schultze-Mosgau S, Ernst T, Guntinas-Lichius O. No impact of time to treatment initiation for head and neck cancer in a tertiary university center in 2003, 2008 and 2013. Eur Arch Otorhinolaryngol 2022; 279:4549-4560. [PMID: 35488907 PMCID: PMC9363340 DOI: 10.1007/s00405-022-07392-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2021] [Accepted: 04/05/2022] [Indexed: 11/30/2022]
Abstract
Background This retrospective study investigated factors influencing time to treatment initiation (TTI) and the influence of TTI on overall survival (OS) of primary head and neck cancer (HNC) patients in cohorts from 2003, 2008 and 2013. Methods Two hundred and ninenty seven patients (78.8% men; median age: 62 years) were included. Kaplan–Meier analyses and multivariate Cox regression were performed to investigate OS. Results Mean times to treatment initiation (TTI) of 2003, 2008 and 2013 were 17.11 ± 18.00, 30.26 ± 30.08 and 17.30 ± 37.04 days, respectively. TTI for patients with T3/T4 tumors was higher than for T1/T2 (p = 0.010). In univariable analysis on OS, TTI > 5 days showed lower OS (p = 0.047). In multivariate analysis, longer TTI had no influence on lower OS [hazard ratio (HR) 1.236; 95% CI 0.852–1.791; p = 0.264], but male gender [HR 2.342; 95% CI 1.229–4.466; p = 0.010], increased age [HR 1.026; 95% CI 1.008–1.045; p = 0.005], M1 [HR 5.823; 95% CI 2.252–15.058; p = 0.003], hypopharynx tumor [HR 2.508; 95% CI 1.571–4.003; p < 0.001] and oral cavity tumor [HR 1.712; CI 1.101–2.661; p = 0.017]. The year of treatment showed no significant effect on OS. Conclusion Median TTI seemed to be very short compared to other studies. There was no clear trend in the impact of TTI on OS from 2003 to 2013. Supplementary Information The online version contains supplementary material available at 10.1007/s00405-022-07392-w.
Collapse
Affiliation(s)
- Mussab Kouka
- Department of Otorhinolaryngology, Jena University Hospital, Am Klinikum 1, 07747, Jena, Germany
| | - Max Engelhardt
- Department of Otorhinolaryngology, Jena University Hospital, Am Klinikum 1, 07747, Jena, Germany
| | - Andrea Wittig
- Department of Radiotherapy and Radiation Oncology, Jena University Hospital, Jena, Germany
| | - Stefan Schultze-Mosgau
- Department of Oromaxillofacial Surgery and Plastic Surgery, Jena University Hospital, Jena, Germany
| | - Thomas Ernst
- University Tumor Center, Jena University Hospital, Jena, Germany
| | - Orlando Guntinas-Lichius
- Department of Otorhinolaryngology, Jena University Hospital, Am Klinikum 1, 07747, Jena, Germany.
| |
Collapse
|
4
|
Suzuki H, Tamaki T, Tsuzuki H, Nishio M, Nishikawa D, Beppu S, Hanai N. Association between treatment package time and clinical predictors in oropharyngeal cancer. Medicine (Baltimore) 2020; 99:e22244. [PMID: 32991419 PMCID: PMC7523759 DOI: 10.1097/md.0000000000022244] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/29/2022] Open
Abstract
The aim of the present study is to investigate the association of treatment package time with the survival outcomes and clinical parameters in patients with oropharyngeal squamous cell carcinoma.A total of 49 patients who underwent definitive treatment were enrolled. The treatment package time was calculated in days from the start of any treatment to the completion of all treatments, including postoperative treatment and salvage surgery for residual tumor.On univariate analyzes, treatment package time ≥118 days, sake index ≥60, Brinkman index ≥450, maximum standardized uptake value ≥42.45, and the presence of synchronous cancer were significantly associated with shorter oropharyngeal squamous cell carcinoma-specific survival. Moreover, a treatment package time of ≥118 days was significantly correlated with shorter overall survival and distant metastasis-free survival. On multivariate analyzes, Brinkman index ≥450 was significantly associated with shorter oropharyngeal squamous cell carcinoma-specific and locoregional recurrence-free survival, and the presence of synchronous cancer was significantly associated with shorter overall and distant metastasis-free survival.In conclusion, a relatively long treatment package time was a predictor of low survival outcomes in oropharyngeal squamous cell carcinoma by univariate analysis.
Collapse
Affiliation(s)
- Hidenori Suzuki
- Department of Head and Neck Surgery, Aichi Cancer Center Hospital
| | - Tsuneo Tamaki
- Department of Radiology, Nagoya Radiological Diagnosis Foundation
| | - Hidenori Tsuzuki
- Department of Otorhinolaryngology, Nagoya University Graduate School of Medicine, Nagoya, Japan
| | - Masami Nishio
- Department of Radiology, Nagoya Radiological Diagnosis Foundation
| | | | - Shintaro Beppu
- Department of Head and Neck Surgery, Aichi Cancer Center Hospital
| | - Nobuhiro Hanai
- Department of Head and Neck Surgery, Aichi Cancer Center Hospital
| |
Collapse
|