1
|
Xue C, Yuan J, Lo GG, Poon DMC, Chu WC. Computational analysis of variability and uncertainty in the clinical reference on magnetic resonance imaging radiomics: modelling and performance. Vis Comput Ind Biomed Art 2024; 7:28. [PMID: 39557758 PMCID: PMC11573982 DOI: 10.1186/s42492-024-00180-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2024] [Accepted: 10/30/2024] [Indexed: 11/20/2024] Open
Abstract
To conduct a computational investigation to explore the influence of clinical reference uncertainty on magnetic resonance imaging (MRI) radiomics feature selection, modelling, and performance. This study used two sets of publicly available prostate cancer MRI = radiomics data (Dataset 1: n = 260; Dataset 2: n = 100) with Gleason score clinical references. Each dataset was divided into training and holdout testing datasets at a ratio of 7:3 and analysed independently. The clinical references of the training set were permuted at different levels (increments of 5%) and repeated 20 times. Four feature selection algorithms and two classifiers were used to construct the models. Cross-validation was employed for training, while a separate hold-out testing set was used for evaluation. The Jaccard similarity coefficient was used to evaluate feature selection, while the area under the curve (AUC) and accuracy were used to assess model performance. An analysis of variance test with Bonferroni correction was conducted to compare the metrics of each model. The consistency of the feature selection performance decreased substantially with the clinical reference permutation. AUCs of the trained models with permutation particularly after 20% were significantly lower (Dataset 1 (with ≥ 20% permutation): 0.67, and Dataset 2 (≥ 20% permutation): 0.74), compared to the AUC of models without permutation (Dataset 1: 0.94, Dataset 2: 0.97). The performances of the models were also associated with larger uncertainties and an increasing number of permuted clinical references. Clinical reference uncertainty can substantially influence MRI radiomic feature selection and modelling. The high accuracy of clinical references should be helpful in building reliable and robust radiomic models. Careful interpretation of the model performance is necessary, particularly for high-dimensional data.
Collapse
Affiliation(s)
- Cindy Xue
- Research Department, Hong Kong Sanatorium and Hospital, Hong Kong, China
- Department of Imaging and Interventional Radiology, The Chinese University of Hong Kong, Hong Kong, China
| | - Jing Yuan
- Research Department, Hong Kong Sanatorium and Hospital, Hong Kong, China
| | - Gladys G Lo
- Department of Diagnostic and Interventional Radiology, Hong Kong Sanatorium and Hospital, Hong Kong, China
| | - Darren M C Poon
- Comprehensive Oncology Centre, Hong Kong Sanatorium and Hospital, Hong Kong, China
| | - Winnie Cw Chu
- Department of Imaging and Interventional Radiology, The Chinese University of Hong Kong, Hong Kong, China.
| |
Collapse
|
2
|
Bourdillon AT. Computer Vision-Radiomics & Pathognomics. Otolaryngol Clin North Am 2024; 57:719-751. [PMID: 38910065 DOI: 10.1016/j.otc.2024.05.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/25/2024]
Abstract
The role of computer vision in extracting radiographic (radiomics) and histopathologic (pathognomics) features is an extension of molecular biomarkers that have been foundational to our understanding across the spectrum of head and neck disorders. Especially within head and neck cancers, machine learning and deep learning applications have yielded advances in the characterization of tumor features, nodal features, and various outcomes. This review aims to overview the landscape of radiomic and pathognomic applications, informing future work to address gaps. Novel methodologies will be needed to potentially engineer ways of integrating multidimensional data inputs to examine disease features to guide prognosis comprehensively and ultimately clinical management.
Collapse
Affiliation(s)
- Alexandra T Bourdillon
- Department of Otolaryngology-Head & Neck Surgery, University of California-San Francisco, San Francisco, CA 94115, USA.
| |
Collapse
|
3
|
Xue C, Yuan J, Lo GG, Chang ATY, Poon DMC, Wong OL, Zhou Y, Chu WCW. Radiomics feature reliability assessed by intraclass correlation coefficient: a systematic review. Quant Imaging Med Surg 2021; 11:4431-4460. [PMID: 34603997 DOI: 10.21037/qims-21-86] [Citation(s) in RCA: 86] [Impact Index Per Article: 21.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2021] [Accepted: 05/17/2021] [Indexed: 12/13/2022]
Abstract
Radiomics research is rapidly growing in recent years, but more concerns on radiomics reliability are also raised. This review attempts to update and overview the current status of radiomics reliability research in the ever expanding medical literature from the perspective of a single reliability metric of intraclass correlation coefficient (ICC). To conduct this systematic review, Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed. After literature search and selection, a total of 481 radiomics studies using CT, PET, or MRI, covering a wide range of subject and disease types, were included for review. In these highly heterogeneous studies, feature reliability to image segmentation was much more investigated than reliability to other factors, such as image acquisition, reconstruction, post-processing, and feature quantification. The reported ICCs also suggested high radiomics feature reliability to image segmentation. Image acquisition was found to introduce much more feature variability than image segmentation, in particular for MRI, based on the reported ICC values. Image post-processing and feature quantification yielded different levels of radiomics reliability and might be used to mitigate image acquisition-induced variability. Some common flaws and pitfalls in ICC use were identified, and suggestions on better ICC use were given. Due to the extremely high study heterogeneities and possible risks of bias, the degree of radiomics feature reliability that has been achieved could not yet be safely synthesized or derived in this review. More future researches on radiomics reliability are warranted.
Collapse
Affiliation(s)
- Cindy Xue
- Medical Physics and Research Department, Hong Kong Sanatorium & Hospital, Happy Valley, Hong Kong, China.,Department of Imaging and Interventional Radiology, Faculty of Medicine, The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong, China
| | - Jing Yuan
- Medical Physics and Research Department, Hong Kong Sanatorium & Hospital, Happy Valley, Hong Kong, China
| | - Gladys G Lo
- Department of Diagnostic & Interventional Radiology, Hong Kong Sanatorium & Hospital, Happy Valley, Hong Kong, China
| | - Amy T Y Chang
- Comprehensive Oncology Centre, Hong Kong Sanatorium & Hospital, Happy Valley, Hong Kong, China
| | - Darren M C Poon
- Comprehensive Oncology Centre, Hong Kong Sanatorium & Hospital, Happy Valley, Hong Kong, China
| | - Oi Lei Wong
- Medical Physics and Research Department, Hong Kong Sanatorium & Hospital, Happy Valley, Hong Kong, China
| | - Yihang Zhou
- Medical Physics and Research Department, Hong Kong Sanatorium & Hospital, Happy Valley, Hong Kong, China
| | - Winnie C W Chu
- Department of Imaging and Interventional Radiology, Faculty of Medicine, The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong, China
| |
Collapse
|
4
|
Bologna M, Corino V, Calareso G, Tenconi C, Alfieri S, Iacovelli NA, Cavallo A, Cavalieri S, Locati L, Bossi P, Romanello DA, Ingargiola R, Rancati T, Pignoli E, Sdao S, Pecorilla M, Facchinetti N, Trama A, Licitra L, Mainardi L, Orlandi E. Baseline MRI-Radiomics Can Predict Overall Survival in Non-Endemic EBV-Related Nasopharyngeal Carcinoma Patients. Cancers (Basel) 2020; 12:E2958. [PMID: 33066161 PMCID: PMC7601980 DOI: 10.3390/cancers12102958] [Citation(s) in RCA: 42] [Impact Index Per Article: 8.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2020] [Revised: 10/07/2020] [Accepted: 10/08/2020] [Indexed: 02/06/2023] Open
Abstract
Advanced stage nasopharyngeal cancer (NPC) shows highly variable treatment outcomes, suggesting the need for independent prognostic factors. This study aims at developing a magnetic resonance imaging (MRI)-based radiomic signature as a prognostic marker for different clinical endpoints in NPC patients from non-endemic areas. A total 136 patients with advanced NPC and available MRI imaging (T1-weighted and T2-weighted) were selected. For each patient, 2144 radiomic features were extracted from the main tumor and largest lymph node. A multivariate Cox regression model was trained on a subset of features to obtain a radiomic signature for overall survival (OS), which was also applied for the prognosis of other clinical endpoints. Validation was performed using 10-fold cross-validation. The added prognostic value of the radiomic features to clinical features and volume was also evaluated. The radiomics-based signature had good prognostic power for OS and loco-regional recurrence-free survival (LRFS), with C-index of 0.68 and 0.72, respectively. In all the cases, the addition of radiomics to clinical features improved the prognostic performance. Radiomic features can provide independent prognostic information in NPC patients from non-endemic areas.
Collapse
Affiliation(s)
- Marco Bologna
- Department of Electronics, Information and Bioengineering (DEIB) Politecnico di Milano, 20133 Milan, Italy; (V.C.); (L.M.)
| | - Valentina Corino
- Department of Electronics, Information and Bioengineering (DEIB) Politecnico di Milano, 20133 Milan, Italy; (V.C.); (L.M.)
| | - Giuseppina Calareso
- Department of Radiology, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy; (G.C.); (S.S.)
| | - Chiara Tenconi
- Department of Oncology and Hemato-Oncology, Università degli studi di Milano, 20133 Milan, Italy; (C.T.); (L.L.)
- Medical Physics Unit, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy; (A.C.); (D.A.R.); (R.I.); (E.P.)
| | - Salvatore Alfieri
- Head and Neck Cancer Medical Oncology 3 Department, Fondazione IRCCS Istituto Nazionale dei Tumori di Milano, 20133 Milan, Italy; (S.A.); (S.C.); (L.L.)
| | - Nicola Alessandro Iacovelli
- Radiotherapy 2 Unit, Fondazione IRCCS Istituto Nazionale dei Tumori di Milano, 20133 Milan, Italy; (N.A.I.); (N.F.); (E.O.)
| | - Anna Cavallo
- Medical Physics Unit, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy; (A.C.); (D.A.R.); (R.I.); (E.P.)
| | - Stefano Cavalieri
- Head and Neck Cancer Medical Oncology 3 Department, Fondazione IRCCS Istituto Nazionale dei Tumori di Milano, 20133 Milan, Italy; (S.A.); (S.C.); (L.L.)
| | - Laura Locati
- Head and Neck Cancer Medical Oncology 3 Department, Fondazione IRCCS Istituto Nazionale dei Tumori di Milano, 20133 Milan, Italy; (S.A.); (S.C.); (L.L.)
| | - Paolo Bossi
- Department of Medical and Surgical Specialties, Radiological Sciences and Public Health, University of Brescia, ASST Spedali Civili, 25123 Brescia, Italy;
| | - Domenico Attilio Romanello
- Medical Physics Unit, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy; (A.C.); (D.A.R.); (R.I.); (E.P.)
| | - Rossana Ingargiola
- Medical Physics Unit, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy; (A.C.); (D.A.R.); (R.I.); (E.P.)
| | - Tiziana Rancati
- Prostate Cancer Program, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy;
| | - Emanuele Pignoli
- Medical Physics Unit, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy; (A.C.); (D.A.R.); (R.I.); (E.P.)
| | - Silvana Sdao
- Department of Radiology, Fondazione IRCCS Istituto Nazionale dei Tumori, 20133 Milan, Italy; (G.C.); (S.S.)
| | - Mattia Pecorilla
- Post-Graduate School in Radiodiagnostics, Università degli Studi di Milano, 20133 Milan, Italy;
| | - Nadia Facchinetti
- Radiotherapy 2 Unit, Fondazione IRCCS Istituto Nazionale dei Tumori di Milano, 20133 Milan, Italy; (N.A.I.); (N.F.); (E.O.)
| | - Annalisa Trama
- Research Department, Fondazione IRCCS Istituto Nazionale dei Tumori di Milano, 20133 Milan, Italy;
| | - Lisa Licitra
- Department of Oncology and Hemato-Oncology, Università degli studi di Milano, 20133 Milan, Italy; (C.T.); (L.L.)
- Head and Neck Cancer Medical Oncology 3 Department, Fondazione IRCCS Istituto Nazionale dei Tumori di Milano, 20133 Milan, Italy; (S.A.); (S.C.); (L.L.)
| | - Luca Mainardi
- Department of Electronics, Information and Bioengineering (DEIB) Politecnico di Milano, 20133 Milan, Italy; (V.C.); (L.M.)
| | - Ester Orlandi
- Radiotherapy 2 Unit, Fondazione IRCCS Istituto Nazionale dei Tumori di Milano, 20133 Milan, Italy; (N.A.I.); (N.F.); (E.O.)
| |
Collapse
|