1
|
McNitt-Gray M, Napel S, Jaggi A, Mattonen SA, Hadjiiski L, Muzi M, Goldgof D, Balagurunathan Y, Pierce LA, Kinahan PE, Jones EF, Nguyen A, Virkud A, Chan HP, Emaminejad N, Wahi-Anwar M, Daly M, Abdalah M, Yang H, Lu L, Lv W, Rahmim A, Gastounioti A, Pati S, Bakas S, Kontos D, Zhao B, Kalpathy-Cramer J, Farahani K. Standardization in Quantitative Imaging: A Multicenter Comparison of Radiomic Features from Different Software Packages on Digital Reference Objects and Patient Data Sets. ACTA ACUST UNITED AC 2021; 6:118-128. [PMID: 32548288 PMCID: PMC7289262 DOI: 10.18383/j.tom.2019.00031] [Citation(s) in RCA: 53] [Impact Index Per Article: 17.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
Abstract
Radiomic features are being increasingly studied for clinical applications. We aimed to assess the agreement among radiomic features when computed by several groups by using different software packages under very tightly controlled conditions, which included standardized feature definitions and common image data sets. Ten sites (9 from the NCI's Quantitative Imaging Network] positron emission tomography–computed tomography working group plus one site from outside that group) participated in this project. Nine common quantitative imaging features were selected for comparison including features that describe morphology, intensity, shape, and texture. The common image data sets were: three 3D digital reference objects (DROs) and 10 patient image scans from the Lung Image Database Consortium data set using a specific lesion in each scan. Each object (DRO or lesion) was accompanied by an already-defined volume of interest, from which the features were calculated. Feature values for each object (DRO or lesion) were reported. The coefficient of variation (CV), expressed as a percentage, was calculated across software packages for each feature on each object. Thirteen sets of results were obtained for the DROs and patient data sets. Five of the 9 features showed excellent agreement with CV < 1%; 1 feature had moderate agreement (CV < 10%), and 3 features had larger variations (CV ≥ 10%) even after attempts at harmonization of feature calculations. This work highlights the value of feature definition standardization as well as the need to further clarify definitions for some features.
Collapse
Affiliation(s)
- M McNitt-Gray
- David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA
| | - S Napel
- Stanford University School of Medicine, Stanford, CA
| | - A Jaggi
- Stanford University School of Medicine, Stanford, CA
| | - S A Mattonen
- Stanford University School of Medicine, Stanford, CA.,The University of Western Ontario, Canada
| | | | - M Muzi
- University of Washington, Seattle, WA
| | - D Goldgof
- University of South Florida, Tampa, FL
| | | | | | | | - E F Jones
- UC San Francisco, School of Medicine, San Francisco, CA
| | - A Nguyen
- UC San Francisco, School of Medicine, San Francisco, CA
| | - A Virkud
- University of Michigan, Ann Arbor, MI
| | - H P Chan
- University of Michigan, Ann Arbor, MI
| | - N Emaminejad
- David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA
| | - M Wahi-Anwar
- David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA
| | - M Daly
- David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA
| | - M Abdalah
- H. Lee Moffitt Cancer Center, Tampa, FL
| | - H Yang
- Columbia University Medical Center, New York, NY
| | - L Lu
- Columbia University Medical Center, New York, NY
| | - W Lv
- BC Cancer Research Centre, Vancouver, BC, Canada
| | - A Rahmim
- BC Cancer Research Centre, Vancouver, BC, Canada
| | - A Gastounioti
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA
| | - S Pati
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA
| | - S Bakas
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA
| | - D Kontos
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA
| | - B Zhao
- Columbia University Medical Center, New York, NY
| | | | | |
Collapse
|