1
|
Vahdati S, Khosravi B, Mahmoudi E, Zhang K, Rouzrokh P, Faghani S, Moassefi M, Tahmasebi A, Andriole KP, Chang P, Farahani K, Flores MG, Folio L, Houshmand S, Giger ML, Gichoya JW, Erickson BJ. A Guideline for Open-Source Tools to Make Medical Imaging Data Ready for Artificial Intelligence Applications: A Society of Imaging Informatics in Medicine (SIIM) Survey. J Imaging Inform Med 2024:10.1007/s10278-024-01083-0. [PMID: 38558368 DOI: 10.1007/s10278-024-01083-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/11/2024] [Revised: 02/29/2024] [Accepted: 03/08/2024] [Indexed: 04/04/2024]
Abstract
In recent years, the role of Artificial Intelligence (AI) in medical imaging has become increasingly prominent, with the majority of AI applications approved by the FDA being in imaging and radiology in 2023. The surge in AI model development to tackle clinical challenges underscores the necessity for preparing high-quality medical imaging data. Proper data preparation is crucial as it fosters the creation of standardized and reproducible AI models while minimizing biases. Data curation transforms raw data into a valuable, organized, and dependable resource and is a fundamental process to the success of machine learning and analytical projects. Considering the plethora of available tools for data curation in different stages, it is crucial to stay informed about the most relevant tools within specific research areas. In the current work, we propose a descriptive outline for different steps of data curation while we furnish compilations of tools collected from a survey applied among members of the Society of Imaging Informatics (SIIM) for each of these stages. This collection has the potential to enhance the decision-making process for researchers as they select the most appropriate tool for their specific tasks.
Collapse
Affiliation(s)
- Sanaz Vahdati
- Artificial Intelligence Laboratory, Department of Radiology, Mayo Clinic, 200 1st Street, SW, Rochester, MN, 55905, USA
| | - Bardia Khosravi
- Artificial Intelligence Laboratory, Department of Radiology, Mayo Clinic, 200 1st Street, SW, Rochester, MN, 55905, USA
| | - Elham Mahmoudi
- Artificial Intelligence Laboratory, Department of Radiology, Mayo Clinic, 200 1st Street, SW, Rochester, MN, 55905, USA
| | - Kuan Zhang
- Artificial Intelligence Laboratory, Department of Radiology, Mayo Clinic, 200 1st Street, SW, Rochester, MN, 55905, USA
| | - Pouria Rouzrokh
- Artificial Intelligence Laboratory, Department of Radiology, Mayo Clinic, 200 1st Street, SW, Rochester, MN, 55905, USA
| | - Shahriar Faghani
- Artificial Intelligence Laboratory, Department of Radiology, Mayo Clinic, 200 1st Street, SW, Rochester, MN, 55905, USA
| | - Mana Moassefi
- Artificial Intelligence Laboratory, Department of Radiology, Mayo Clinic, 200 1st Street, SW, Rochester, MN, 55905, USA
| | - Aylin Tahmasebi
- Department of Radiology, Thomas Jefferson University, Philadelphia, PA, USA
| | - Katherine P Andriole
- Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Peter Chang
- Department of Radiological Sciences, Irvine Medical Center, University of California, Orange, CA, USA
| | - Keyvan Farahani
- Center for Biomedical Informatics and Information Technology, National Cancer Institute, Bethesda, MD, USA
| | | | - Les Folio
- Diagnostic Imaging & Interventional Radiology Moffitt Cancer Center, Tampa, FL, USA
| | - Sina Houshmand
- Department of Radiology and Biomedical Imaging, University of California San Francisco, San Francisco, CA, USA
| | - Maryellen L Giger
- Department of Radiology, The University of Chicago, Chicago, IL, USA
| | - Judy W Gichoya
- Department of Radiology, Emory University School of Medicine, Atlanta, GA, USA
| | - Bradley J Erickson
- Artificial Intelligence Laboratory, Department of Radiology, Mayo Clinic, 200 1st Street, SW, Rochester, MN, 55905, USA.
| |
Collapse
|
2
|
Kazerooni AF, Khalili N, Liu X, Haldar D, Jiang Z, Anwar SM, Albrecht J, Adewole M, Anazodo U, Anderson H, Bagheri S, Baid U, Bergquist T, Borja AJ, Calabrese E, Chung V, Conte GM, Dako F, Eddy J, Ezhov I, Familiar A, Farahani K, Haldar S, Iglesias JE, Janas A, Johansen E, Jones BV, Kofler F, LaBella D, Lai HA, Leemput KV, Li HB, Maleki N, McAllister AS, Meier Z, Menze B, Moawad AW, Nandolia KK, Pavaine J, Piraud M, Poussaint T, Prabhu SP, Reitman Z, Rodriguez A, Rudie JD, Sanchez-Montano M, Shaikh IS, Shah LM, Sheth N, Shinohara RT, Tu W, Viswanathan K, Wang C, Ware JB, Wiestler B, Wiggins W, Zapaishchykova A, Aboian M, Bornhorst M, de Blank P, Deutsch M, Fouladi M, Hoffman L, Kann B, Lazow M, Mikael L, Nabavizadeh A, Packer R, Resnick A, Rood B, Vossough A, Bakas S, Linguraru MG. The Brain Tumor Segmentation (BraTS) Challenge 2023: Focus on Pediatrics (CBTN-CONNECT-DIPGR-ASNR-MICCAI BraTS-PEDs). ArXiv 2024:arXiv:2305.17033v6. [PMID: 37292481 PMCID: PMC10246083] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Pediatric tumors of the central nervous system are the most common cause of cancer-related death in children. The five-year survival rate for high-grade gliomas in children is less than 20\%. Due to their rarity, the diagnosis of these entities is often delayed, their treatment is mainly based on historic treatment concepts, and clinical trials require multi-institutional collaborations. The MICCAI Brain Tumor Segmentation (BraTS) Challenge is a landmark community benchmark event with a successful history of 12 years of resource creation for the segmentation and analysis of adult glioma. Here we present the CBTN-CONNECT-DIPGR-ASNR-MICCAI BraTS-PEDs 2023 challenge, which represents the first BraTS challenge focused on pediatric brain tumors with data acquired across multiple international consortia dedicated to pediatric neuro-oncology and clinical trials. The BraTS-PEDs 2023 challenge focuses on benchmarking the development of volumentric segmentation algorithms for pediatric brain glioma through standardized quantitative performance evaluation metrics utilized across the BraTS 2023 cluster of challenges. Models gaining knowledge from the BraTS-PEDs multi-parametric structural MRI (mpMRI) training data will be evaluated on separate validation and unseen test mpMRI dataof high-grade pediatric glioma. The CBTN-CONNECT-DIPGR-ASNR-MICCAI BraTS-PEDs 2023 challenge brings together clinicians and AI/imaging scientists to lead to faster development of automated segmentation techniques that could benefit clinical trials, and ultimately the care of children with brain tumors.
Collapse
|
3
|
Reinke A, Tizabi MD, Baumgartner M, Eisenmann M, Heckmann-Nötzel D, Kavur AE, Rädsch T, Sudre CH, Acion L, Antonelli M, Arbel T, Bakas S, Benis A, Buettner F, Cardoso MJ, Cheplygina V, Chen J, Christodoulou E, Cimini BA, Farahani K, Ferrer L, Galdran A, van Ginneken B, Glocker B, Godau P, Hashimoto DA, Hoffman MM, Huisman M, Isensee F, Jannin P, Kahn CE, Kainmueller D, Kainz B, Karargyris A, Kleesiek J, Kofler F, Kooi T, Kopp-Schneider A, Kozubek M, Kreshuk A, Kurc T, Landman BA, Litjens G, Madani A, Maier-Hein K, Martel AL, Meijering E, Menze B, Moons KGM, Müller H, Nichyporuk B, Nickel F, Petersen J, Rafelski SM, Rajpoot N, Reyes M, Riegler MA, Rieke N, Saez-Rodriguez J, Sánchez CI, Shetty S, Summers RM, Taha AA, Tiulpin A, Tsaftaris SA, Van Calster B, Varoquaux G, Yaniv ZR, Jäger PF, Maier-Hein L. Understanding metric-related pitfalls in image analysis validation. Nat Methods 2024; 21:182-194. [PMID: 38347140 DOI: 10.1038/s41592-023-02150-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2023] [Accepted: 12/12/2023] [Indexed: 02/15/2024]
Abstract
Validation metrics are key for tracking scientific progress and bridging the current chasm between artificial intelligence research and its translation into practice. However, increasing evidence shows that, particularly in image analysis, metrics are often chosen inadequately. Although taking into account the individual strengths, weaknesses and limitations of validation metrics is a critical prerequisite to making educated choices, the relevant knowledge is currently scattered and poorly accessible to individual researchers. Based on a multistage Delphi process conducted by a multidisciplinary expert consortium as well as extensive community feedback, the present work provides a reliable and comprehensive common point of access to information on pitfalls related to validation metrics in image analysis. Although focused on biomedical image analysis, the addressed pitfalls generalize across application domains and are categorized according to a newly created, domain-agnostic taxonomy. The work serves to enhance global comprehension of a key topic in image analysis validation.
Collapse
Affiliation(s)
- Annika Reinke
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany.
- German Cancer Research Center (DKFZ) Heidelberg, HI Helmholtz Imaging, Heidelberg, Germany.
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany.
| | - Minu D Tizabi
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany.
- National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, Heidelberg, Germany.
| | - Michael Baumgartner
- German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Heidelberg, Germany
| | - Matthias Eisenmann
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany
| | - Doreen Heckmann-Nötzel
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany
- National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, Heidelberg, Germany
| | - A Emre Kavur
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany
- German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Heidelberg, Germany
- German Cancer Research Center (DKFZ) Heidelberg, HI Applied Computer Vision Lab, Heidelberg, Germany
| | - Tim Rädsch
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany
- German Cancer Research Center (DKFZ) Heidelberg, HI Helmholtz Imaging, Heidelberg, Germany
| | - Carole H Sudre
- MRC Unit for Lifelong Health and Ageing at UCL and Centre for Medical Image Computing, Department of Computer Science, University College London, London, UK
- School of Biomedical Engineering and Imaging Science, King's College London, London, UK
| | - Laura Acion
- Instituto de Cálculo, CONICET - Universidad de Buenos Aires, Buenos Aires, Argentina
| | - Michela Antonelli
- School of Biomedical Engineering and Imaging Science, King's College London, London, UK
- Centre for Medical Image Computing, University College London, London, UK
| | - Tal Arbel
- Centre for Intelligent Machines and MILA (Quebec Artificial Intelligence Institute), McGill University, Montréal, Quebec, Canada
| | - Spyridon Bakas
- Division of Computational Pathology, Dept of Pathology & Laboratory Medicine, Indiana University School of Medicine, Indianapolis, IN, USA
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
| | - Arriel Benis
- Department of Digital Medical Technologies, Holon Institute of Technology, Holon, Israel
- European Federation for Medical Informatics, Le Mont-sur-Lausanne, Switzerland
| | - Florian Buettner
- German Cancer Consortium (DKTK), partner site Frankfurt/Mainz, a partnership between DKFZ and UCT Frankfurt-Marburg, Frankfurt am Main, Germany
- German Cancer Research Center (DKFZ) Heidelberg, Heidelberg, Germany
- Goethe University Frankfurt, Department of Medicine, Frankfurt am Main, Germany
- Goethe University Frankfurt, Department of Informatics, Frankfurt am Main, Germany
- Frankfurt Cancer Insititute, Frankfurt am Main, Germany
| | - M Jorge Cardoso
- School of Biomedical Engineering and Imaging Science, King's College London, London, UK
| | - Veronika Cheplygina
- Department of Computer Science, IT University of Copenhagen, Copenhagen, Denmark
| | - Jianxu Chen
- Leibniz-Institut für Analytische Wissenschaften - ISAS - e.V., Dortmund, Germany
| | - Evangelia Christodoulou
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany
| | - Beth A Cimini
- Imaging Platform, Broad Institute of MIT and Harvard, Cambridge, MA, USA
| | - Keyvan Farahani
- Center for Biomedical Informatics and Information Technology, National Cancer Institute, Bethesda, MD, USA
| | - Luciana Ferrer
- Instituto de Investigación en Ciencias de la Computación (ICC), CONICET-UBA, Ciudad Autónoma de Buenos Aires, Buenos Aires, Argentina
| | - Adrian Galdran
- Universitat Pompeu Fabra, Barcelona, Spain
- University of Adelaide, Adelaide, South Australia, Australia
| | - Bram van Ginneken
- Fraunhofer MEVIS, Bremen, Germany
- Radboud Institute for Health Sciences, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Ben Glocker
- Department of Computing, Imperial College London, South Kensington Campus, London, UK
| | - Patrick Godau
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
- National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, Heidelberg, Germany
| | - Daniel A Hashimoto
- Department of Surgery, Perelman School of Medicine, Philadelphia, PA, USA
- General Robotics Automation Sensing and Perception Laboratory, School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, PA, USA
| | - Michael M Hoffman
- Princess Margaret Cancer Centre, University Health Network, Toronto, Ontario, Canada
- Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada
- Department of Computer Science, University of Toronto, Toronto, Ontario, Canada
- Vector Institute for Artificial Intelligence, Toronto, Ontario, Canada
| | - Merel Huisman
- Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Fabian Isensee
- German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Heidelberg, Germany
- German Cancer Research Center (DKFZ) Heidelberg, HI Applied Computer Vision Lab, Heidelberg, Germany
| | - Pierre Jannin
- Laboratoire Traitement du Signal et de l'Image - UMR_S 1099, Université de Rennes 1, Rennes, France
- INSERM, Paris, France
| | - Charles E Kahn
- Department of Radiology and Institute for Biomedical Informatics, University of Pennsylvania, Philadelphia, PA, USA
| | - Dagmar Kainmueller
- Max-Delbrück Center for Molecular Medicine in the Helmholtz Association (MDC), Biomedical Image Analysis and HI Helmholtz Imaging, Berlin, Germany
- University of Potsdam, Digital Engineering Faculty, Potsdam, Germany
| | - Bernhard Kainz
- Department of Computing, Faculty of Engineering, Imperial College London, London, UK
- Department AIBE, Friedrich-Alexander-Universität (FAU), Erlangen-Nürnberg, Germany
| | | | - Jens Kleesiek
- Translational Image-guided Oncology (TIO), Institute for AI in Medicine (IKIM), University Medicine Essen, Essen, Germany
| | | | | | - Annette Kopp-Schneider
- German Cancer Research Center (DKFZ) Heidelberg, Division of Biostatistics, Heidelberg, Germany
| | - Michal Kozubek
- Centre for Biomedical Image Analysis and Faculty of Informatics, Masaryk University, Brno, Czech Republic
| | - Anna Kreshuk
- Cell Biology and Biophysics Unit, European Molecular Biology Laboratory (EMBL), Heidelberg, Germany
| | - Tahsin Kurc
- Department of Biomedical Informatics, Stony Brook University, Health Science Center, Stony Brook, NY, USA
| | | | - Geert Litjens
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Amin Madani
- Department of Surgery, University Health Network, Philadelphia, PA, USA
| | - Klaus Maier-Hein
- German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Heidelberg, Germany
- Pattern Analysis and Learning Group, Department of Radiation Oncology, Heidelberg University Hospital, Heidelberg, Germany
| | - Anne L Martel
- Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada
- Physical Sciences, Sunnybrook Research Institute, Toronto, Ontario, Canada
| | - Erik Meijering
- School of Computer Science and Engineering, University of New South Wales, UNSW Sydney, Kensington, New South Wales, Australia
| | - Bjoern Menze
- Department of Quantitative Biomedicine, University of Zurich, Zurich, Switzerland
| | - Karel G M Moons
- Julius Center for Health Sciences and Primary Care, UMC Utrecht, Utrecht University, Utrecht, the Netherlands
| | - Henning Müller
- Information Systems Institute, University of Applied Sciences Western Switzerland (HES-SO), Sierre, Switzerland
- Medical Faculty, University of Geneva, Geneva, Switzerland
| | - Brennan Nichyporuk
- MILA (Quebec Artificial Intelligence Institute), Montréal, Quebec, Canada
| | - Felix Nickel
- Department of General, Visceral and Thoracic Surgery, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Jens Petersen
- German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Heidelberg, Germany
| | | | - Nasir Rajpoot
- Tissue Image Analytics Laboratory, Department of Computer Science, University of Warwick, Coventry, UK
| | - Mauricio Reyes
- ARTORG Center for Biomedical Engineering Research, University of Bern, Bern, Switzerland
- Department of Radiation Oncology, University Hospital Bern, University of Bern, Bern, Switzerland
| | - Michael A Riegler
- Simula Metropolitan Center for Digital Engineering, Oslo, Norway
- UiT The Arctic University of Norway, Tromsø, Norway
| | | | - Julio Saez-Rodriguez
- Institute for Computational Biomedicine, Heidelberg University, Heidelberg, Germany
- Faculty of Medicine, Heidelberg University Hospital, Heidelberg, Germany
| | - Clara I Sánchez
- Informatics Institute, Faculty of Science, University of Amsterdam, Amsterdam, the Netherlands
| | | | - Ronald M Summers
- National Institutes of Health Clinical Center, Bethesda, MD, USA
| | - Abdel A Taha
- Institute of Information Systems Engineering, TU Wien, Vienna, Austria
| | - Aleksei Tiulpin
- Research Unit of Health Sciences and Technology, Faculty of Medicine, University of Oulu, Oulu, Finland
- Neurocenter Oulu, Oulu University Hospital, Oulu, Finland
| | | | - Ben Van Calster
- Department of Development and Regeneration and EPI-centre, KU Leuven, Leuven, Belgium
- Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands
| | - Gaël Varoquaux
- Parietal project team, INRIA Saclay-Île de France, Palaiseau, France
| | - Ziv R Yaniv
- National Institute of Allergy and Infectious Diseases, Bethesda, MD, USA
| | - Paul F Jäger
- German Cancer Research Center (DKFZ) Heidelberg, HI Helmholtz Imaging, Heidelberg, Germany.
- German Cancer Research Center (DKFZ) Heidelberg, Interactive Machine Learning Group, Heidelberg, Germany.
| | - Lena Maier-Hein
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany.
- German Cancer Research Center (DKFZ) Heidelberg, HI Helmholtz Imaging, Heidelberg, Germany.
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany.
- National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, Heidelberg, Germany.
- Faculty of Medicine, Heidelberg University Hospital, Heidelberg, Germany.
| |
Collapse
|
4
|
Maier-Hein L, Reinke A, Godau P, Tizabi MD, Buettner F, Christodoulou E, Glocker B, Isensee F, Kleesiek J, Kozubek M, Reyes M, Riegler MA, Wiesenfarth M, Kavur AE, Sudre CH, Baumgartner M, Eisenmann M, Heckmann-Nötzel D, Rädsch T, Acion L, Antonelli M, Arbel T, Bakas S, Benis A, Blaschko MB, Cardoso MJ, Cheplygina V, Cimini BA, Collins GS, Farahani K, Ferrer L, Galdran A, van Ginneken B, Haase R, Hashimoto DA, Hoffman MM, Huisman M, Jannin P, Kahn CE, Kainmueller D, Kainz B, Karargyris A, Karthikesalingam A, Kofler F, Kopp-Schneider A, Kreshuk A, Kurc T, Landman BA, Litjens G, Madani A, Maier-Hein K, Martel AL, Mattson P, Meijering E, Menze B, Moons KGM, Müller H, Nichyporuk B, Nickel F, Petersen J, Rajpoot N, Rieke N, Saez-Rodriguez J, Sánchez CI, Shetty S, van Smeden M, Summers RM, Taha AA, Tiulpin A, Tsaftaris SA, Van Calster B, Varoquaux G, Jäger PF. Metrics reloaded: recommendations for image analysis validation. Nat Methods 2024; 21:195-212. [PMID: 38347141 DOI: 10.1038/s41592-023-02151-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2023] [Accepted: 12/12/2023] [Indexed: 02/15/2024]
Abstract
Increasing evidence shows that flaws in machine learning (ML) algorithm validation are an underestimated global problem. In biomedical image analysis, chosen performance metrics often do not reflect the domain interest, and thus fail to adequately measure scientific progress and hinder translation of ML techniques into practice. To overcome this, we created Metrics Reloaded, a comprehensive framework guiding researchers in the problem-aware selection of metrics. Developed by a large international consortium in a multistage Delphi process, it is based on the novel concept of a problem fingerprint-a structured representation of the given problem that captures all aspects that are relevant for metric selection, from the domain interest to the properties of the target structure(s), dataset and algorithm output. On the basis of the problem fingerprint, users are guided through the process of choosing and applying appropriate validation metrics while being made aware of potential pitfalls. Metrics Reloaded targets image analysis problems that can be interpreted as classification tasks at image, object or pixel level, namely image-level classification, object detection, semantic segmentation and instance segmentation tasks. To improve the user experience, we implemented the framework in the Metrics Reloaded online tool. Following the convergence of ML methodology across application domains, Metrics Reloaded fosters the convergence of validation methodology. Its applicability is demonstrated for various biomedical use cases.
Collapse
Affiliation(s)
- Lena Maier-Hein
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany.
- German Cancer Research Center (DKFZ) Heidelberg, HI Helmholtz Imaging, Heidelberg, Germany.
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany.
- Medical Faculty, Heidelberg University, Heidelberg, Germany.
- National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, Heidelberg, Germany.
| | - Annika Reinke
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany.
- German Cancer Research Center (DKFZ) Heidelberg, HI Helmholtz Imaging, Heidelberg, Germany.
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany.
| | - Patrick Godau
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany
- Faculty of Mathematics and Computer Science, Heidelberg University, Heidelberg, Germany
- National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, Heidelberg, Germany
| | - Minu D Tizabi
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany
- National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, Heidelberg, Germany
| | - Florian Buettner
- German Cancer Consortium (DKTK), partner site Frankfurt/Mainz, a partnership between DKFZ and UCT Frankfurt-Marburg, Frankfurt am Main, Germany
- German Cancer Research Center (DKFZ) Heidelberg, Heidelberg, Germany
- Department of Medicine, Goethe University Frankfurt, Frankfurt am Main, Germany
- Department of Informatics, Goethe University Frankfurt, Frankfurt am Main, Germany
- Frankfurt Cancer Insititute, Frankfurt am Main, Germany
| | - Evangelia Christodoulou
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany
| | - Ben Glocker
- Department of Computing, Imperial College London, South Kensington Campus, London, UK
| | - Fabian Isensee
- German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Heidelberg, Germany
- German Cancer Research Center (DKFZ) Heidelberg, HI Applied Computer Vision Lab, Heidelberg, Germany
| | - Jens Kleesiek
- Institute for AI in Medicine, University Medicine Essen, Essen, Germany
| | - Michal Kozubek
- Centre for Biomedical Image Analysis and Faculty of Informatics, Masaryk University, Brno, Czech Republic
| | - Mauricio Reyes
- ARTORG Center for Biomedical Engineering Research, University of Bern, Bern, Switzerland
- Department of Radiation Oncology, University Hospital Bern, University of Bern, Bern, Switzerland
| | - Michael A Riegler
- Simula Metropolitan Center for Digital Engineering, Oslo, Norway
- Department of Computer Science, UiT The Arctic University of Norway, Tromsø, Norway
| | - Manuel Wiesenfarth
- German Cancer Research Center (DKFZ) Heidelberg, Division of Biostatistics, Heidelberg, Germany
| | - A Emre Kavur
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany
- German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Heidelberg, Germany
- German Cancer Research Center (DKFZ) Heidelberg, HI Applied Computer Vision Lab, Heidelberg, Germany
| | - Carole H Sudre
- MRC Unit for Lifelong Health and Ageing at UCL and Centre for Medical Image Computing, Department of Computer Science, University College London, London, UK
- School of Biomedical Engineering and Imaging Science, King's College London, London, UK
| | - Michael Baumgartner
- German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Heidelberg, Germany
| | - Matthias Eisenmann
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany
| | - Doreen Heckmann-Nötzel
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany
- National Center for Tumor Diseases (NCT), NCT Heidelberg, a partnership between DKFZ and University Medical Center Heidelberg, Heidelberg, Germany
| | - Tim Rädsch
- German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Heidelberg, Germany
- German Cancer Research Center (DKFZ) Heidelberg, HI Helmholtz Imaging, Heidelberg, Germany
| | - Laura Acion
- Instituto de Cálculo, CONICET - Universidad de Buenos Aires, Buenos Aires, Argentina
| | - Michela Antonelli
- School of Biomedical Engineering and Imaging Science, King's College London, London, UK
- Centre for Medical Image Computing, University College London, London, UK
| | - Tal Arbel
- Centre for Intelligent Machines and MILA (Québec Artificial Intelligence Institute), McGill University, Montréal, Quebec, Canada
| | - Spyridon Bakas
- Division of Computational Pathology, Department of Pathology & Laboratory Medicine, Indiana University School of Medicine, IU Health Information and Translational Sciences Building, Indianapolis, IN, USA
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
| | - Arriel Benis
- Department of Digital Medical Technologies, Holon Institute of Technology, Holon, Israel
- European Federation for Medical Informatics, Le Mont-sur-Lausanne, Switzerland
| | - Matthew B Blaschko
- Center for Processing Speech and Images, Department of Electrical Engineering, KU Leuven, Leuven, Belgium
| | - M Jorge Cardoso
- School of Biomedical Engineering and Imaging Science, King's College London, London, UK
| | - Veronika Cheplygina
- Department of Computer Science, IT University of Copenhagen, Copenhagen, Denmark
| | - Beth A Cimini
- Imaging Platform, Broad Institute of MIT and Harvard, Cambridge, MA, USA
| | - Gary S Collins
- Centre for Statistics in Medicine, University of Oxford, Nuffield Orthopaedic Centre, Oxford, UK
| | - Keyvan Farahani
- Center for Biomedical Informatics and Information Technology, National Cancer Institute, Bethesda, MD, USA
| | - Luciana Ferrer
- Instituto de Investigación en Ciencias de la Computación (ICC), CONICET-UBA, Ciudad Autónoma de Buenos Aires, Buenos Aires, Argentina
| | - Adrian Galdran
- BCN Medtech, Universitat Pompeu Fabra, Barcelona, Spain
- Australian Institute for Machine Learning AIML, University of Adelaide, Adelaide, South Australia, Australia
| | - Bram van Ginneken
- Fraunhofer MEVIS, Bremen, Germany
- Radboud Institute for Health Sciences, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Robert Haase
- Technische Universität (TU) Dresden, DFG Cluster of Excellence 'Physics of Life', Dresden, Germany
- Center for Systems Biology, Dresden, Germany
- Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI), Leipzig University, Leipzig, Germany
| | - Daniel A Hashimoto
- Department of Surgery, Perelman School of Medicine, Philadelphia, PA, USA
- General Robotics Automation Sensing and Perception Laboratory, School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, PA, USA
| | - Michael M Hoffman
- Princess Margaret Cancer Centre, University Health Network, Toronto, Ontario, Canada
- Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada
- Vector Institute for Artificial Intelligence, Toronto, Ontario, Canada
- Department of Computer Science, University of Toronto, Toronto, Ontario, Canada
| | - Merel Huisman
- Department of Radiology and Nuclear Medicine, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Pierre Jannin
- Laboratoire Traitement du Signal et de l'Image - UMR_S 1099, Université de Rennes 1, Rennes, France
- INSERM, Paris, France
| | - Charles E Kahn
- Department of Radiology and Institute for Biomedical Informatics, University of Pennsylvania, Philadelphia, PA, USA
| | - Dagmar Kainmueller
- Max-Delbrück Center for Molecular Medicine in the Helmholtz Association (MDC), Biomedical Image Analysis and HI Helmholtz Imaging, Berlin, Germany
- Digital Engineering Faculty, University of Potsdam, Potsdam, Germany
| | - Bernhard Kainz
- Department of Computing, Faculty of Engineering, Imperial College London, London, UK
- Department AIBE, Friedrich-Alexander-Universität (FAU), Erlangen-Nürnberg, Germany
| | | | | | | | - Annette Kopp-Schneider
- German Cancer Research Center (DKFZ) Heidelberg, Division of Biostatistics, Heidelberg, Germany
| | - Anna Kreshuk
- Cell Biology and Biophysics Unit, European Molecular Biology Laboratory (EMBL), Heidelberg, Germany
| | - Tahsin Kurc
- Department of Biomedical Informatics, Stony Brook University, Health Science Center, Stony Brook, NY, USA
| | | | - Geert Litjens
- Department of Pathology, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Amin Madani
- Department of Surgery, University Health Network, Philadelphia, PA, USA
| | - Klaus Maier-Hein
- German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Heidelberg, Germany
- Pattern Analysis and Learning Group, Department of Radiation Oncology, Heidelberg University Hospital, Heidelberg, Germany
| | - Anne L Martel
- Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada
- Department of Computer Science, University of Toronto, Toronto, Ontario, Canada
- Physical Sciences, Sunnybrook Research Institute, Toronto, Ontario, Canada
| | - Peter Mattson
- Google, 1600 Amphitheatre Pkwy, Mountain View, CA, USA
| | - Erik Meijering
- School of Computer Science and Engineering, University of New South Wales, UNSW Sydney, Kensington, New South Wales, Australia
| | - Bjoern Menze
- Department of Quantitative Biomedicine, University of Zurich, Zurich, Switzerland
| | - Karel G M Moons
- Julius Center for Health Sciences and Primary Care, UMC Utrecht, Utrecht University, Utrecht, the Netherlands
| | - Henning Müller
- Information Systems Institute, University of Applied Sciences Western Switzerland (HES-SO), Sierre, Switzerland
- Medical Faculty, University of Geneva, Geneva, Switzerland
| | - Brennan Nichyporuk
- MILA (Québec Artificial Intelligence Institute), Montréal, Quebec, Canada
| | - Felix Nickel
- Department of General, Visceral and Thoracic Surgery, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Jens Petersen
- German Cancer Research Center (DKFZ) Heidelberg, Division of Medical Image Computing, Heidelberg, Germany
| | - Nasir Rajpoot
- Tissue Image Analytics Laboratory, Department of Computer Science, University of Warwick, Coventry, UK
| | | | - Julio Saez-Rodriguez
- Institute for Computational Biomedicine, Heidelberg University, Heidelberg, Germany
- Faculty of Medicine, Heidelberg University Hospital, Heidelberg, Germany
| | - Clara I Sánchez
- Informatics Institute, Faculty of Science, University of Amsterdam, Amsterdam, the Netherlands
| | | | - Maarten van Smeden
- Julius Center for Health Sciences and Primary Care, UMC Utrecht, Utrecht University, Utrecht, the Netherlands
| | - Ronald M Summers
- National Institutes of Health Clinical Center, Bethesda, MD, USA
| | - Abdel A Taha
- Institute of Information Systems Engineering, TU Wien, Vienna, Austria
| | - Aleksei Tiulpin
- Research Unit of Health Sciences and Technology, Faculty of Medicine, University of Oulu, Oulu, Finland
- Neurocenter Oulu, Oulu University Hospital, Oulu, Finland
| | | | - Ben Van Calster
- Department of Development and Regeneration and EPI-centre, KU Leuven, Leuven, Belgium
- Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands
| | - Gaël Varoquaux
- Parietal project team, INRIA Saclay-Île de France, Palaiseau, France
| | - Paul F Jäger
- German Cancer Research Center (DKFZ) Heidelberg, HI Helmholtz Imaging, Heidelberg, Germany.
- German Cancer Research Center (DKFZ) Heidelberg, Interactive Machine Learning Group, Heidelberg, Germany.
| |
Collapse
|
5
|
Fedorov A, Longabaugh WJR, Pot D, Clunie DA, Pieper SD, Gibbs DL, Bridge C, Herrmann MD, Homeyer A, Lewis R, Aerts HJWL, Krishnaswamy D, Thiriveedhi VK, Ciausu C, Schacherer DP, Bontempi D, Pihl T, Wagner U, Farahani K, Kim E, Kikinis R. National Cancer Institute Imaging Data Commons: Toward Transparency, Reproducibility, and Scalability in Imaging Artificial Intelligence. Radiographics 2023; 43:e230180. [PMID: 37999984 DOI: 10.1148/rg.230180] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2023]
Abstract
The remarkable advances of artificial intelligence (AI) technology are revolutionizing established approaches to the acquisition, interpretation, and analysis of biomedical imaging data. Development, validation, and continuous refinement of AI tools requires easy access to large high-quality annotated datasets, which are both representative and diverse. The National Cancer Institute (NCI) Imaging Data Commons (IDC) hosts large and diverse publicly available cancer image data collections. By harmonizing all data based on industry standards and colocalizing it with analysis and exploration resources, the IDC aims to facilitate the development, validation, and clinical translation of AI tools and address the well-documented challenges of establishing reproducible and transparent AI processing pipelines. Balanced use of established commercial products with open-source solutions, interconnected by standard interfaces, provides value and performance, while preserving sufficient agility to address the evolving needs of the research community. Emphasis on the development of tools, use cases to demonstrate the utility of uniform data representation, and cloud-based analysis aim to ease adoption and help define best practices. Integration with other data in the broader NCI Cancer Research Data Commons infrastructure opens opportunities for multiomics studies incorporating imaging data to further empower the research community to accelerate breakthroughs in cancer detection, diagnosis, and treatment. Published under a CC BY 4.0 license.
Collapse
Affiliation(s)
- Andrey Fedorov
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - William J R Longabaugh
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - David Pot
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - David A Clunie
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Steven D Pieper
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - David L Gibbs
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Christopher Bridge
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Markus D Herrmann
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - André Homeyer
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Rob Lewis
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Hugo J W L Aerts
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Deepa Krishnaswamy
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Vamsi Krishna Thiriveedhi
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Cosmin Ciausu
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Daniela P Schacherer
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Dennis Bontempi
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Todd Pihl
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Ulrike Wagner
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Keyvan Farahani
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Erika Kim
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Ron Kikinis
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| |
Collapse
|
6
|
Mayo CS, Feng MU, Brock KK, Kudner R, Balter P, Buchsbaum JC, Caissie A, Covington E, Daugherty EC, Dekker AL, Fuller CD, Hallstrom AL, Hong DS, Hong JC, Kamran SC, Katsoulakis E, Kildea J, Krauze AV, Kruse JJ, McNutt T, Mierzwa M, Moreno A, Palta JR, Popple R, Purdie TG, Richardson S, Sharp GC, Satomi S, Tarbox LR, Venkatesan AM, Witztum A, Woods KE, Yao Y, Farahani K, Aneja S, Gabriel PE, Hadjiiski L, Ruan D, Siewerdsen JH, Bratt S, Casagni M, Chen S, Christodouleas JC, DiDonato A, Hayman J, Kapoor R, Kravitz S, Sebastian S, Von Siebenthal M, Bosch W, Hurkmans C, Yom SS, Xiao Y. Operational Ontology for Oncology (O3): A Professional Society-Based, Multistakeholder, Consensus-Driven Informatics Standard Supporting Clinical and Research Use of Real-World Data From Patients Treated for Cancer. Int J Radiat Oncol Biol Phys 2023; 117:533-550. [PMID: 37244628 PMCID: PMC10741247 DOI: 10.1016/j.ijrobp.2023.05.033] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2022] [Revised: 05/17/2023] [Accepted: 05/19/2023] [Indexed: 05/29/2023]
Abstract
PURPOSE The ongoing lack of data standardization severely undermines the potential for automated learning from the vast amount of information routinely archived in electronic health records (EHRs), radiation oncology information systems, treatment planning systems, and other cancer care and outcomes databases. We sought to create a standardized ontology for clinical data, social determinants of health, and other radiation oncology concepts and interrelationships. METHODS AND MATERIALS The American Association of Physicists in Medicine's Big Data Science Committee was initiated in July 2019 to explore common ground from the stakeholders' collective experience of issues that typically compromise the formation of large inter- and intra-institutional databases from EHRs. The Big Data Science Committee adopted an iterative, cyclical approach to engaging stakeholders beyond its membership to optimize the integration of diverse perspectives from the community. RESULTS We developed the Operational Ontology for Oncology (O3), which identified 42 key elements, 359 attributes, 144 value sets, and 155 relationships ranked in relative importance of clinical significance, likelihood of availability in EHRs, and the ability to modify routine clinical processes to permit aggregation. Recommendations are provided for best use and development of the O3 to 4 constituencies: device manufacturers, centers of clinical care, researchers, and professional societies. CONCLUSIONS O3 is designed to extend and interoperate with existing global infrastructure and data science standards. The implementation of these recommendations will lower the barriers for aggregation of information that could be used to create large, representative, findable, accessible, interoperable, and reusable data sets to support the scientific objectives of grant programs. The construction of comprehensive "real-world" data sets and application of advanced analytical techniques, including artificial intelligence, holds the potential to revolutionize patient management and improve outcomes by leveraging increased access to information derived from larger, more representative data sets.
Collapse
Affiliation(s)
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Dan Ruan
- University of California, Los Angeles
| | | | | | | | | | | | | | | | | | | | | | | | | | | | - Sue S Yom
- University of California, San Francisco
| | | |
Collapse
|
7
|
Familiar AM, Kazerooni AF, Anderson H, Lubneuski A, Viswanathan K, Breslow R, Khalili N, Bagheri S, Haldar D, Kim MC, Arif S, Madhogarhia R, Nguyen TQ, Frenkel EA, Helili Z, Harrison J, Farahani K, Linguraru MG, Bagci U, Velichko Y, Stevens J, Leary S, Lober RM, Campion S, Smith AA, Morinigo D, Rood B, Diamond K, Pollack IF, Williams M, Vossough A, Ware JB, Mueller S, Storm PB, Heath AP, Waanders AJ, Lilly J, Mason JL, Resnick AC, Nabavizadeh A. A multi-institutional pediatric dataset of clinical radiology MRIs by the Children's Brain Tumor Network. ArXiv 2023:arXiv:2310.01413v1. [PMID: 38106459 PMCID: PMC10723526] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Subscribe] [Scholar Register] [Indexed: 12/19/2023]
Abstract
Pediatric brain and spinal cancers remain the leading cause of cancer-related death in children. Advancements in clinical decision-support in pediatric neuro-oncology utilizing the wealth of radiology imaging data collected through standard care, however, has significantly lagged other domains. Such data is ripe for use with predictive analytics such as artificial intelligence (AI) methods, which require large datasets. To address this unmet need, we provide a multi-institutional, large-scale pediatric dataset of 23,101 multi-parametric MRI exams acquired through routine care for 1,526 brain tumor patients, as part of the Children's Brain Tumor Network. This includes longitudinal MRIs across various cancer diagnoses, with associated patient-level clinical information, digital pathology slides, as well as tissue genotype and omics data. To facilitate downstream analysis, treatment-naïve images for 370 subjects were processed and released through the NCI Childhood Cancer Data Initiative via the Cancer Data Service. Through ongoing efforts to continuously build these imaging repositories, our aim is to accelerate discovery and translational AI models with real-world data, to ultimately empower precision medicine for children.
Collapse
Affiliation(s)
- Ariana M. Familiar
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
| | - Anahita Fathi Kazerooni
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Hannah Anderson
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Aliaksandr Lubneuski
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
| | - Karthik Viswanathan
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
| | - Rocky Breslow
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
| | - Nastaran Khalili
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
| | - Sina Bagheri
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Debanjan Haldar
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
| | - Meen Chul Kim
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
| | - Sherjeel Arif
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
| | - Rachel Madhogarhia
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
| | - Thinh Q. Nguyen
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
| | - Elizabeth A. Frenkel
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
| | - Zeinab Helili
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
| | - Jessica Harrison
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
| | | | - Marius George Linguraru
- Sheikh Zayed Institute for Pediatric Surgical Innovation, Children’s National Hospital, Washington, DC, USA
- Departments of Radiology and Pediatrics, George Washington University School of Medicine and Health Sciences, Washington, DC, USA
| | - Ulas Bagci
- Department of Radiology, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| | - Yury Velichko
- Department of Radiology, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| | - Jeffrey Stevens
- Department of Hematology and Oncology, Seattle Children’s, Seattle, WA, USA
| | - Sarah Leary
- Department of Hematology and Oncology, Seattle Children’s, Seattle, WA, USA
| | - Robert M. Lober
- Division of Neurosurgery, Dayton Children’s Hospital, Dayton, OH, USA
| | - Stephani Campion
- Department of Pediatric Hematology & Oncology, Orlando Health Arnold Palmer Hospital for Children, Orlando, FL, USA
| | - Amy A. Smith
- Department of Pediatric Hematology & Oncology, Orlando Health Arnold Palmer Hospital for Children, Orlando, FL, USA
| | - Denise Morinigo
- Department of Hematology-Oncology, Children’s National Hospital, Washington, DC, USA
| | - Brian Rood
- Department of Hematology-Oncology, Children’s National Hospital, Washington, DC, USA
| | - Kimberly Diamond
- Department of Pediatric Neurosurgery, UPMC Children’s Hospital of Pittsburgh, Pittsburgh, PA, USA
| | - Ian F. Pollack
- Department of Pediatric Neurosurgery, UPMC Children’s Hospital of Pittsburgh, Pittsburgh, PA, USA
| | - Melissa Williams
- Division of Hematology, Oncology, NeuroOncology, and Transplant, Ann & Robert H Lurie Children’s Hospital of Chicago, Chicago, IL, USA
| | - Arastoo Vossough
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
- Department of Radiology, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
| | - Jeffrey B. Ware
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Sabine Mueller
- Department of Neurology, Division of Child Neurology, University of San Francisco, San Francisco, CA, USA
| | - Phillip B. Storm
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
| | - Allison P. Heath
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
| | - Angela J. Waanders
- Division of Hematology, Oncology, NeuroOncology, and Transplant, Ann & Robert H Lurie Children’s Hospital of Chicago, Chicago, IL, USA
- Department of Pediatrics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Jena Lilly
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
| | - Jennifer L. Mason
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
| | - Adam C. Resnick
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Neurosurgery, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
| | - Ali Nabavizadeh
- Center for Data-Driven Discovery in Biomedicine, Children’s Hospital of Philadelphia, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| |
Collapse
|
8
|
Moassefi M, Rouzrokh P, Conte GM, Vahdati S, Fu T, Tahmasebi A, Younis M, Farahani K, Gentili A, Kline T, Kitamura FC, Huo Y, Kuanar S, Younis K, Erickson BJ, Faghani S. Reproducibility of Deep Learning Algorithms Developed for Medical Imaging Analysis: A Systematic Review. J Digit Imaging 2023; 36:2306-2312. [PMID: 37407841 PMCID: PMC10501962 DOI: 10.1007/s10278-023-00870-5] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2022] [Revised: 06/08/2023] [Accepted: 06/09/2023] [Indexed: 07/07/2023] Open
Abstract
Since 2000, there have been more than 8000 publications on radiology artificial intelligence (AI). AI breakthroughs allow complex tasks to be automated and even performed beyond human capabilities. However, the lack of details on the methods and algorithm code undercuts its scientific value. Many science subfields have recently faced a reproducibility crisis, eroding trust in processes and results, and influencing the rise in retractions of scientific papers. For the same reasons, conducting research in deep learning (DL) also requires reproducibility. Although several valuable manuscript checklists for AI in medical imaging exist, they are not focused specifically on reproducibility. In this study, we conducted a systematic review of recently published papers in the field of DL to evaluate if the description of their methodology could allow the reproducibility of their findings. We focused on the Journal of Digital Imaging (JDI), a specialized journal that publishes papers on AI and medical imaging. We used the keyword "Deep Learning" and collected the articles published between January 2020 and January 2022. We screened all the articles and included the ones which reported the development of a DL tool in medical imaging. We extracted the reported details about the dataset, data handling steps, data splitting, model details, and performance metrics of each included article. We found 148 articles. Eighty were included after screening for articles that reported developing a DL model for medical image analysis. Five studies have made their code publicly available, and 35 studies have utilized publicly available datasets. We provided figures to show the ratio and absolute count of reported items from included studies. According to our cross-sectional study, in JDI publications on DL in medical imaging, authors infrequently report the key elements of their study to make it reproducible.
Collapse
Affiliation(s)
- Mana Moassefi
- Artificial Intelligence Lab, Department of Radiology, Mayo Clinic, Rochester, MN, USA.
| | - Pouria Rouzrokh
- Artificial Intelligence Lab, Department of Radiology, Mayo Clinic, Rochester, MN, USA
- Orthopedic Surgery Artificial Intelligence Laboratory (OSAIL), Department of Orthopedic Surgery, Mayo Clinic, Rochester, MN, USA
| | - Gian Marco Conte
- Artificial Intelligence Lab, Department of Radiology, Mayo Clinic, Rochester, MN, USA
| | - Sanaz Vahdati
- Artificial Intelligence Lab, Department of Radiology, Mayo Clinic, Rochester, MN, USA
| | - Tianyuan Fu
- Department of Radiology, University Hospitals Cleveland, Cleveland, OH, USA
| | - Aylin Tahmasebi
- Department of Radiology, Thomas Jefferson University, Philadelphia, PA, USA
| | - Mira Younis
- Cleveland Clinic Children's, Cleveland, OH, USA
| | - Keyvan Farahani
- National Cancer Institute, National Institutes of Health, Bethesda, MA, USA
| | - Amilcare Gentili
- Department of Radiology, University of California, San Diego, CA, USA
| | - Timothy Kline
- Department of Radiology, Mayo Clinic, Rochester, MN, USA
| | | | - Yuankai Huo
- Department of Electrical Engineering & Computer Science, Vanderbilt University, Nashville, TN, USA
| | - Shiba Kuanar
- Artificial Intelligence Lab, Department of Radiology, Mayo Clinic, Rochester, MN, USA
| | | | - Bradley J Erickson
- Artificial Intelligence Lab, Department of Radiology, Mayo Clinic, Rochester, MN, USA
| | - Shahriar Faghani
- Artificial Intelligence Lab, Department of Radiology, Mayo Clinic, Rochester, MN, USA.
| |
Collapse
|
9
|
Supanich M, Siewerdsen J, Fahrig R, Farahani K, Gang GJ, Helm P, Jans J, Jones K, Koenig T, Kuhls-Gilcrist A, Lin M, Riddell C, Ritschl L, Schafer S, Schueler B, Silver M, Timmer J, Trousset Y, Zhang J. AAPM Task Group Report 238: 3D C-arms with volumetric imaging capability. Med Phys 2023; 50:e904-e945. [PMID: 36710257 DOI: 10.1002/mp.16245] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2022] [Revised: 12/21/2022] [Accepted: 01/09/2023] [Indexed: 01/31/2023] Open
Abstract
This report reviews the image acquisition and reconstruction characteristics of C-arm Cone Beam Computed Tomography (C-arm CBCT) systems and provides guidance on quality control of C-arm systems with this volumetric imaging capability. The concepts of 3D image reconstruction, geometric calibration, image quality, and dosimetry covered in this report are also pertinent to CBCT for Image-Guided Radiation Therapy (IGRT). However, IGRT systems introduce a number of additional considerations, such as geometric alignment of the imaging at treatment isocenter, which are beyond the scope of the charge to the task group and the report. Section 1 provides an introduction to C-arm CBCT systems and reviews a variety of clinical applications. Section 2 briefly presents nomenclature specific or unique to these systems. A short review of C-arm fluoroscopy quality control (QC) in relation to 3D C-arm imaging is given in Section 3. Section 4 discusses system calibration, including geometric calibration and uniformity calibration. A review of the unique approaches and challenges to 3D reconstruction of data sets acquired by C-arm CBCT systems is give in Section 5. Sections 6 and 7 go in greater depth to address the performance assessment of C-arm CBCT units. First, Section 6 describes testing approaches and phantoms that may be used to evaluate image quality (spatial resolution and image noise and artifacts) and identifies several factors that affect image quality. Section 7 describes both free-in-air and in-phantom approaches to evaluating radiation dose indices. The methodologies described for assessing image quality and radiation dose may be used for annual constancy assessment and comparisons among different systems to help medical physicists determine when a system is not operating as expected. Baseline measurements taken either at installation or after a full preventative maintenance service call can also provide valuable data to help determine whether the performance of the system is acceptable. Collecting image quality and radiation dose data on existing phantoms used for CT image quality and radiation dose assessment, or on newly developed phantoms, will inform the development of performance criteria and standards. Phantom images are also useful for identifying and evaluating artifacts. In particular, comparing baseline data with those from current phantom images can reveal the need for system calibration before image artifacts are detected in clinical practice. Examples of artifacts are provided in Sections 4, 5, and 6.
Collapse
Affiliation(s)
- Mark Supanich
- Rush University Medical Center, Chicago, Illinois, USA
| | | | | | | | | | - Pat Helm
- Medtronic Inc., Minneapolis, Minnesota, USA
| | | | - Kyle Jones
- University of Texas MD Anderson Cancer Center, Houston, Texas, USA
| | | | | | - MingDe Lin
- Yale University, New Haven, Connecticut, USA
| | | | | | | | | | - Mike Silver
- Canon Medical Systems USA, Long Beach, California, USA
| | | | | | - Jie Zhang
- University of Kentucky, Lexington, Kentucky
| |
Collapse
|
10
|
Li HB, Conte GM, Anwar SM, Kofler F, Ezhov I, van Leemput K, Piraud M, Diaz M, Cole B, Calabrese E, Rudie J, Meissen F, Adewole M, Janas A, Kazerooni AF, LaBella D, Moawad AW, Farahani K, Eddy J, Bergquist T, Chung V, Shinohara RT, Dako F, Wiggins W, Reitman Z, Wang C, Liu X, Jiang Z, Familiar A, Johanson E, Meier Z, Davatzikos C, Freymann J, Kirby J, Bilello M, Fathallah-Shaykh HM, Wiest R, Kirschke J, Colen RR, Kotrotsou A, Lamontagne P, Marcus D, Milchenko M, Nazeri A, Weber MA, Mahajan A, Mohan S, Mongan J, Hess C, Cha S, Villanueva-Meyer J, Colak E, Crivellaro P, Jakab A, Albrecht J, Anazodo U, Aboian M, Yu T, Chung V, Bergquist T, Eddy J, Albrecht J, Baid U, Bakas S, Linguraru MG, Menze B, Iglesias JE, Wiestler B. The Brain Tumor Segmentation (BraTS) Challenge 2023: Brain MR Image Synthesis for Tumor Segmentation (BraSyn). ArXiv 2023:arXiv:2305.09011v5. [PMID: 37608932 PMCID: PMC10441440] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 08/24/2023]
Abstract
Automated brain tumor segmentation methods have become well-established and reached performance levels offering clear clinical utility. These methods typically rely on four input magnetic resonance imaging (MRI) modalities: T1-weighted images with and without contrast enhancement, T2-weighted images, and FLAIR images. However, some sequences are often missing in clinical practice due to time constraints or image artifacts, such as patient motion. Consequently, the ability to substitute missing modalities and gain segmentation performance is highly desirable and necessary for the broader adoption of these algorithms in the clinical routine. In this work, we present the establishment of the Brain MR Image Synthesis Benchmark (BraSyn) in conjunction with the Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2023. The primary objective of this challenge is to evaluate image synthesis methods that can realistically generate missing MRI modalities when multiple available images are provided. The ultimate aim is to facilitate automated brain tumor segmentation pipelines. The image dataset used in the benchmark is diverse and multi-modal, created through collaboration with various hospitals and research institutions.
Collapse
Affiliation(s)
- Hongwei Bran Li
- University of Zurich, Switzerland
- Department of Informatics, Technical University Munich, Germany
- Klinikum rechts der Isar, Technical University of Munich, Germany
| | | | - Syed Muhammad Anwar
- Children's National Hospital, Washington DC, USA
- George Washington University, Washington DC, USA
| | - Florian Kofler
- Helmholtz AI, Helmholtz Munich, Germany
- Department of Informatics, Technical University Munich, Germany
- TranslaTUM - Central Institute for Translational Cancer Research, Technical University of Munich, Germany
- Department of Diagnostic and Interventional Neuroradiology, School of Medicine, Klinikum rechts der Isar, Technical University of Munich, Germany
| | - Ivan Ezhov
- Department of Informatics, Technical University Munich, Germany
| | | | | | | | | | - Evan Calabrese
- Duke University Medical Center, Department of Radiology, USA
- University of California San Francisco, CA, USA
| | - Jeff Rudie
- University of California San Francisco, CA, USA
| | - Felix Meissen
- Department of Informatics, Technical University Munich, Germany
| | - Maruf Adewole
- Medical Artificial Intelligence (MAI) Lab, Crestview Radiology, Lagos, Nigeria
| | | | - Anahita Fathi Kazerooni
- Children's Hospital of Philadelphia, University of Pennsylvania, Philadelphia, PA, USA
- Center for AI and Data Science for Integrated Diagnostics (AI2D) I& Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
| | - Dominic LaBella
- Duke University Medical Center, Department of Radiation Oncology, USA
| | | | - Keyvan Farahani
- Cancer Imaging Program, National Cancer Institute, National Institutes of Health, Bethesda, MD 20814, USA
| | | | | | | | - Russell Takeshi Shinohara
- Center for AI and Data Science for Integrated Diagnostics (AI2D) I& Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
- Center for Clinical Epidemiology and Biostatistics, University of Pennsylvania, Philadelphia, USA
| | - Farouk Dako
- Center for Global Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Walter Wiggins
- Duke University Medical Center, Department of Radiology, USA
| | - Zachary Reitman
- Duke University Medical Center, Department of Radiation Oncology, USA
| | - Chunhao Wang
- Duke University Medical Center, Department of Radiation Oncology, USA
| | - Xinyang Liu
- Children's National Hospital, Washington DC, USA
- George Washington University, Washington DC, USA
| | - Zhifan Jiang
- Children's National Hospital, Washington DC, USA
- George Washington University, Washington DC, USA
| | - Ariana Familiar
- Children's Hospital of Philadelphia, University of Pennsylvania, Philadelphia, PA, USA
| | - Elaine Johanson
- PrecisionFDA, U.S. Food and Drug Administration, Silver Spring, MD, USA
| | | | - Christos Davatzikos
- Center for AI and Data Science for Integrated Diagnostics (AI2D) I& Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - John Freymann
- Leidos Biomedical Research, Inc., Frederick National Laboratory for Cancer Research, Frederick, MD 21701, USA
- Cancer Imaging Program, National Cancer Institute, National Institutes of Health, Bethesda, MD 20814, USA
| | - Justin Kirby
- Leidos Biomedical Research, Inc., Frederick National Laboratory for Cancer Research, Frederick, MD 21701, USA
- Cancer Imaging Program, National Cancer Institute, National Institutes of Health, Bethesda, MD 20814, USA
| | - Michel Bilello
- Center for AI and Data Science for Integrated Diagnostics (AI2D) I& Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | | | - Roland Wiest
- Institute for Surgical Technology and Biomechanics, University of Bern, Bern, Switzerland
- Support Centre for Advanced Neuroimaging Inselspital, Institute for Diagnostic and Interventional Neuroradiology, Bern University Hospital, Bern, Switzerland
| | - Jan Kirschke
- Department of Neuroradiology, Technical University of Munich, Munich, Germany
| | - Rivka R Colen
- University of Pittsburgh Medical Center, PA, USA
- Department of Diagnostic Radiology, University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Aikaterini Kotrotsou
- Department of Diagnostic Radiology, University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | | | - Daniel Marcus
- Neuroimaging Informatics and Analysis Center, Washington University, St. Louis, MO, USA
- Department of Radiology, Washington University, St. Louis, MO, USA
| | - Mikhail Milchenko
- Neuroimaging Informatics and Analysis Center, Washington University, St. Louis, MO, USA
- Department of Radiology, Washington University, St. Louis, MO, USA
| | - Arash Nazeri
- Department of Radiology, Washington University, St. Louis, MO, USA
| | - Marc-André Weber
- Institute of Diagnostic and Interventional Radiology, Pediatric Radiology and Neuroradiology, University Medical Center Rostock, Ernst-Heydemann-Str. 6, 18057 Rostock, Germany
| | - Abhishek Mahajan
- Tata Memorial Centre, Homi Bhabha National Institute, Mumbai, India
| | - Suyash Mohan
- Center for AI and Data Science for Integrated Diagnostics (AI2D) I& Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - John Mongan
- University of California San Francisco, CA, USA
| | | | - Soonmee Cha
- University of California San Francisco, CA, USA
| | | | | | | | | | | | - Udunna Anazodo
- Montreal Neurological Institute (MNI), McGill University, Montreal, QC, Canada
| | | | - Thomas Yu
- Athinoula A. Martinos Center for Biomedical Imaging, Harvard Medical School, USA
| | | | | | | | | | - Ujjwal Baid
- Center for AI and Data Science for Integrated Diagnostics (AI2D) I& Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
- Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Spyridon Bakas
- Center for AI and Data Science for Integrated Diagnostics (AI2D) I& Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
- Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | | | | | | | | |
Collapse
|
11
|
Moawad AW, Janas A, Baid U, Ramakrishnan D, Jekel L, Krantchev K, Moy H, Saluja R, Osenberg K, Wilms K, Kaur M, Avesta A, Pedersen GC, Maleki N, Salimi M, Merkaj S, von Reppert M, Tillmans N, Lost J, Bousabarah K, Holler W, Lin M, Westerhoff M, Maresca R, Link KE, Tahon NH, Marcus D, Sotiras A, LaMontagne P, Chakrabarty S, Teytelboym O, Youssef A, Nada A, Velichko YS, Gennaro N, Cramer J, Johnson DR, Kwan BY, Petrovic B, Patro SN, Wu L, So T, Thompson G, Kam A, Perez-Carrillo GG, Lall N, Albrecht J, Anazodo U, Lingaru MG, Menze BH, Wiestler B, Adewole M, Anwar SM, Labella D, Li HB, Iglesias JE, Farahani K, Eddy J, Bergquist T, Chung V, Shinohara RT, Dako F, Wiggins W, Reitman Z, Wang C, Liu X, Jiang Z, Van Leemput K, Piraud M, Ezhov I, Johanson E, Meier Z, Familiar A, Kazerooni AF, Kofler F, Calabrese E, Aneja S, Chiang V, Ikuta I, Shafique U, Memon F, Conte GM, Bakas S, Rudie J, Aboian M. The Brain Tumor Segmentation (BraTS-METS) Challenge 2023: Brain Metastasis Segmentation on Pre-treatment MRI. ArXiv 2023:arXiv:2306.00838v1. [PMID: 37396600 PMCID: PMC10312806] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Subscribe] [Scholar Register] [Indexed: 07/04/2023]
Abstract
Clinical monitoring of metastatic disease to the brain can be a laborious and timeconsuming process, especially in cases involving multiple metastases when the assessment is performed manually. The Response Assessment in Neuro-Oncology Brain Metastases (RANO-BM) guideline, which utilizes the unidimensional longest diameter, is commonly used in clinical and research settings to evaluate response to therapy in patients with brain metastases. However, accurate volumetric assessment of the lesion and surrounding peri-lesional edema holds significant importance in clinical decision-making and can greatly enhance outcome prediction. The unique challenge in performing segmentations of brain metastases lies in their common occurrence as small lesions. Detection and segmentation of lesions that are smaller than 10 mm in size has not demonstrated high accuracy in prior publications. The brain metastases challenge sets itself apart from previously conducted MICCAI challenges on glioma segmentation due to the significant variability in lesion size. Unlike gliomas, which tend to be larger on presentation scans, brain metastases exhibit a wide range of sizes and tend to include small lesions. We hope that the BraTS-METS dataset and challenge will advance the field of automated brain metastasis detection and segmentation.
Collapse
Affiliation(s)
| | - Anastasia Janas
- Yale University School of Medicine, Department of Radiology, New Haven, CT
- ImagineQuant, Yale University School of Medicine, Department of Radiology, New Haven, CT
- Charité - Universitatsmedizin, Berlin, Germany
| | - Ujjwal Baid
- Center for Biomedical Image Computing and Analytics, University of Pennsylvania School of Medicine, Philadelphia, PA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Divya Ramakrishnan
- Yale University School of Medicine, Department of Radiology, New Haven, CT
- ImagineQuant, Yale University School of Medicine, Department of Radiology, New Haven, CT
| | - Leon Jekel
- ImagineQuant, Yale University School of Medicine, Department of Radiology, New Haven, CT
- DKFZ Division of Translational Neurooncology at the WTZ, German Cancer Consortium, DKTK Partner Site, University Hospital Essen, Essen, Germany
- German Cancer Research Center, Heidelberg, Germany
- University of Ulm, Ulm, Germany
| | - Kiril Krantchev
- ImagineQuant, Yale University School of Medicine, Department of Radiology, New Haven, CT
- Charité - Universitatsmedizin, Berlin, Germany
| | - Harrison Moy
- Yale University School of Medicine, Department of Radiology, New Haven, CT
- ImagineQuant, Yale University School of Medicine, Department of Radiology, New Haven, CT
| | | | - Klara Osenberg
- Yale University School of Medicine, Department of Radiology, New Haven, CT
- ImagineQuant, Yale University School of Medicine, Department of Radiology, New Haven, CT
- University of Leipzig, Leipzig, Germany
| | - Klara Wilms
- Yale University School of Medicine, Department of Radiology, New Haven, CT
- ImagineQuant, Yale University School of Medicine, Department of Radiology, New Haven, CT
- University of Leipzig, Leipzig, Germany
| | - Manpreet Kaur
- Yale University School of Medicine, Department of Radiology, New Haven, CT
- ImagineQuant, Yale University School of Medicine, Department of Radiology, New Haven, CT
- Ludwig Maximillian University, Munich, Germany
| | - Arman Avesta
- Yale University School of Medicine, Department of Radiology, New Haven, CT
| | - Gabriel Cassinelli Pedersen
- Yale University School of Medicine, Department of Radiology, New Haven, CT
- ImagineQuant, Yale University School of Medicine, Department of Radiology, New Haven, CT
| | - Nazanin Maleki
- Yale University School of Medicine, Department of Radiology, New Haven, CT
- ImagineQuant, Yale University School of Medicine, Department of Radiology, New Haven, CT
| | - Mahdi Salimi
- Yale University School of Medicine, Department of Radiology, New Haven, CT
- ImagineQuant, Yale University School of Medicine, Department of Radiology, New Haven, CT
| | - Sarah Merkaj
- Yale University School of Medicine, Department of Radiology, New Haven, CT
- ImagineQuant, Yale University School of Medicine, Department of Radiology, New Haven, CT
- University of Ulm, Ulm, Germany
| | - Marc von Reppert
- Yale University School of Medicine, Department of Radiology, New Haven, CT
- ImagineQuant, Yale University School of Medicine, Department of Radiology, New Haven, CT
- University of Leipzig, Leipzig, Germany
| | - Niklas Tillmans
- Yale University School of Medicine, Department of Radiology, New Haven, CT
- ImagineQuant, Yale University School of Medicine, Department of Radiology, New Haven, CT
- University of Dusseldorf, Medical Faculty, Department of Diagnostic and Interventional Radiology, Dusseldorf, Germany
| | - Jan Lost
- Yale University School of Medicine, Department of Radiology, New Haven, CT
- ImagineQuant, Yale University School of Medicine, Department of Radiology, New Haven, CT
- University of Dusseldorf, Medical Faculty, Department of Diagnostic and Interventional Radiology, Dusseldorf, Germany
| | | | | | - MingDe Lin
- Visage Imaging, Inc, San Diego, California, USA
| | | | - Ryan Maresca
- Yale University School of Medicine, Department of Therapeutic Radiology, New Haven, CT
| | | | | | | | | | | | | | | | - Ayda Youssef
- Yale University School of Medicine, Department of Radiology, New Haven, CT
| | | | - Yuri S. Velichko
- Northwestern University, Department of Radiology, Feinberg School of Medicine, Chicago, IL
| | - Nicolo Gennaro
- Northwestern University, Department of Radiology, Feinberg School of Medicine, Chicago, IL
| | - Connectome Students
- Connectome – Student Association for Neurosurgery, Neurology and Neurosciences E.V
| | | | | | | | - Benjamin Y.M. Kwan
- Queen’s University, Department of Diagnostic Radiology, Kingston, Canada
| | | | - Satya N. Patro
- University of Arkansas for Medical Sciences, Little Rock, AR, USA
| | - Lei Wu
- University of Washington Department of Radiology, Seattle, WA
| | - Tiffany So
- Department of Imaging and Interventional Radiology, Faculty of Medicine, The Chinese University of Hong Kong
| | | | - Anthony Kam
- Loyola University Medical Center, Chicago, IL
| | | | - Neil Lall
- Children’s Healthcare of Atlanta, Atlanta, GA
| | - Group of Approvers
- Connectome – Student Association for Neurosurgery, Neurology and Neurosciences E.V
| | | | - Udunna Anazodo
- Montreal Neurological Institute (MNI), McGill University, Montreal, CA
| | | | - Bjoern H Menze
- Biomedical Image Analysis & Machine Learning, Department of Quantitative Biomedicine, University of Zurich, Switzerland
| | - Benedikt Wiestler
- Department of Neuroradiology, Technical University of Munich, Munich, Germany
| | - Maruf Adewole
- Medical Artificial Intelligence (MAI) Lab, Crestview Radiology, Lagos, Nigeria
| | | | | | - Hongwei Bran Li
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, MA
| | - Juan Eugenio Iglesias
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, MA
| | - Keyvan Farahani
- Cancer Imaging Program, National Cancer Institute, National Institutes of Health, Bethesda, MD
| | | | | | | | - Russel Takeshi Shinohara
- Center for Clinical Epidemiology and Biostatistics, University of Pennsylvania, Philadelphia, PA
| | - Farouk Dako
- Center for Global Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | | | | | | | - Xinyang Liu
- Children’s National Hospital, Washington DC, USA
| | - Zhifan Jiang
- Children’s National Hospital, Washington DC, USA
| | - Koen Van Leemput
- Department of Applied Mathematics and Computer Science, Technical University of Denmark, Denmark
| | | | - Ivan Ezhov
- Department of Informatics, Technical University Munich, Germany
| | - Elaine Johanson
- PrecisionFDA, U.S. Food and Drug Administration, Silver Spring, MD
| | | | - Ariana Familiar
- Children’s Hospital of Philadelphia, University of Pennsylvania, Philadelphia, PA
| | | | | | | | - Sanjay Aneja
- Yale University School of Medicine, Department of Therapeutic Radiology, New Haven, CT
| | - Veronica Chiang
- Yale University School of Medicine, Department of Neurosurgery, New Haven, CT
| | | | | | - Fatima Memon
- Yale University School of Medicine, Department of Radiology, New Haven, CT
- ImagineQuant, Yale University School of Medicine, Department of Radiology, New Haven, CT
| | | | - Spyridon Bakas
- Center for Biomedical Image Computing and Analytics, University of Pennsylvania School of Medicine, Philadelphia, PA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Jeffrey Rudie
- University of California San Diego, San Diego, CA
- University of California San Francisco, San Francisco, CA
| | - Mariam Aboian
- Yale University School of Medicine, Department of Radiology, New Haven, CT
- ImagineQuant, Yale University School of Medicine, Department of Radiology, New Haven, CT
| |
Collapse
|
12
|
Adewole M, Rudie JD, Gbdamosi A, Toyobo O, Raymond C, Zhang D, Omidiji O, Akinola R, Suwaid MA, Emegoakor A, Ojo N, Aguh K, Kalaiwo C, Babatunde G, Ogunleye A, Gbadamosi Y, Iorpagher K, Calabrese E, Aboian M, Linguraru M, Albrecht J, Wiestler B, Kofler F, Janas A, LaBella D, Kzerooni AF, Li HB, Iglesias JE, Farahani K, Eddy J, Bergquist T, Chung V, Shinohara RT, Wiggins W, Reitman Z, Wang C, Liu X, Jiang Z, Familiar A, Van Leemput K, Bukas C, Piraud M, Conte GM, Johansson E, Meier Z, Menze BH, Baid U, Bakas S, Dako F, Fatade A, Anazodo UC. The Brain Tumor Segmentation (BraTS) Challenge 2023: Glioma Segmentation in Sub-Saharan Africa Patient Population (BraTS-Africa). ArXiv 2023:arXiv:2305.19369v1. [PMID: 37396608 PMCID: PMC10312814] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Subscribe] [Scholar Register] [Indexed: 07/04/2023]
Abstract
Gliomas are the most common type of primary brain tumors. Although gliomas are relatively rare, they are among the deadliest types of cancer, with a survival rate of less than 2 years after diagnosis. Gliomas are challenging to diagnose, hard to treat and inherently resistant to conventional therapy. Years of extensive research to improve diagnosis and treatment of gliomas have decreased mortality rates across the Global North, while chances of survival among individuals in low- and middle-income countries (LMICs) remain unchanged and are significantly worse in Sub-Saharan Africa (SSA) populations. Long-term survival with glioma is associated with the identification of appropriate pathological features on brain MRI and confirmation by histopathology. Since 2012, the Brain Tumor Segmentation (BraTS) Challenge have evaluated state-of-the-art machine learning methods to detect, characterize, and classify gliomas. However, it is unclear if the state-of-the-art methods can be widely implemented in SSA given the extensive use of lower-quality MRI technology, which produces poor image contrast and resolution and more importantly, the propensity for late presentation of disease at advanced stages as well as the unique characteristics of gliomas in SSA (i.e., suspected higher rates of gliomatosis cerebri). Thus, the BraTS-Africa Challenge provides a unique opportunity to include brain MRI glioma cases from SSA in global efforts through the BraTS Challenge to develop and evaluate computer-aided-diagnostic (CAD) methods for the detection and characterization of glioma in resource-limited settings, where the potential for CAD tools to transform healthcare are more likely.
Collapse
Affiliation(s)
- Maruf Adewole
- Medical Artificial Intelligence Laboratory (MAI Lab), Lagos, Nigeria
- Department of Radiation Biology, Radiotherapy and Radiodiagnosis, University of Lagos, Lagos, Nigeria
| | - Jeffrey D Rudie
- Department of Radiology, University of California, San Diego
| | - Anu Gbdamosi
- Medical Artificial Intelligence Laboratory (MAI Lab), Lagos, Nigeria
- Crestview Radiology Limited, Lagos, Nigeria
| | - Oluyemisi Toyobo
- Medical Artificial Intelligence Laboratory (MAI Lab), Lagos, Nigeria
- Crestview Radiology Limited, Lagos, Nigeria
| | | | - Dong Zhang
- Medical Artificial Intelligence Laboratory (MAI Lab), Lagos, Nigeria
| | - Olubukola Omidiji
- Medical Artificial Intelligence Laboratory (MAI Lab), Lagos, Nigeria
- Lagos University Teaching Hospital, Lagos, Nigeria
| | - Rachel Akinola
- Lagos State University Teaching Hospital, Ikeja, Lagos, Nigeria
| | | | - Adaobi Emegoakor
- Nnamdi Azikiwe University Teaching Hospital, Nnewi, Anambra State, Nigeria
| | - Nancy Ojo
- Federal Medical Centre, Abeokuta, Ogun State, Nigeria
| | - Kenneth Aguh
- Federal Medical Centre, Umahia, Abia State, Nigeria
| | | | | | | | | | - Kator Iorpagher
- Benue State University Teaching Hospital, Markurdi, Benue State, Nigeria
| | - Evan Calabrese
- Duke University Medical Center, Department of Radiology, USA
- University of California San Francisco, CA, USA
| | | | - Marius Linguraru
- Children's National Hospital, Washington DC, USA
- George Washington University, Washington DC, USA
| | | | - Benedikt Wiestler
- Department of Neuroradiology, Technical University of Munich, Munich, Germany
| | - Florian Kofler
- Department of Neuroradiology, Technical University of Munich, Munich, Germany
- Helmholtz Research Center, Munich, Germany
| | | | - Dominic LaBella
- Duke University Medical Center, Department of Radiation Oncology, USA
| | - Anahita Fathi Kzerooni
- Children's Hospital of Philadelphia, University of Pennsylvania, Philadelphia, PA, USA
- Center for AI and Data Science for Integrated Diagnostics (AI2D) & Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
| | - Hongwei Bran Li
- Department of Neuroradiology, Technical University of Munich, Munich, Germany
- Athinoula A Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, MA, USA
- University of Zurich, Switzerland
| | - Juan Eugenio Iglesias
- Athinoula A Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, MA, USA
| | - Keyvan Farahani
- Cancer Imaging Program, National Cancer Institute, National Institutes of Health, Bethesda, MD 20814, USA
| | | | | | | | - Russell Takeshi Shinohara
- Center for AI and Data Science for Integrated Diagnostics (AI2D) & Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
- Center for Clinical Epidemiology and Biostatistics, University of Pennsylvania, Philadelphia, USA
| | - Walter Wiggins
- Duke University Medical Center, Department of Radiology, USA
| | - Zachary Reitman
- Duke University Medical Center, Department of Radiation Oncology, USA
| | - Chunhao Wang
- Duke University Medical Center, Department of Radiation Oncology, USA
| | - Xinyang Liu
- Children's National Hospital, Washington DC, USA
- George Washington University, Washington DC, USA
| | - Zhifan Jiang
- Children's National Hospital, Washington DC, USA
- George Washington University, Washington DC, USA
| | - Ariana Familiar
- Children's Hospital of Philadelphia, University of Pennsylvania, Philadelphia, PA, USA
| | - Koen Van Leemput
- Department of Applied Mathematics and Computer Science, Technical University of Denmark, Denmark
| | | | | | | | - Elaine Johansson
- Precision FDA, U.S. Food and Drug Administration, Silver Spring, MD, USA
| | | | - Bjoern H Menze
- Department of Neuroradiology, Technical University of Munich, Munich, Germany
- University of Zurich, Switzerland
| | - Ujjwal Baid
- Center for AI and Data Science for Integrated Diagnostics (AI2D) & Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
- Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Spyridon Bakas
- Center for AI and Data Science for Integrated Diagnostics (AI2D) & Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
- Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Farouk Dako
- Center for Global Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Abiodun Fatade
- Medical Artificial Intelligence Laboratory (MAI Lab), Lagos, Nigeria
- Crestview Radiology Limited, Lagos, Nigeria
| | - Udunna C Anazodo
- Medical Artificial Intelligence Laboratory (MAI Lab), Lagos, Nigeria
- Montreal Neurological Institute, McGill University, Montreal, Canada
- Department of Medicine, University of Cape Town, South Africa
- Department of Radiation Medicine, University of Cape Town, South Africa
| |
Collapse
|
13
|
LaBella D, Adewole M, Alonso-Basanta M, Altes T, Anwar SM, Baid U, Bergquist T, Bhalerao R, Chen S, Chung V, Conte GM, Dako F, Eddy J, Ezhov I, Godfrey D, Hilal F, Familiar A, Farahani K, Iglesias JE, Jiang Z, Johanson E, Kazerooni AF, Kent C, Kirkpatrick J, Kofler F, Leemput KV, Li HB, Liu X, Mahtabfar A, McBurney-Lin S, McLean R, Meier Z, Moawad AW, Mongan J, Nedelec P, Pajot M, Piraud M, Rashid A, Reitman Z, Shinohara RT, Velichko Y, Wang C, Warman P, Wiggins W, Aboian M, Albrecht J, Anazodo U, Bakas S, Flanders A, Janas A, Khanna G, Linguraru MG, Menze B, Nada A, Rauschecker AM, Rudie J, Tahon NH, Villanueva-Meyer J, Wiestler B, Calabrese E. The ASNR-MICCAI Brain Tumor Segmentation (BraTS) Challenge 2023: Intracranial Meningioma. ArXiv 2023:arXiv:2305.07642v1. [PMID: 37608937 PMCID: PMC10441446] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Subscribe] [Scholar Register] [Indexed: 08/24/2023]
Abstract
Meningiomas are the most common primary intracranial tumor in adults and can be associated with significant morbidity and mortality. Radiologists, neurosurgeons, neuro-oncologists, and radiation oncologists rely on multiparametric MRI (mpMRI) for diagnosis, treatment planning, and longitudinal treatment monitoring; yet automated, objective, and quantitative tools for non-invasive assessment of meningiomas on mpMRI are lacking. The BraTS meningioma 2023 challenge will provide a community standard and benchmark for state-of-the-art automated intracranial meningioma segmentation models based on the largest expert annotated multilabel meningioma mpMRI dataset to date. Challenge competitors will develop automated segmentation models to predict three distinct meningioma sub-regions on MRI including enhancing tumor, non-enhancing tumor core, and surrounding nonenhancing T2/FLAIR hyperintensity. Models will be evaluated on separate validation and held-out test datasets using standardized metrics utilized across the BraTS 2023 series of challenges including the Dice similarity coefficient and Hausdorff distance. The models developed during the course of this challenge will aid in incorporation of automated meningioma MRI segmentation into clinical practice, which will ultimately improve care of patients with meningioma.
Collapse
|
14
|
Clunie DA, Flanders A, Taylor A, Erickson B, Bialecki B, Brundage D, Gutman D, Prior F, Seibert JA, Perry J, Gichoya JW, Kirby J, Andriole K, Geneslaw L, Moore S, Fitzgerald TJ, Tellis W, Xiao Y, Farahani K, Luo J, Rosenthal A, Kandarpa K, Rosen R, Goetz K, Babcock D, Xu B, Hsiao J. Report of the Medical Image De-Identification (MIDI) Task Group - Best Practices and Recommendations. ArXiv 2023:arXiv:2303.10473v2. [PMID: 37033463 PMCID: PMC10081345] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Grants] [Subscribe] [Scholar Register] [Indexed: 04/11/2023]
Affiliation(s)
| | | | | | | | | | | | | | - Fred Prior
- University of Arkansas for Medical Sciences
| | | | | | | | - Justin Kirby
- Frederick National Laboratory for Cancer Research
| | | | | | | | | | | | - Ying Xiao
- University of Pennsylvania Health System
| | | | - James Luo
- National Heart, Lung, and Blood Institute (NHLBI)
| | - Alex Rosenthal
- National Institute of Allergy and Infectious Diseases (NIAID)
| | - Kris Kandarpa
- National Institute of Biomedical Imaging and Bioengineering (NIBIB)
| | - Rebecca Rosen
- Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD)
| | | | - Debra Babcock
- National Institute of Neurological Disorders and Stroke (NINDS)
| | - Ben Xu
- National Institute on Alcohol Abuse and Alcoholism (NIAAA)
| | | |
Collapse
|
15
|
Konz N, Buda M, Gu H, Saha A, Yang J, Chłędowski J, Park J, Witowski J, Geras KJ, Shoshan Y, Gilboa-Solomon F, Khapun D, Ratner V, Barkan E, Ozery-Flato M, Martí R, Omigbodun A, Marasinou C, Nakhaei N, Hsu W, Sahu P, Hossain MB, Lee J, Santos C, Przelaskowski A, Kalpathy-Cramer J, Bearce B, Cha K, Farahani K, Petrick N, Hadjiiski L, Drukker K, Armato SG, Mazurowski MA. A Competition, Benchmark, Code, and Data for Using Artificial Intelligence to Detect Lesions in Digital Breast Tomosynthesis. JAMA Netw Open 2023; 6:e230524. [PMID: 36821110 PMCID: PMC9951043 DOI: 10.1001/jamanetworkopen.2023.0524] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/24/2023] Open
Abstract
IMPORTANCE An accurate and robust artificial intelligence (AI) algorithm for detecting cancer in digital breast tomosynthesis (DBT) could significantly improve detection accuracy and reduce health care costs worldwide. OBJECTIVES To make training and evaluation data for the development of AI algorithms for DBT analysis available, to develop well-defined benchmarks, and to create publicly available code for existing methods. DESIGN, SETTING, AND PARTICIPANTS This diagnostic study is based on a multi-institutional international grand challenge in which research teams developed algorithms to detect lesions in DBT. A data set of 22 032 reconstructed DBT volumes was made available to research teams. Phase 1, in which teams were provided 700 scans from the training set, 120 from the validation set, and 180 from the test set, took place from December 2020 to January 2021, and phase 2, in which teams were given the full data set, took place from May to July 2021. MAIN OUTCOMES AND MEASURES The overall performance was evaluated by mean sensitivity for biopsied lesions using only DBT volumes with biopsied lesions; ties were broken by including all DBT volumes. RESULTS A total of 8 teams participated in the challenge. The team with the highest mean sensitivity for biopsied lesions was the NYU B-Team, with 0.957 (95% CI, 0.924-0.984), and the second-place team, ZeDuS, had a mean sensitivity of 0.926 (95% CI, 0.881-0.964). When the results were aggregated, the mean sensitivity for all submitted algorithms was 0.879; for only those who participated in phase 2, it was 0.926. CONCLUSIONS AND RELEVANCE In this diagnostic study, an international competition produced algorithms with high sensitivity for using AI to detect lesions on DBT images. A standardized performance benchmark for the detection task using publicly available clinical imaging data was released, with detailed descriptions and analyses of submitted algorithms accompanied by a public release of their predictions and code for selected methods. These resources will serve as a foundation for future research on computer-assisted diagnosis methods for DBT, significantly lowering the barrier of entry for new researchers.
Collapse
Affiliation(s)
- Nicholas Konz
- Department of Electrical and Computer Engineering, Duke University, Durham, North Carolina
| | - Mateusz Buda
- Department of Radiology, Duke University Medical Center, Durham, North Carolina
- Faculty of Mathematics and Information Science, Warsaw University of Technology, Warsaw, Poland
| | - Hanxue Gu
- Department of Electrical and Computer Engineering, Duke University, Durham, North Carolina
| | - Ashirbani Saha
- Department of Radiology, Duke University Medical Center, Durham, North Carolina
- Department of Oncology, McMaster University, Hamilton, Ontario, Canada
| | | | - Jakub Chłędowski
- Jagiellonian University, Kraków, Poland
- Department of Radiology, NYU Grossman School of Medicine, New York, New York
| | - Jungkyu Park
- Department of Radiology, NYU Grossman School of Medicine, New York, New York
| | - Jan Witowski
- Department of Radiology, NYU Grossman School of Medicine, New York, New York
| | - Krzysztof J. Geras
- Department of Radiology, NYU Grossman School of Medicine, New York, New York
| | - Yoel Shoshan
- Medical Image Analytics, IBM Research, Haifa, Israel
| | | | - Daniel Khapun
- Medical Image Analytics, IBM Research, Haifa, Israel
| | - Vadim Ratner
- Medical Image Analytics, IBM Research, Haifa, Israel
| | - Ella Barkan
- Medical Image Analytics, IBM Research, Haifa, Israel
| | | | - Robert Martí
- Institute of Computer Vision and Robotics, University of Girona, Girona, Spain
| | - Akinyinka Omigbodun
- Medical and Imaging Informatics Group, Department of Radiological Sciences, David Geffen School of Medicine, University of California Los Angeles
| | - Chrysostomos Marasinou
- Medical and Imaging Informatics Group, Department of Radiological Sciences, David Geffen School of Medicine, University of California Los Angeles
| | - Noor Nakhaei
- Medical and Imaging Informatics Group, Department of Radiological Sciences, David Geffen School of Medicine, University of California Los Angeles
| | - William Hsu
- Medical and Imaging Informatics Group, Department of Radiological Sciences, David Geffen School of Medicine, University of California Los Angeles
- Department of Radiological Sciences, David Geffen School of Medicine, University of California Los Angeles
- Department of Bioengineering, University of California Los Angeles Samueli School of Engineering
| | - Pranjal Sahu
- Department of Computer Science, Stony Brook University, Stony Brook, New York
| | - Md Belayat Hossain
- Department of Radiology, University of Pittsburgh, Pittsburgh, Pennsylvania
| | - Juhun Lee
- Department of Radiology, University of Pittsburgh, Pittsburgh, Pennsylvania
| | - Carlos Santos
- Department of Radiology, Duke University Medical Center, Durham, North Carolina
| | - Artur Przelaskowski
- Faculty of Mathematics and Information Science, Warsaw University of Technology, Warsaw, Poland
| | - Jayashree Kalpathy-Cramer
- Department of Radiology, Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown
| | - Benjamin Bearce
- Department of Radiology, Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Charlestown
| | - Kenny Cha
- US Food and Drug Administration, Silver Spring, Maryland
| | - Keyvan Farahani
- Center for Biomedical Informatics and Information Technology, National Cancer Institute, Bethesda, Maryland
| | | | | | - Karen Drukker
- Department of Radiology, University of Chicago, Chicago, Illinois
| | - Samuel G. Armato
- Department of Radiology, University of Chicago, Chicago, Illinois
| | - Maciej A. Mazurowski
- Department of Electrical and Computer Engineering, Duke University, Durham, North Carolina
- Department of Radiology, Duke University Medical Center, Durham, North Carolina
- Department of Computer Science, Duke University, Durham, North Carolina
- Department of Biostatistics and Bioinformatics, Duke University Medical Center, Durham, North Carolina
| |
Collapse
|
16
|
Rankin J, Whelan B, Pollard-Larkin J, Paradis KC, Scarpelli M, Sun C, Mehta C, Farahani K, Castillo R. Diversity and Professional Advancement in Medical Physics. Adv Radiat Oncol 2022; 8:101057. [PMID: 36213550 PMCID: PMC9539787 DOI: 10.1016/j.adro.2022.101057] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2022] [Accepted: 08/10/2022] [Indexed: 11/30/2022] Open
Abstract
Purpose While disparities in the inclusion and advancement of women and minorities in science, technology, engineering, mathematics, and medical fields have been well documented, less work has focused on medical physics specifically. In this study, we evaluate historical and current diversity within the medical physics workforce, in cohorts representative of professional advancement (PA) in the field, and within National Institutes of Health (NIH)–funded medical physics research activities. Methods and Materials The 2020 American Association of Physicists in Medicine (AAPM) membership was queried as surrogate for the medical physics workforce. Select subsets of the AAPM membership were queried as surrogate for PA and early career professional advancement (ECPA) in medical physics. Self-reported AAPM-member demographics data representative of study analysis groups were identified and analyzed. Demographic characteristics of the 2020 AAPM membership were compared with those of the PA and ECPA cohorts and United States (US) population. The AAPM-NIH Research Database was appended with principal investigator (PI) demographics data and analyzed to evaluate trends in grant allocation by PI demographic characteristics. Results Women, Hispanic/Latinx/Spanish individuals, and individuals reporting a race other than White or Asian alone comprised 50.8%, 18.7%, and 32.4% of the US population, respectively, but only 23.9%, 9.1%, and 7.9% of the 2020 AAPM membership, respectively. In general, representation of women and minorities was further decreased in the PA cohort; however, significantly higher proportions of women (P < .001) and Hispanic/Latinx/Spanish members (P < .05) were observed in the ECPA cohort than the 2020 AAPM membership. Analysis of historical data revealed modest increases in diversity within the AAPM membership since 2002. Across NIH grants awarded to AAPM members between 1985 and 2020, only 9.4%, 5.3%, and 1.7% were awarded to women, Hispanic/Latinx/Spanish, and non-White, non-Asian PIs, respectively. Conclusions Diversity within medical physics is limited. Proactive policy should be implemented to ensure diverse, equitable, and inclusive representation within research activities, roles representative of PA, and the profession at large.
Collapse
Affiliation(s)
- Jillian Rankin
- Emory University School of Medicine, Department of Radiation Oncology, Atlanta, GA
| | | | - Julianne Pollard-Larkin
- The University of Texas MD Anderson Cancer Center, Department of Radiation Physics, Houston, TX
| | - Kelly C. Paradis
- University of Michigan, Department of Radiation Oncology, Ann Arbor, MI
| | | | - Chenbo Sun
- Emory University, Rollins School of Public Health, Department of Biostatistics and Bioinformatics, Atlanta, GA
| | - Christina Mehta
- Emory University School of Medicine, Department of Medicine, Division of Infectious Diseases, Atlanta, GA
| | - Keyvan Farahani
- National Cancer Institute, Center for Biomedical Informatics and Information Technology, Bethesda, MD
| | - Richard Castillo
- Emory University School of Medicine, Department of Radiation Oncology, Atlanta, GA
- Corresponding author
| |
Collapse
|
17
|
Antonelli M, Reinke A, Bakas S, Farahani K, Kopp-Schneider A, Landman BA, Litjens G, Menze B, Ronneberger O, Summers RM, van Ginneken B, Bilello M, Bilic P, Christ PF, Do RKG, Gollub MJ, Heckers SH, Huisman H, Jarnagin WR, McHugo MK, Napel S, Pernicka JSG, Rhode K, Tobon-Gomez C, Vorontsov E, Meakin JA, Ourselin S, Wiesenfarth M, Arbeláez P, Bae B, Chen S, Daza L, Feng J, He B, Isensee F, Ji Y, Jia F, Kim I, Maier-Hein K, Merhof D, Pai A, Park B, Perslev M, Rezaiifar R, Rippel O, Sarasua I, Shen W, Son J, Wachinger C, Wang L, Wang Y, Xia Y, Xu D, Xu Z, Zheng Y, Simpson AL, Maier-Hein L, Cardoso MJ. The Medical Segmentation Decathlon. Nat Commun 2022; 13:4128. [PMID: 35840566 PMCID: PMC9287542 DOI: 10.1038/s41467-022-30695-9] [Citation(s) in RCA: 115] [Impact Index Per Article: 57.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2021] [Accepted: 05/13/2022] [Indexed: 02/05/2023] Open
Abstract
International challenges have become the de facto standard for comparative assessment of image analysis algorithms. Although segmentation is the most widely investigated medical image processing task, the various challenges have been organized to focus only on specific clinical tasks. We organized the Medical Segmentation Decathlon (MSD)-a biomedical image analysis challenge, in which algorithms compete in a multitude of both tasks and modalities to investigate the hypothesis that a method capable of performing well on multiple tasks will generalize well to a previously unseen task and potentially outperform a custom-designed solution. MSD results confirmed this hypothesis, moreover, MSD winner continued generalizing well to a wide range of other clinical problems for the next two years. Three main conclusions can be drawn from this study: (1) state-of-the-art image segmentation algorithms generalize well when retrained on unseen tasks; (2) consistent algorithmic performance across multiple tasks is a strong surrogate of algorithmic generalizability; (3) the training of accurate AI segmentation models is now commoditized to scientists that are not versed in AI model training.
Collapse
Affiliation(s)
- Michela Antonelli
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK.
| | - Annika Reinke
- Div. Computer Assisted Medical Interventions, German Cancer Research Center (DKFZ), Heidelberg, Germany.,HI Helmholtz Imaging, German Cancer Research Center (DKFZ), Heidelberg, Germany.,Faculty of Mathematics and Computer Science, University of Heidelberg, Heidelberg, Germany
| | - Spyridon Bakas
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA.,Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.,Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Keyvan Farahani
- Center for Biomedical Informatics and Information Technology, National Cancer Institute (NIH), Bethesda, MD, USA
| | | | - Bennett A Landman
- Electrical Engineering and Computer Science, Vanderbilt University, Nashville, TN, USA
| | - Geert Litjens
- Radboud University Medical Center, Radboud Institute for Health Sciences, Nijmegen, The Netherlands
| | - Bjoern Menze
- Quantitative Biomedicine, University of Zurich, Zurich, Switzerland
| | | | - Ronald M Summers
- Imaging Biomarkers and Computer-Aided Diagnosis Laboratory, Department of Radiology and Imaging Sciences, National Institutes of Health Clinical Center (NIH), Bethesda, MD, USA
| | - Bram van Ginneken
- Radboud University Medical Center, Radboud Institute for Health Sciences, Nijmegen, The Netherlands
| | - Michel Bilello
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
| | - Patrick Bilic
- Department of Informatics, Technische Universität München, München, Germany
| | - Patrick F Christ
- Department of Informatics, Technische Universität München, München, Germany
| | - Richard K G Do
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Marc J Gollub
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Stephan H Heckers
- Department of Psychiatry & Behavioral Sciences, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Henkjan Huisman
- Radboud University Medical Center, Radboud Institute for Health Sciences, Nijmegen, The Netherlands
| | - William R Jarnagin
- Department of Surgery, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Maureen K McHugo
- Department of Psychiatry & Behavioral Sciences, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Sandy Napel
- Department of Radiology, Stanford University, Stanford, CA, USA
| | | | - Kawal Rhode
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| | - Catalina Tobon-Gomez
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| | - Eugene Vorontsov
- Department of Computer Science and Software Engineering, École Polytechnique de Montréal, Montréal, QC, Canada
| | - James A Meakin
- Radboud University Medical Center, Radboud Institute for Health Sciences, Nijmegen, The Netherlands
| | - Sebastien Ourselin
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| | - Manuel Wiesenfarth
- Div. Biostatistics, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | | | | | | | - Laura Daza
- Universidad de los Andes, Bogota, Colombia
| | - Jianjiang Feng
- Department of Automation, Tsinghua University, Beijing, China
| | - Baochun He
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Fabian Isensee
- HI Applied Computer Vision Lab, Division of Medical Image Computing, German Cancer Research Center (DKFZ), Heidelberg, Germany
| | - Yuanfeng Ji
- Department of Computer Science, Xiamen University, Xiamen, China
| | - Fucang Jia
- Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Ildoo Kim
- Kakao Brain, Seongnam-si, Republic of Korea
| | - Klaus Maier-Hein
- Cerebriu A/S, Copenhagen, Denmark.,Pattern Analysis and Learning Group, Department of Radiation Oncology, Heidelberg University Hospital, Heidelberg, Germany
| | - Dorit Merhof
- Institute of Imaging & Computer Vision, RWTH Aachen University, Aachen, Germany.,Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany
| | - Akshay Pai
- Cerebriu A/S, Copenhagen, Denmark.,Department of Computer Science, University of Copenhagen, Copenhagen, Denmark
| | | | - Mathias Perslev
- Department of Computer Science, University of Copenhagen, Copenhagen, Denmark
| | | | - Oliver Rippel
- Institute of Imaging & Computer Vision, RWTH Aachen University, Aachen, Germany
| | - Ignacio Sarasua
- Lab for Artificial Intelligence in Medical Imaging (AI-Med), Department of Child and Adolescent Psychiatry, University Hospital, LMU München, Germany
| | - Wei Shen
- MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai, China
| | | | - Christian Wachinger
- Lab for Artificial Intelligence in Medical Imaging (AI-Med), Department of Child and Adolescent Psychiatry, University Hospital, LMU München, Germany
| | - Liansheng Wang
- Department of Computer Science, Xiamen University, Xiamen, China
| | - Yan Wang
- Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University, Shanghai, China
| | - Yingda Xia
- Johns Hopkins University, Baltimore, MD, USA
| | | | - Zhanwei Xu
- Department of Automation, Tsinghua University, Beijing, China
| | | | - Amber L Simpson
- School of Computing/Department of Biomedical and Molecular Sciences, Queen's University, Kingston, ON, Canada
| | - Lena Maier-Hein
- Div. Computer Assisted Medical Interventions, German Cancer Research Center (DKFZ), Heidelberg, Germany.,HI Helmholtz Imaging, German Cancer Research Center (DKFZ), Heidelberg, Germany.,Faculty of Mathematics and Computer Science, University of Heidelberg, Heidelberg, Germany.,Medical Faculty, University of Heidelberg, Heidelberg, Germany
| | - M Jorge Cardoso
- School of Biomedical Engineering & Imaging Sciences, King's College London, London, UK
| |
Collapse
|
18
|
Afshar P, Rafiee MJ, Naderkhani F, Heidarian S, Enshaei N, Oikonomou A, Babaki Fard F, Anconina R, Farahani K, Plataniotis KN, Mohammadi A. Human-level COVID-19 diagnosis from low-dose CT scans using a two-stage time-distributed capsule network. Sci Rep 2022; 12:4827. [PMID: 35318368 PMCID: PMC8940967 DOI: 10.1038/s41598-022-08796-8] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2021] [Accepted: 03/01/2022] [Indexed: 01/01/2023] Open
Abstract
Reverse transcription-polymerase chain reaction is currently the gold standard in COVID-19 diagnosis. It can, however, take days to provide the diagnosis, and false negative rate is relatively high. Imaging, in particular chest computed tomography (CT), can assist with diagnosis and assessment of this disease. Nevertheless, it is shown that standard dose CT scan gives significant radiation burden to patients, especially those in need of multiple scans. In this study, we consider low-dose and ultra-low-dose (LDCT and ULDCT) scan protocols that reduce the radiation exposure close to that of a single X-ray, while maintaining an acceptable resolution for diagnosis purposes. Since thoracic radiology expertise may not be widely available during the pandemic, we develop an Artificial Intelligence (AI)-based framework using a collected dataset of LDCT/ULDCT scans, to study the hypothesis that the AI model can provide human-level performance. The AI model uses a two stage capsule network architecture and can rapidly classify COVID-19, community acquired pneumonia (CAP), and normal cases, using LDCT/ULDCT scans. Based on a cross validation, the AI model achieves COVID-19 sensitivity of \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$89.5\%\pm 0.11$$\end{document}89.5%±0.11, CAP sensitivity of \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$95\%\pm 0.11$$\end{document}95%±0.11, normal cases sensitivity (specificity) of \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$85.7\%\pm 0.16$$\end{document}85.7%±0.16, and accuracy of \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$90\%\pm 0.06$$\end{document}90%±0.06. By incorporating clinical data (demographic and symptoms), the performance further improves to COVID-19 sensitivity of \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$94.3\%\pm 0.05$$\end{document}94.3%±0.05, CAP sensitivity of \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$96.7\%\pm 0.07$$\end{document}96.7%±0.07, normal cases sensitivity (specificity) of \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$91\%\pm 0.09$$\end{document}91%±0.09 , and accuracy of \documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$94.1\%\pm 0.03$$\end{document}94.1%±0.03. The proposed AI model achieves human-level diagnosis based on the LDCT/ULDCT scans with reduced radiation exposure. We believe that the proposed AI model has the potential to assist the radiologists to accurately and promptly diagnose COVID-19 infection and help control the transmission chain during the pandemic.
Collapse
Affiliation(s)
- Parnian Afshar
- Concordia Institute for Information Systems Engineering (CIISE), Concordia University, Montreal, Canada.,Department of Electrical and Computer Engineering, University of Toronto, Toronto, Canada
| | - Moezedin Javad Rafiee
- Department of Medicine and Diagnostic Radiology, McGill University Health Center-Research Institute, Montreal, QC, Canada
| | - Farnoosh Naderkhani
- Concordia Institute for Information Systems Engineering (CIISE), Concordia University, Montreal, Canada
| | - Shahin Heidarian
- Department of Electrical and Computer Engineering, Concordia University, Montreal, QC, Canada
| | - Nastaran Enshaei
- Concordia Institute for Information Systems Engineering (CIISE), Concordia University, Montreal, Canada
| | - Anastasia Oikonomou
- Department of Medical Imaging, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Canada
| | | | - Reut Anconina
- Department of Medical Imaging, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Canada
| | - Keyvan Farahani
- Center for Biomedical Informatics and Information Technology, National Cancer Institute (NCI), Rockville, MD, USA
| | | | - Arash Mohammadi
- Concordia Institute for Information Systems Engineering (CIISE), Concordia University, Montreal, Canada.
| |
Collapse
|
19
|
Jacobs C, Setio AAA, Scholten ET, Gerke PK, Bhattacharya H, M Hoesein FA, Brink M, Ranschaert E, de Jong PA, Silva M, Geurts B, Chung K, Schalekamp S, Meersschaert J, Devaraj A, Pinsky PF, Lam SC, van Ginneken B, Farahani K. Deep Learning for Lung Cancer Detection on Screening CT Scans: Results of a Large-Scale Public Competition and an Observer Study with 11 Radiologists. Radiol Artif Intell 2021; 3:e210027. [PMID: 34870218 DOI: 10.1148/ryai.2021210027] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2021] [Revised: 10/11/2021] [Accepted: 10/13/2021] [Indexed: 12/15/2022]
Abstract
Purpose To determine whether deep learning algorithms developed in a public competition could identify lung cancer on low-dose CT scans with a performance similar to that of radiologists. Materials and Methods In this retrospective study, a dataset consisting of 300 patient scans was used for model assessment; 150 patient scans were from the competition set and 150 were from an independent dataset. Both test datasets contained 50 cancer-positive scans and 100 cancer-negative scans. The reference standard was set by histopathologic examination for cancer-positive scans and imaging follow-up for at least 2 years for cancer-negative scans. The test datasets were applied to the three top-performing algorithms from the Kaggle Data Science Bowl 2017 public competition: grt123, Julian de Wit and Daniel Hammack (JWDH), and Aidence. Model outputs were compared with an observer study of 11 radiologists that assessed the same test datasets. Each scan was scored on a continuous scale by both the deep learning algorithms and the radiologists. Performance was measured using multireader, multicase receiver operating characteristic analysis. Results The area under the receiver operating characteristic curve (AUC) was 0.877 (95% CI: 0.842, 0.910) for grt123, 0.902 (95% CI: 0.871, 0.932) for JWDH, and 0.900 (95% CI: 0.870, 0.928) for Aidence. The average AUC of the radiologists was 0.917 (95% CI: 0.889, 0.945), which was significantly higher than grt123 (P = .02); however, no significant difference was found between the radiologists and JWDH (P = .29) or Aidence (P = .26). Conclusion Deep learning algorithms developed in a public competition for lung cancer detection in low-dose CT scans reached performance close to that of radiologists.Keywords: Lung, CT, Thorax, Screening, Oncology Supplemental material is available for this article. © RSNA, 2021.
Collapse
Affiliation(s)
- Colin Jacobs
- Department of Radiology, Nuclear Medicine and Anatomy, Radboud University Medical Center, Geert Grooteplein 10, 6525 GA, Nijmegen, the Netherlands (C.J., A.A.A.S., E.T.S., P.K.G., H.B., M.B., B.G., S.S., B.v.G.); Department of Digital Technology & Innovation, Siemens Healthineers, Erlangen, Germany (A.A.A.S.); Department of Radiology, University Medical Center Utrecht, Utrecht, the Netherlands (F.A.M.H., P.A.d.J.); ETZ (Elisabeth-TweeSteden Ziekenhuis), Tilburg, the Netherlands (E.R.); Section of Radiology, Department of Medicine and Surgery (DiMeC), University of Parma, Parma, Italy (M.S.); Department of Radiology, Meander Medical Center, Amersfoort, the Netherlands (K.C., S.S.); Department of Radiology, AZ Zeno, Knokke-Heist, Belgium (J.M.); Department of Imaging, Royal Brompton Hospital, London, England (A.D.); Division of Cancer Prevention (P.F.P.) and Center for Biomedical Informatics & Information Technology (K.F.), National Cancer Institute, National Institutes of Health, Bethesda, Md; British Columbia Cancer Agency and the University of British Columbia, Vancouver, Canada (S.C.L.); and Fraunhofer MEVIS, Bremen, Germany (B.v.G.)
| | - Arnaud A A Setio
- Department of Radiology, Nuclear Medicine and Anatomy, Radboud University Medical Center, Geert Grooteplein 10, 6525 GA, Nijmegen, the Netherlands (C.J., A.A.A.S., E.T.S., P.K.G., H.B., M.B., B.G., S.S., B.v.G.); Department of Digital Technology & Innovation, Siemens Healthineers, Erlangen, Germany (A.A.A.S.); Department of Radiology, University Medical Center Utrecht, Utrecht, the Netherlands (F.A.M.H., P.A.d.J.); ETZ (Elisabeth-TweeSteden Ziekenhuis), Tilburg, the Netherlands (E.R.); Section of Radiology, Department of Medicine and Surgery (DiMeC), University of Parma, Parma, Italy (M.S.); Department of Radiology, Meander Medical Center, Amersfoort, the Netherlands (K.C., S.S.); Department of Radiology, AZ Zeno, Knokke-Heist, Belgium (J.M.); Department of Imaging, Royal Brompton Hospital, London, England (A.D.); Division of Cancer Prevention (P.F.P.) and Center for Biomedical Informatics & Information Technology (K.F.), National Cancer Institute, National Institutes of Health, Bethesda, Md; British Columbia Cancer Agency and the University of British Columbia, Vancouver, Canada (S.C.L.); and Fraunhofer MEVIS, Bremen, Germany (B.v.G.)
| | - Ernst T Scholten
- Department of Radiology, Nuclear Medicine and Anatomy, Radboud University Medical Center, Geert Grooteplein 10, 6525 GA, Nijmegen, the Netherlands (C.J., A.A.A.S., E.T.S., P.K.G., H.B., M.B., B.G., S.S., B.v.G.); Department of Digital Technology & Innovation, Siemens Healthineers, Erlangen, Germany (A.A.A.S.); Department of Radiology, University Medical Center Utrecht, Utrecht, the Netherlands (F.A.M.H., P.A.d.J.); ETZ (Elisabeth-TweeSteden Ziekenhuis), Tilburg, the Netherlands (E.R.); Section of Radiology, Department of Medicine and Surgery (DiMeC), University of Parma, Parma, Italy (M.S.); Department of Radiology, Meander Medical Center, Amersfoort, the Netherlands (K.C., S.S.); Department of Radiology, AZ Zeno, Knokke-Heist, Belgium (J.M.); Department of Imaging, Royal Brompton Hospital, London, England (A.D.); Division of Cancer Prevention (P.F.P.) and Center for Biomedical Informatics & Information Technology (K.F.), National Cancer Institute, National Institutes of Health, Bethesda, Md; British Columbia Cancer Agency and the University of British Columbia, Vancouver, Canada (S.C.L.); and Fraunhofer MEVIS, Bremen, Germany (B.v.G.)
| | - Paul K Gerke
- Department of Radiology, Nuclear Medicine and Anatomy, Radboud University Medical Center, Geert Grooteplein 10, 6525 GA, Nijmegen, the Netherlands (C.J., A.A.A.S., E.T.S., P.K.G., H.B., M.B., B.G., S.S., B.v.G.); Department of Digital Technology & Innovation, Siemens Healthineers, Erlangen, Germany (A.A.A.S.); Department of Radiology, University Medical Center Utrecht, Utrecht, the Netherlands (F.A.M.H., P.A.d.J.); ETZ (Elisabeth-TweeSteden Ziekenhuis), Tilburg, the Netherlands (E.R.); Section of Radiology, Department of Medicine and Surgery (DiMeC), University of Parma, Parma, Italy (M.S.); Department of Radiology, Meander Medical Center, Amersfoort, the Netherlands (K.C., S.S.); Department of Radiology, AZ Zeno, Knokke-Heist, Belgium (J.M.); Department of Imaging, Royal Brompton Hospital, London, England (A.D.); Division of Cancer Prevention (P.F.P.) and Center for Biomedical Informatics & Information Technology (K.F.), National Cancer Institute, National Institutes of Health, Bethesda, Md; British Columbia Cancer Agency and the University of British Columbia, Vancouver, Canada (S.C.L.); and Fraunhofer MEVIS, Bremen, Germany (B.v.G.)
| | - Haimasree Bhattacharya
- Department of Radiology, Nuclear Medicine and Anatomy, Radboud University Medical Center, Geert Grooteplein 10, 6525 GA, Nijmegen, the Netherlands (C.J., A.A.A.S., E.T.S., P.K.G., H.B., M.B., B.G., S.S., B.v.G.); Department of Digital Technology & Innovation, Siemens Healthineers, Erlangen, Germany (A.A.A.S.); Department of Radiology, University Medical Center Utrecht, Utrecht, the Netherlands (F.A.M.H., P.A.d.J.); ETZ (Elisabeth-TweeSteden Ziekenhuis), Tilburg, the Netherlands (E.R.); Section of Radiology, Department of Medicine and Surgery (DiMeC), University of Parma, Parma, Italy (M.S.); Department of Radiology, Meander Medical Center, Amersfoort, the Netherlands (K.C., S.S.); Department of Radiology, AZ Zeno, Knokke-Heist, Belgium (J.M.); Department of Imaging, Royal Brompton Hospital, London, England (A.D.); Division of Cancer Prevention (P.F.P.) and Center for Biomedical Informatics & Information Technology (K.F.), National Cancer Institute, National Institutes of Health, Bethesda, Md; British Columbia Cancer Agency and the University of British Columbia, Vancouver, Canada (S.C.L.); and Fraunhofer MEVIS, Bremen, Germany (B.v.G.)
| | - Firdaus A M Hoesein
- Department of Radiology, Nuclear Medicine and Anatomy, Radboud University Medical Center, Geert Grooteplein 10, 6525 GA, Nijmegen, the Netherlands (C.J., A.A.A.S., E.T.S., P.K.G., H.B., M.B., B.G., S.S., B.v.G.); Department of Digital Technology & Innovation, Siemens Healthineers, Erlangen, Germany (A.A.A.S.); Department of Radiology, University Medical Center Utrecht, Utrecht, the Netherlands (F.A.M.H., P.A.d.J.); ETZ (Elisabeth-TweeSteden Ziekenhuis), Tilburg, the Netherlands (E.R.); Section of Radiology, Department of Medicine and Surgery (DiMeC), University of Parma, Parma, Italy (M.S.); Department of Radiology, Meander Medical Center, Amersfoort, the Netherlands (K.C., S.S.); Department of Radiology, AZ Zeno, Knokke-Heist, Belgium (J.M.); Department of Imaging, Royal Brompton Hospital, London, England (A.D.); Division of Cancer Prevention (P.F.P.) and Center for Biomedical Informatics & Information Technology (K.F.), National Cancer Institute, National Institutes of Health, Bethesda, Md; British Columbia Cancer Agency and the University of British Columbia, Vancouver, Canada (S.C.L.); and Fraunhofer MEVIS, Bremen, Germany (B.v.G.)
| | - Monique Brink
- Department of Radiology, Nuclear Medicine and Anatomy, Radboud University Medical Center, Geert Grooteplein 10, 6525 GA, Nijmegen, the Netherlands (C.J., A.A.A.S., E.T.S., P.K.G., H.B., M.B., B.G., S.S., B.v.G.); Department of Digital Technology & Innovation, Siemens Healthineers, Erlangen, Germany (A.A.A.S.); Department of Radiology, University Medical Center Utrecht, Utrecht, the Netherlands (F.A.M.H., P.A.d.J.); ETZ (Elisabeth-TweeSteden Ziekenhuis), Tilburg, the Netherlands (E.R.); Section of Radiology, Department of Medicine and Surgery (DiMeC), University of Parma, Parma, Italy (M.S.); Department of Radiology, Meander Medical Center, Amersfoort, the Netherlands (K.C., S.S.); Department of Radiology, AZ Zeno, Knokke-Heist, Belgium (J.M.); Department of Imaging, Royal Brompton Hospital, London, England (A.D.); Division of Cancer Prevention (P.F.P.) and Center for Biomedical Informatics & Information Technology (K.F.), National Cancer Institute, National Institutes of Health, Bethesda, Md; British Columbia Cancer Agency and the University of British Columbia, Vancouver, Canada (S.C.L.); and Fraunhofer MEVIS, Bremen, Germany (B.v.G.)
| | - Erik Ranschaert
- Department of Radiology, Nuclear Medicine and Anatomy, Radboud University Medical Center, Geert Grooteplein 10, 6525 GA, Nijmegen, the Netherlands (C.J., A.A.A.S., E.T.S., P.K.G., H.B., M.B., B.G., S.S., B.v.G.); Department of Digital Technology & Innovation, Siemens Healthineers, Erlangen, Germany (A.A.A.S.); Department of Radiology, University Medical Center Utrecht, Utrecht, the Netherlands (F.A.M.H., P.A.d.J.); ETZ (Elisabeth-TweeSteden Ziekenhuis), Tilburg, the Netherlands (E.R.); Section of Radiology, Department of Medicine and Surgery (DiMeC), University of Parma, Parma, Italy (M.S.); Department of Radiology, Meander Medical Center, Amersfoort, the Netherlands (K.C., S.S.); Department of Radiology, AZ Zeno, Knokke-Heist, Belgium (J.M.); Department of Imaging, Royal Brompton Hospital, London, England (A.D.); Division of Cancer Prevention (P.F.P.) and Center for Biomedical Informatics & Information Technology (K.F.), National Cancer Institute, National Institutes of Health, Bethesda, Md; British Columbia Cancer Agency and the University of British Columbia, Vancouver, Canada (S.C.L.); and Fraunhofer MEVIS, Bremen, Germany (B.v.G.)
| | - Pim A de Jong
- Department of Radiology, Nuclear Medicine and Anatomy, Radboud University Medical Center, Geert Grooteplein 10, 6525 GA, Nijmegen, the Netherlands (C.J., A.A.A.S., E.T.S., P.K.G., H.B., M.B., B.G., S.S., B.v.G.); Department of Digital Technology & Innovation, Siemens Healthineers, Erlangen, Germany (A.A.A.S.); Department of Radiology, University Medical Center Utrecht, Utrecht, the Netherlands (F.A.M.H., P.A.d.J.); ETZ (Elisabeth-TweeSteden Ziekenhuis), Tilburg, the Netherlands (E.R.); Section of Radiology, Department of Medicine and Surgery (DiMeC), University of Parma, Parma, Italy (M.S.); Department of Radiology, Meander Medical Center, Amersfoort, the Netherlands (K.C., S.S.); Department of Radiology, AZ Zeno, Knokke-Heist, Belgium (J.M.); Department of Imaging, Royal Brompton Hospital, London, England (A.D.); Division of Cancer Prevention (P.F.P.) and Center for Biomedical Informatics & Information Technology (K.F.), National Cancer Institute, National Institutes of Health, Bethesda, Md; British Columbia Cancer Agency and the University of British Columbia, Vancouver, Canada (S.C.L.); and Fraunhofer MEVIS, Bremen, Germany (B.v.G.)
| | - Mario Silva
- Department of Radiology, Nuclear Medicine and Anatomy, Radboud University Medical Center, Geert Grooteplein 10, 6525 GA, Nijmegen, the Netherlands (C.J., A.A.A.S., E.T.S., P.K.G., H.B., M.B., B.G., S.S., B.v.G.); Department of Digital Technology & Innovation, Siemens Healthineers, Erlangen, Germany (A.A.A.S.); Department of Radiology, University Medical Center Utrecht, Utrecht, the Netherlands (F.A.M.H., P.A.d.J.); ETZ (Elisabeth-TweeSteden Ziekenhuis), Tilburg, the Netherlands (E.R.); Section of Radiology, Department of Medicine and Surgery (DiMeC), University of Parma, Parma, Italy (M.S.); Department of Radiology, Meander Medical Center, Amersfoort, the Netherlands (K.C., S.S.); Department of Radiology, AZ Zeno, Knokke-Heist, Belgium (J.M.); Department of Imaging, Royal Brompton Hospital, London, England (A.D.); Division of Cancer Prevention (P.F.P.) and Center for Biomedical Informatics & Information Technology (K.F.), National Cancer Institute, National Institutes of Health, Bethesda, Md; British Columbia Cancer Agency and the University of British Columbia, Vancouver, Canada (S.C.L.); and Fraunhofer MEVIS, Bremen, Germany (B.v.G.)
| | - Bram Geurts
- Department of Radiology, Nuclear Medicine and Anatomy, Radboud University Medical Center, Geert Grooteplein 10, 6525 GA, Nijmegen, the Netherlands (C.J., A.A.A.S., E.T.S., P.K.G., H.B., M.B., B.G., S.S., B.v.G.); Department of Digital Technology & Innovation, Siemens Healthineers, Erlangen, Germany (A.A.A.S.); Department of Radiology, University Medical Center Utrecht, Utrecht, the Netherlands (F.A.M.H., P.A.d.J.); ETZ (Elisabeth-TweeSteden Ziekenhuis), Tilburg, the Netherlands (E.R.); Section of Radiology, Department of Medicine and Surgery (DiMeC), University of Parma, Parma, Italy (M.S.); Department of Radiology, Meander Medical Center, Amersfoort, the Netherlands (K.C., S.S.); Department of Radiology, AZ Zeno, Knokke-Heist, Belgium (J.M.); Department of Imaging, Royal Brompton Hospital, London, England (A.D.); Division of Cancer Prevention (P.F.P.) and Center for Biomedical Informatics & Information Technology (K.F.), National Cancer Institute, National Institutes of Health, Bethesda, Md; British Columbia Cancer Agency and the University of British Columbia, Vancouver, Canada (S.C.L.); and Fraunhofer MEVIS, Bremen, Germany (B.v.G.)
| | - Kaman Chung
- Department of Radiology, Nuclear Medicine and Anatomy, Radboud University Medical Center, Geert Grooteplein 10, 6525 GA, Nijmegen, the Netherlands (C.J., A.A.A.S., E.T.S., P.K.G., H.B., M.B., B.G., S.S., B.v.G.); Department of Digital Technology & Innovation, Siemens Healthineers, Erlangen, Germany (A.A.A.S.); Department of Radiology, University Medical Center Utrecht, Utrecht, the Netherlands (F.A.M.H., P.A.d.J.); ETZ (Elisabeth-TweeSteden Ziekenhuis), Tilburg, the Netherlands (E.R.); Section of Radiology, Department of Medicine and Surgery (DiMeC), University of Parma, Parma, Italy (M.S.); Department of Radiology, Meander Medical Center, Amersfoort, the Netherlands (K.C., S.S.); Department of Radiology, AZ Zeno, Knokke-Heist, Belgium (J.M.); Department of Imaging, Royal Brompton Hospital, London, England (A.D.); Division of Cancer Prevention (P.F.P.) and Center for Biomedical Informatics & Information Technology (K.F.), National Cancer Institute, National Institutes of Health, Bethesda, Md; British Columbia Cancer Agency and the University of British Columbia, Vancouver, Canada (S.C.L.); and Fraunhofer MEVIS, Bremen, Germany (B.v.G.)
| | - Steven Schalekamp
- Department of Radiology, Nuclear Medicine and Anatomy, Radboud University Medical Center, Geert Grooteplein 10, 6525 GA, Nijmegen, the Netherlands (C.J., A.A.A.S., E.T.S., P.K.G., H.B., M.B., B.G., S.S., B.v.G.); Department of Digital Technology & Innovation, Siemens Healthineers, Erlangen, Germany (A.A.A.S.); Department of Radiology, University Medical Center Utrecht, Utrecht, the Netherlands (F.A.M.H., P.A.d.J.); ETZ (Elisabeth-TweeSteden Ziekenhuis), Tilburg, the Netherlands (E.R.); Section of Radiology, Department of Medicine and Surgery (DiMeC), University of Parma, Parma, Italy (M.S.); Department of Radiology, Meander Medical Center, Amersfoort, the Netherlands (K.C., S.S.); Department of Radiology, AZ Zeno, Knokke-Heist, Belgium (J.M.); Department of Imaging, Royal Brompton Hospital, London, England (A.D.); Division of Cancer Prevention (P.F.P.) and Center for Biomedical Informatics & Information Technology (K.F.), National Cancer Institute, National Institutes of Health, Bethesda, Md; British Columbia Cancer Agency and the University of British Columbia, Vancouver, Canada (S.C.L.); and Fraunhofer MEVIS, Bremen, Germany (B.v.G.)
| | - Joke Meersschaert
- Department of Radiology, Nuclear Medicine and Anatomy, Radboud University Medical Center, Geert Grooteplein 10, 6525 GA, Nijmegen, the Netherlands (C.J., A.A.A.S., E.T.S., P.K.G., H.B., M.B., B.G., S.S., B.v.G.); Department of Digital Technology & Innovation, Siemens Healthineers, Erlangen, Germany (A.A.A.S.); Department of Radiology, University Medical Center Utrecht, Utrecht, the Netherlands (F.A.M.H., P.A.d.J.); ETZ (Elisabeth-TweeSteden Ziekenhuis), Tilburg, the Netherlands (E.R.); Section of Radiology, Department of Medicine and Surgery (DiMeC), University of Parma, Parma, Italy (M.S.); Department of Radiology, Meander Medical Center, Amersfoort, the Netherlands (K.C., S.S.); Department of Radiology, AZ Zeno, Knokke-Heist, Belgium (J.M.); Department of Imaging, Royal Brompton Hospital, London, England (A.D.); Division of Cancer Prevention (P.F.P.) and Center for Biomedical Informatics & Information Technology (K.F.), National Cancer Institute, National Institutes of Health, Bethesda, Md; British Columbia Cancer Agency and the University of British Columbia, Vancouver, Canada (S.C.L.); and Fraunhofer MEVIS, Bremen, Germany (B.v.G.)
| | - Anand Devaraj
- Department of Radiology, Nuclear Medicine and Anatomy, Radboud University Medical Center, Geert Grooteplein 10, 6525 GA, Nijmegen, the Netherlands (C.J., A.A.A.S., E.T.S., P.K.G., H.B., M.B., B.G., S.S., B.v.G.); Department of Digital Technology & Innovation, Siemens Healthineers, Erlangen, Germany (A.A.A.S.); Department of Radiology, University Medical Center Utrecht, Utrecht, the Netherlands (F.A.M.H., P.A.d.J.); ETZ (Elisabeth-TweeSteden Ziekenhuis), Tilburg, the Netherlands (E.R.); Section of Radiology, Department of Medicine and Surgery (DiMeC), University of Parma, Parma, Italy (M.S.); Department of Radiology, Meander Medical Center, Amersfoort, the Netherlands (K.C., S.S.); Department of Radiology, AZ Zeno, Knokke-Heist, Belgium (J.M.); Department of Imaging, Royal Brompton Hospital, London, England (A.D.); Division of Cancer Prevention (P.F.P.) and Center for Biomedical Informatics & Information Technology (K.F.), National Cancer Institute, National Institutes of Health, Bethesda, Md; British Columbia Cancer Agency and the University of British Columbia, Vancouver, Canada (S.C.L.); and Fraunhofer MEVIS, Bremen, Germany (B.v.G.)
| | - Paul F Pinsky
- Department of Radiology, Nuclear Medicine and Anatomy, Radboud University Medical Center, Geert Grooteplein 10, 6525 GA, Nijmegen, the Netherlands (C.J., A.A.A.S., E.T.S., P.K.G., H.B., M.B., B.G., S.S., B.v.G.); Department of Digital Technology & Innovation, Siemens Healthineers, Erlangen, Germany (A.A.A.S.); Department of Radiology, University Medical Center Utrecht, Utrecht, the Netherlands (F.A.M.H., P.A.d.J.); ETZ (Elisabeth-TweeSteden Ziekenhuis), Tilburg, the Netherlands (E.R.); Section of Radiology, Department of Medicine and Surgery (DiMeC), University of Parma, Parma, Italy (M.S.); Department of Radiology, Meander Medical Center, Amersfoort, the Netherlands (K.C., S.S.); Department of Radiology, AZ Zeno, Knokke-Heist, Belgium (J.M.); Department of Imaging, Royal Brompton Hospital, London, England (A.D.); Division of Cancer Prevention (P.F.P.) and Center for Biomedical Informatics & Information Technology (K.F.), National Cancer Institute, National Institutes of Health, Bethesda, Md; British Columbia Cancer Agency and the University of British Columbia, Vancouver, Canada (S.C.L.); and Fraunhofer MEVIS, Bremen, Germany (B.v.G.)
| | - Stephen C Lam
- Department of Radiology, Nuclear Medicine and Anatomy, Radboud University Medical Center, Geert Grooteplein 10, 6525 GA, Nijmegen, the Netherlands (C.J., A.A.A.S., E.T.S., P.K.G., H.B., M.B., B.G., S.S., B.v.G.); Department of Digital Technology & Innovation, Siemens Healthineers, Erlangen, Germany (A.A.A.S.); Department of Radiology, University Medical Center Utrecht, Utrecht, the Netherlands (F.A.M.H., P.A.d.J.); ETZ (Elisabeth-TweeSteden Ziekenhuis), Tilburg, the Netherlands (E.R.); Section of Radiology, Department of Medicine and Surgery (DiMeC), University of Parma, Parma, Italy (M.S.); Department of Radiology, Meander Medical Center, Amersfoort, the Netherlands (K.C., S.S.); Department of Radiology, AZ Zeno, Knokke-Heist, Belgium (J.M.); Department of Imaging, Royal Brompton Hospital, London, England (A.D.); Division of Cancer Prevention (P.F.P.) and Center for Biomedical Informatics & Information Technology (K.F.), National Cancer Institute, National Institutes of Health, Bethesda, Md; British Columbia Cancer Agency and the University of British Columbia, Vancouver, Canada (S.C.L.); and Fraunhofer MEVIS, Bremen, Germany (B.v.G.)
| | - Bram van Ginneken
- Department of Radiology, Nuclear Medicine and Anatomy, Radboud University Medical Center, Geert Grooteplein 10, 6525 GA, Nijmegen, the Netherlands (C.J., A.A.A.S., E.T.S., P.K.G., H.B., M.B., B.G., S.S., B.v.G.); Department of Digital Technology & Innovation, Siemens Healthineers, Erlangen, Germany (A.A.A.S.); Department of Radiology, University Medical Center Utrecht, Utrecht, the Netherlands (F.A.M.H., P.A.d.J.); ETZ (Elisabeth-TweeSteden Ziekenhuis), Tilburg, the Netherlands (E.R.); Section of Radiology, Department of Medicine and Surgery (DiMeC), University of Parma, Parma, Italy (M.S.); Department of Radiology, Meander Medical Center, Amersfoort, the Netherlands (K.C., S.S.); Department of Radiology, AZ Zeno, Knokke-Heist, Belgium (J.M.); Department of Imaging, Royal Brompton Hospital, London, England (A.D.); Division of Cancer Prevention (P.F.P.) and Center for Biomedical Informatics & Information Technology (K.F.), National Cancer Institute, National Institutes of Health, Bethesda, Md; British Columbia Cancer Agency and the University of British Columbia, Vancouver, Canada (S.C.L.); and Fraunhofer MEVIS, Bremen, Germany (B.v.G.)
| | - Keyvan Farahani
- Department of Radiology, Nuclear Medicine and Anatomy, Radboud University Medical Center, Geert Grooteplein 10, 6525 GA, Nijmegen, the Netherlands (C.J., A.A.A.S., E.T.S., P.K.G., H.B., M.B., B.G., S.S., B.v.G.); Department of Digital Technology & Innovation, Siemens Healthineers, Erlangen, Germany (A.A.A.S.); Department of Radiology, University Medical Center Utrecht, Utrecht, the Netherlands (F.A.M.H., P.A.d.J.); ETZ (Elisabeth-TweeSteden Ziekenhuis), Tilburg, the Netherlands (E.R.); Section of Radiology, Department of Medicine and Surgery (DiMeC), University of Parma, Parma, Italy (M.S.); Department of Radiology, Meander Medical Center, Amersfoort, the Netherlands (K.C., S.S.); Department of Radiology, AZ Zeno, Knokke-Heist, Belgium (J.M.); Department of Imaging, Royal Brompton Hospital, London, England (A.D.); Division of Cancer Prevention (P.F.P.) and Center for Biomedical Informatics & Information Technology (K.F.), National Cancer Institute, National Institutes of Health, Bethesda, Md; British Columbia Cancer Agency and the University of British Columbia, Vancouver, Canada (S.C.L.); and Fraunhofer MEVIS, Bremen, Germany (B.v.G.)
| |
Collapse
|
20
|
Balagurunathan Y, Beers A, McNitt-Gray M, Hadjiiski L, Napel S, Goldgof D, Perez G, Arbelaez P, Mehrtash A, Kapur T, Yang E, Moon JW, Bernardino G, Delgado-Gonzalo R, Farhangi MM, Amini AA, Ni R, Feng X, Bagari A, Vaidhya K, Veasey B, Safta W, Frigui H, Enguehard J, Gholipour A, Castillo LS, Daza LA, Pinsky P, Kalpathy-Cramer J, Farahani K. Lung Nodule Malignancy Prediction in Sequential CT Scans: Summary of ISBI 2018 Challenge. IEEE Trans Med Imaging 2021; 40:3748-3761. [PMID: 34264825 PMCID: PMC9531053 DOI: 10.1109/tmi.2021.3097665] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
Lung cancer is by far the leading cause of cancer death in the US. Recent studies have demonstrated the effectiveness of screening using low dose CT (LDCT) in reducing lung cancer related mortality. While lung nodules are detected with a high rate of sensitivity, this exam has a low specificity rate and it is still difficult to separate benign and malignant lesions. The ISBI 2018 Lung Nodule Malignancy Prediction Challenge, developed by a team from the Quantitative Imaging Network of the National Cancer Institute, was focused on the prediction of lung nodule malignancy from two sequential LDCT screening exams using automated (non-manual) algorithms. We curated a cohort of 100 subjects who participated in the National Lung Screening Trial and had established pathological diagnoses. Data from 30 subjects were randomly selected for training and the remaining was used for testing. Participants were evaluated based on the area under the receiver operating characteristic curve (AUC) of nodule-wise malignancy scores generated by their algorithms on the test set. The challenge had 17 participants, with 11 teams submitting reports with method description, mandated by the challenge rules. Participants used quantitative methods, resulting in a reporting test AUC ranging from 0.698 to 0.913. The top five contestants used deep learning approaches, reporting an AUC between 0.87 - 0.91. The team's predictor did not achieve significant differences from each other nor from a volume change estimate (p =.05 with Bonferroni-Holm's correction).
Collapse
Affiliation(s)
| | | | | | | | - Sandy Napel
- Dept. of Radiology, School of Medicine, Stanford University (SU), CA
| | | | - Gustavo Perez
- Biomedical computer vision lab (BCV), Universidad de los Andes, Colombia
| | - Pablo Arbelaez
- Biomedical computer vision lab (BCV), Universidad de los Andes, Colombia
| | - Alireza Mehrtash
- Robotics and Control Laboratory (RCL), Department of Electrical and Computer Engineering, University of British Columbia, Vancouver, BC
- Surgical Planning Laboratory (SPL), Radiology Department, Brigham and Women’s Hospital, Boston, MA, 02130
| | - Tina Kapur
- Surgical Planning Laboratory (SPL), Radiology Department, Brigham and Women’s Hospital, Boston, MA, 02130
| | - Ehwa Yang
- Sungkyunkwan University School of Medicine, Seoul 06351, Korea
| | - Jung Won Moon
- Human Medical Imaging & Intervention Center, Seoul 06524, Korea
| | - Gabriel Bernardino
- Centre Suisse d’Électronique et de Microtechnique, Neuchâtel, Switzerland
| | | | - M. Mehdi Farhangi
- Medical Imaging Laboratory, University of Louisville, Louisville, KY. USA
- Computer Engineering and Computer Science, University of Louisville
| | - Amir A. Amini
- Medical Imaging Laboratory, University of Louisville, Louisville, KY. USA
- Electrical and Computer Engineering Department, University of Louisville, Louisville, KY. USA
| | | | - Xue Feng
- Spingbok Inc
- Department of Biomedical Engineering, University of Virginia, Charlottesville
| | | | | | - Benjamin Veasey
- Medical Imaging Laboratory, University of Louisville, Louisville, KY. USA
- Electrical and Computer Engineering Department, University of Louisville, Louisville, KY. USA
| | - Wiem Safta
- Computer Engineering and Computer Science, University of Louisville
| | - Hichem Frigui
- Computer Engineering and Computer Science, University of Louisville
| | - Joseph Enguehard
- Department of Radiology, Boston Children’s Hospital, and Harvard Medical School
| | - Ali Gholipour
- Department of Radiology, Boston Children’s Hospital, and Harvard Medical School
| | | | - Laura Alexandra Daza
- Department of Biomedical Engineering, Universidad de los Andes, Bogota, Colombia
| | - Paul Pinsky
- Divsion of Cancer Prevention, National Cancer Institute (NCI), Washington DC
| | | | - Keyvan Farahani
- Center for Biomedical Informatics and Information Technology, National Cancer Institute (NCI), Washington DC
| |
Collapse
|
21
|
Rutherford M, Mun SK, Levine B, Bennett W, Smith K, Farmer P, Jarosz Q, Wagner U, Freyman J, Blake G, Tarbox L, Farahani K, Prior F. A DICOM dataset for evaluation of medical image de-identification. Sci Data 2021; 8:183. [PMID: 34272388 PMCID: PMC8285420 DOI: 10.1038/s41597-021-00967-y] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2021] [Accepted: 06/07/2021] [Indexed: 11/23/2022] Open
Abstract
We developed a DICOM dataset that can be used to evaluate the performance of de-identification algorithms. DICOM objects (a total of 1,693 CT, MRI, PET, and digital X-ray images) were selected from datasets published in the Cancer Imaging Archive (TCIA). Synthetic Protected Health Information (PHI) was generated and inserted into selected DICOM Attributes to mimic typical clinical imaging exams. The DICOM Standard and TCIA curation audit logs guided the insertion of synthetic PHI into standard and non-standard DICOM data elements. A TCIA curation team tested the utility of the evaluation dataset. With this publication, the evaluation dataset (containing synthetic PHI) and de-identified evaluation dataset (the result of TCIA curation) are released on TCIA in advance of a competition, sponsored by the National Cancer Institute (NCI), for algorithmic de-identification of medical image datasets. The competition will use a much larger evaluation dataset constructed in the same manner. This paper describes the creation of the evaluation datasets and guidelines for their use. Measurement(s) | Deidentification • Clinical Data | Technology Type(s) | data synthesis • digital curation | Factor Type(s) | imaging type | Sample Characteristic - Organism | Homo sapiens |
Machine-accessible metadata file describing the reported data: 10.6084/m9.figshare.14802774
Collapse
Affiliation(s)
- Michael Rutherford
- Department of Biomedical Informatics, University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA
| | - Seong K Mun
- Arlington Innovation Center: Health Research, Virginia Tech, Arlington, Virginia, USA
| | - Betty Levine
- Department of Biomedical Informatics, University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA
| | - William Bennett
- Department of Biomedical Informatics, University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA
| | - Kirk Smith
- Department of Biomedical Informatics, University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA
| | - Phil Farmer
- Department of Biomedical Informatics, University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA
| | - Quasar Jarosz
- Department of Biomedical Informatics, University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA
| | - Ulrike Wagner
- Frederick National Laboratory for Cancer Research, Frederick, Maryland, USA
| | - John Freyman
- Frederick National Laboratory for Cancer Research, Frederick, Maryland, USA
| | - Geri Blake
- Department of Biomedical Informatics, University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA
| | - Lawrence Tarbox
- Department of Biomedical Informatics, University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA
| | - Keyvan Farahani
- Center for Biomedical Informatics and Information Technology, National Cancer Institute, Bethesda, Maryland, USA
| | - Fred Prior
- Department of Biomedical Informatics, University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA. .,Department of Radiology, University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA.
| |
Collapse
|
22
|
Payne A, Chopra R, Ellens N, Chen L, Ghanouni P, Sammet S, Diederich C, Ter Haar G, Parker D, Moonen C, Stafford J, Moros E, Schlesinger D, Benedict S, Wear K, Partanen A, Farahani K. AAPM Task Group 241: A medical physicist's guide to MRI-guided focused ultrasound body systems. Med Phys 2021; 48:e772-e806. [PMID: 34224149 DOI: 10.1002/mp.15076] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2020] [Revised: 04/28/2021] [Accepted: 06/21/2021] [Indexed: 11/07/2022] Open
Abstract
Magnetic resonance-guided focused ultrasound (MRgFUS) is a completely non-invasive technology that has been approved by FDA to treat several diseases. This report, prepared by the American Association of Physicist in Medicine (AAPM) Task Group 241, provides background on MRgFUS technology with a focus on clinical body MRgFUS systems. The report addresses the issues of interest to the medical physics community, specific to the body MRgFUS system configuration, and provides recommendations on how to successfully implement and maintain a clinical MRgFUS program. The following sections describe the key features of typical MRgFUS systems and clinical workflow and provide key points and best practices for the medical physicist. Commonly used terms, metrics and physics are defined and sources of uncertainty that affect MRgFUS procedures are described. Finally, safety and quality assurance procedures are explained, the recommended role of the medical physicist in MRgFUS procedures is described, and regulatory requirements for planning clinical trials are detailed. Although this report is limited in scope to clinical body MRgFUS systems that are approved or currently undergoing clinical trials in the United States, much of the material presented is also applicable to systems designed for other applications.
Collapse
Affiliation(s)
- Allison Payne
- Department of Radiology and Imaging Sciences, University of Utah, Salt Lake City, UT, USA
| | - Rajiv Chopra
- Department of Radiology, UT Southwestern Medical Center, Dallas, TX, USA
| | | | - Lili Chen
- Department of Radiation Oncology, Fox Chase Cancer Center, Philadelphia, PA, USA
| | - Pejman Ghanouni
- Department of Radiology, Stanford University, Stanford, CA, USA
| | - Steffen Sammet
- Department of Radiology, University of Chicago, Chicago, IL, USA
| | - Chris Diederich
- Department of Radiation Oncology, University of California San Francisco, San Francisco, CA, USA
| | | | - Dennis Parker
- Department of Radiology and Imaging Sciences, University of Utah, Salt Lake City, UT, USA
| | - Chrit Moonen
- Imaging Division, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Jason Stafford
- Department of Imaging Physics, MD Anderson Cancer Center, Houston, TX, USA
| | - Eduardo Moros
- Department of Radiation Oncology, Moffitt Cancer Center, Tampa, FL, USA
| | - David Schlesinger
- Department of Radiation Oncology, University of Virginia, Charlottesville, VA, USA
| | | | - Keith Wear
- U.S. Food and Drug Administration, Silver Spring, MD, USA
| | | | - Keyvan Farahani
- National Cancer Institute, National Institutes of Health, Rockville, MD, USA
| |
Collapse
|
23
|
McNitt-Gray M, Napel S, Jaggi A, Mattonen SA, Hadjiiski L, Muzi M, Goldgof D, Balagurunathan Y, Pierce LA, Kinahan PE, Jones EF, Nguyen A, Virkud A, Chan HP, Emaminejad N, Wahi-Anwar M, Daly M, Abdalah M, Yang H, Lu L, Lv W, Rahmim A, Gastounioti A, Pati S, Bakas S, Kontos D, Zhao B, Kalpathy-Cramer J, Farahani K. Standardization in Quantitative Imaging: A Multicenter Comparison of Radiomic Features from Different Software Packages on Digital Reference Objects and Patient Data Sets. ACTA ACUST UNITED AC 2021; 6:118-128. [PMID: 32548288 PMCID: PMC7289262 DOI: 10.18383/j.tom.2019.00031] [Citation(s) in RCA: 53] [Impact Index Per Article: 17.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
Abstract
Radiomic features are being increasingly studied for clinical applications. We aimed to assess the agreement among radiomic features when computed by several groups by using different software packages under very tightly controlled conditions, which included standardized feature definitions and common image data sets. Ten sites (9 from the NCI's Quantitative Imaging Network] positron emission tomography–computed tomography working group plus one site from outside that group) participated in this project. Nine common quantitative imaging features were selected for comparison including features that describe morphology, intensity, shape, and texture. The common image data sets were: three 3D digital reference objects (DROs) and 10 patient image scans from the Lung Image Database Consortium data set using a specific lesion in each scan. Each object (DRO or lesion) was accompanied by an already-defined volume of interest, from which the features were calculated. Feature values for each object (DRO or lesion) were reported. The coefficient of variation (CV), expressed as a percentage, was calculated across software packages for each feature on each object. Thirteen sets of results were obtained for the DROs and patient data sets. Five of the 9 features showed excellent agreement with CV < 1%; 1 feature had moderate agreement (CV < 10%), and 3 features had larger variations (CV ≥ 10%) even after attempts at harmonization of feature calculations. This work highlights the value of feature definition standardization as well as the need to further clarify definitions for some features.
Collapse
Affiliation(s)
- M McNitt-Gray
- David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA
| | - S Napel
- Stanford University School of Medicine, Stanford, CA
| | - A Jaggi
- Stanford University School of Medicine, Stanford, CA
| | - S A Mattonen
- Stanford University School of Medicine, Stanford, CA.,The University of Western Ontario, Canada
| | | | - M Muzi
- University of Washington, Seattle, WA
| | - D Goldgof
- University of South Florida, Tampa, FL
| | | | | | | | - E F Jones
- UC San Francisco, School of Medicine, San Francisco, CA
| | - A Nguyen
- UC San Francisco, School of Medicine, San Francisco, CA
| | - A Virkud
- University of Michigan, Ann Arbor, MI
| | - H P Chan
- University of Michigan, Ann Arbor, MI
| | - N Emaminejad
- David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA
| | - M Wahi-Anwar
- David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA
| | - M Daly
- David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA
| | - M Abdalah
- H. Lee Moffitt Cancer Center, Tampa, FL
| | - H Yang
- Columbia University Medical Center, New York, NY
| | - L Lu
- Columbia University Medical Center, New York, NY
| | - W Lv
- BC Cancer Research Centre, Vancouver, BC, Canada
| | - A Rahmim
- BC Cancer Research Centre, Vancouver, BC, Canada
| | - A Gastounioti
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA
| | - S Pati
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA
| | - S Bakas
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA
| | - D Kontos
- Center for Biomedical Image Computing and Analytics (CBICA), University of Pennsylvania, Philadelphia, PA
| | - B Zhao
- Columbia University Medical Center, New York, NY
| | | | | |
Collapse
|
24
|
Fedorov A, Longabaugh WJR, Pot D, Clunie DA, Pieper S, Aerts HJWL, Homeyer A, Lewis R, Akbarzadeh A, Bontempi D, Clifford W, Herrmann MD, Höfener H, Octaviano I, Osborne C, Paquette S, Petts J, Punzo D, Reyes M, Schacherer DP, Tian M, White G, Ziegler E, Shmulevich I, Pihl T, Wagner U, Farahani K, Kikinis R. NCI Imaging Data Commons. Cancer Res 2021; 81:4188-4193. [PMID: 34185678 PMCID: PMC8373794 DOI: 10.1158/0008-5472.can-21-0950] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2021] [Revised: 05/25/2021] [Accepted: 06/14/2021] [Indexed: 11/16/2022]
Abstract
The National Cancer Institute (NCI) Cancer Research Data Commons (CRDC) aims to establish a national cloud-based data science infrastructure. Imaging Data Commons (IDC) is a new component of CRDC supported by the Cancer Moonshot. The goal of IDC is to enable a broad spectrum of cancer researchers, with and without imaging expertise, to easily access and explore the value of deidentified imaging data and to support integrated analyses with nonimaging data. We achieve this goal by colocating versatile imaging collections with cloud-based computing resources and data exploration, visualization, and analysis tools. The IDC pilot was released in October 2020 and is being continuously populated with radiology and histopathology collections. IDC provides access to curated imaging collections, accompanied by documentation, a user forum, and a growing number of analysis use cases that aim to demonstrate the value of a data commons framework applied to cancer imaging research. SIGNIFICANCE: This study introduces NCI Imaging Data Commons, a new repository of the NCI Cancer Research Data Commons, which will support cancer imaging research on the cloud.
Collapse
Affiliation(s)
- Andrey Fedorov
- Brigham and Women's Hospital, Department of Radiology, Harvard Medical School, Boston, Massachusetts.
| | | | - David Pot
- General Dynamics, Bethesda, Maryland
| | | | | | - Hugo J W L Aerts
- Artificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, Massachusetts.,Departments of Radiation Oncology & Radiology, Brigham and Women's Hospital, Dana-Farber Cancer Institute, Harvard Medical School, Boston, Massachusetts.,Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands
| | | | - Rob Lewis
- Radical Imaging, Boston, Massachusetts
| | - Afshin Akbarzadeh
- Brigham and Women's Hospital, Department of Radiology, Harvard Medical School, Boston, Massachusetts
| | - Dennis Bontempi
- Artificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, Massachusetts
| | | | - Markus D Herrmann
- Massachusetts General Hospital, Department of Radiology, Harvard Medical School, Boston, Massachusetts
| | | | | | | | | | | | | | | | | | - Mi Tian
- Institute for Systems Biology, Seattle, Washington
| | - George White
- Institute for Systems Biology, Seattle, Washington
| | | | | | - Todd Pihl
- Frederick National Laboratory for Cancer Research, Frederick, Maryland
| | - Ulrike Wagner
- Frederick National Laboratory for Cancer Research, Frederick, Maryland
| | | | - Ron Kikinis
- Brigham and Women's Hospital, Department of Radiology, Harvard Medical School, Boston, Massachusetts
| |
Collapse
|
25
|
Kirby J, Prior F, Petrick N, Hadjiski L, Farahani K, Drukker K, Kalpathy-Cramer J, Glide-Hurst C, El Naqa I. Introduction to special issue on datasets hosted in The Cancer Imaging Archive (TCIA). Med Phys 2021; 47:6026-6028. [PMID: 33202038 DOI: 10.1002/mp.14595] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2020] [Revised: 11/03/2020] [Accepted: 11/09/2020] [Indexed: 01/19/2023] Open
Affiliation(s)
- Justin Kirby
- Frederick National Laboratory for Cancer Research, Cancer Imaging Informatics Lab, National Institute of Health, Frederick, MD, USA
| | - Fred Prior
- Department of Biomedical Informatics, University of Arkansas for Medical Sciences, Little Rock, AR, USA
| | - Nicholas Petrick
- Center for Devices and Radiological Health, U.S. Food and Drug Administration, Silver Spring, MD, USA
| | - Lubomir Hadjiski
- Department of Radiology, University of Michigan, Ann Arbor, MI, USA
| | - Keyvan Farahani
- Center for Biomedical Informatics and Information Technology, National Cancer Institute, Bethesda, MD, USA
| | | | | | - Carri Glide-Hurst
- Department of Radiation Oncology, University of Wisconsin, Madison, WI, USA
| | - Issam El Naqa
- Department of Machine Learning, Moffitt Cancer Center, Tampa, FL, USA
| |
Collapse
|
26
|
Wang JH, Wahid KA, van Dijk LV, Farahani K, Thompson RF, Fuller CD. Radiomic biomarkers of tumor immune biology and immunotherapy response. Clin Transl Radiat Oncol 2021; 28:97-115. [PMID: 33937530 PMCID: PMC8076712 DOI: 10.1016/j.ctro.2021.03.006] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/21/2020] [Revised: 03/20/2021] [Accepted: 03/24/2021] [Indexed: 02/08/2023] Open
Abstract
Immunotherapies are leading to improved outcomes for many cancers, including those with devastating prognoses. As therapies like immune checkpoint inhibitors (ICI) become a mainstay in treatment regimens, many concurrent challenges have arisen - for instance, delineating clinical responders from non-responders. Predicting response has proven to be difficult given a lack of consistent and accurate biomarkers, heterogeneity of the tumor microenvironment (TME), and a poor understanding of resistance mechanisms. For the most part, imaging data have remained an untapped, yet abundant, resource to address these challenges. In recent years, quantitative image analyses have highlighted the utility of medical imaging in predicting tumor phenotypes, prognosis, and therapeutic response. These studies have been fueled by an explosion of resources in high-throughput mining of image features (i.e. radiomics) and artificial intelligence. In this review, we highlight current progress in radiomics to understand tumor immune biology and predict clinical responses to immunotherapies. We also discuss limitations in these studies and future directions for the field, particularly if high-dimensional imaging data are to play a larger role in precision medicine.
Collapse
Affiliation(s)
- Jarey H. Wang
- Medical Scientist Training Program, Baylor College of Medicine, Houston, TX, United States
| | - Kareem A. Wahid
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Lisanne V. van Dijk
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Keyvan Farahani
- Center for Biomedical Informatics and Information Technology, National Cancer Institute, Bethesda, MD, United States
| | - Reid F. Thompson
- Department of Radiation Medicine, Oregon Health & Science University, Portland, OR, United States
| | - Clifton David Fuller
- Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX, United States
| |
Collapse
|
27
|
Scarpelli M, Whelan B, Farahani K. Domain classification and analysis of national institutes of health-funded medical physics research. Med Phys 2021; 48:605-614. [PMID: 32970862 DOI: 10.1002/mp.14469] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2020] [Revised: 07/24/2020] [Accepted: 08/22/2020] [Indexed: 11/08/2022] Open
Abstract
PURPOSE The American Association of Physicists in Medicine (AAPM) previously developed a research database consisting of the National Institutes of Health (NIH) grants that were awarded to its members. The purpose of this report is to classify these NIH grants into various medical physics subdisciplines and analyze the scope of AAPM member research. METHODS For this report, an algorithm classified grant topics into medical physics research subdisciplines (grants from 2002 to 2019 were analyzed). This algorithm utilized a search for common words and phrases within grant titles, keywords, abstracts, and activity codes to perform the classification. AAPM member grants were compared with non-AAPM member grants in various relevant subcategories to assess what percentage of these grants was held by AAPM members. RESULTS The percentage of AAPM member grants that included words relating to both imaging and therapy (image-guided therapy grants) increased from 13% (27/207) in 2002 to 27% (79/293) in 2019. The percentage of AAPM member grants utilizing words relating to artificial intelligence increased from 8% in 2002 to 20% in 2019. From 2002 to 2019, AAPM member grants referenced cancer more than all other diseases combined. The majority of AAPM member grants included words relating to clinical research (81% of grants in 2002 and 99% in 2019). When comparing AAPM member with non-AAPM member grants it was found that in 2019 AAPM members held a substantial fraction of all NIH grants that referenced stereotactic radiation therapies (41%), radionuclide therapies (10%), brachytherapies (35%), intensity-modulated radiation therapies (45%), and external beam particle therapies (55%). From 2002 to 2019, the percentage of AAPM membership holding NIH grants decreased for males (3.2% down to 2.3%) and increased for females (0.8% up to 1.3%) CONCLUSIONS: The majority of grants awarded to AAPM members focus on clinical research, which underlies the translational aspect of medical physics and suggests medical physicists are uniquely positioned to help translate new technologies such as artificial intelligence into the clinic. Since 2002, NIH grants awarded to AAPM members have increasingly referenced some form of image-guided therapy, suggesting opportunities for continued innovation of imaging technologies. A substantial fraction of all radiotherapy-related research grants were awarded to AAPM members, emphasizing the important role physicists have in developing radiotherapy-related treatments.
Collapse
Affiliation(s)
- Matthew Scarpelli
- Department of Neuroimaging, Barrow Neurological Institute, Phoenix, AZ, 85012, USA
| | - Brendan Whelan
- Image X institute, University of Sydney, Eveleigh, NSW, 2015, Australia
| | - Keyvan Farahani
- Center for Biomedical Informatics and Information Technology, National Cancer Institute, Bethesda, MD, 20892, USA
| |
Collapse
|
28
|
Parasa S, Wallace M, Bagci U, Antonino M, Berzin T, Byrne M, Celik H, Farahani K, Golding M, Gross S, Jamali V, Mendonca P, Mori Y, Ninh A, Repici A, Rex D, Skrinak K, Thakkar SJ, van Hooft JE, Vargo J, Yu H, Xu Z, Sharma P. Proceedings from the First Global Artificial Intelligence in Gastroenterology and Endoscopy Summit. Gastrointest Endosc 2020; 92:938-945.e1. [PMID: 32343978 DOI: 10.1016/j.gie.2020.04.044] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/29/2020] [Accepted: 04/16/2020] [Indexed: 02/08/2023]
Abstract
BACKGROUND AND AIMS Artificial intelligence (AI), specifically deep learning, offers the potential to enhance the field of GI endoscopy in areas ranging from lesion detection and classification to quality metrics and documentation. Progress in this field will be measured by whether AI implementation can lead to improved patient outcomes and more efficient clinical workflow for GI endoscopists. The aims of this article are to report the findings of a multidisciplinary group of experts focusing on issues in AI research and applications related to gastroenterology and endoscopy, to review the current status of the field, and to produce recommendations for investigators developing and studying new AI technologies for gastroenterology. METHODS A multidisciplinary meeting was held on September 28, 2019, bringing together academic, industry, and regulatory experts in diverse fields including gastroenterology, computer and imaging sciences, machine learning, computer vision, U.S. Food and Drug Administration, and the National Institutes of Health. Recent and ongoing studies in gastroenterology and current technology in AI were presented and discussed, key gaps in knowledge were identified, and recommendations were made for research that would have the highest impact in making advances and implementation in the field of AI to gastroenterology. RESULTS There was a consensus that AI will transform the field of gastroenterology, particularly endoscopy and image interpretation. Powered by advanced machine learning algorithms, the use of computer vision in endoscopy has the potential to result in better prediction and treatment outcomes for patients with gastroenterology disorders and cancer. Large libraries of endoscopic images, "EndoNet," will be important to facilitate development and application of AI systems. The regulatory environment for implementation of AI systems is evolving, but common outcomes such as colon polyp detection have been highlighted as potential clinical trial endpoints. Other threshold outcomes will be important, as well as clarity on iterative improvement of clinical systems. CONCLUSIONS Gastroenterology is a prime candidate for early adoption of AI. AI is rapidly moving from an experimental phase to a clinical implementation phase in gastroenterology. It is anticipated that the implementation of AI in gastroenterology over the next decade will have a significant and positive impact on patient care and clinical workflows. Ongoing collaboration among gastroenterologists, industry experts, and regulatory agencies will be important to ensure that progress is rapid and clinically meaningful. However, several constraints and areas will benefit from further exploration, including potential clinical applications, implementation, structure and governance, role of gastroenterologists, and potential impact of AI in gastroenterology.
Collapse
Affiliation(s)
- Sravanthi Parasa
- Department of Gastroenterology, Swedish Medical Center, Seattle, Washington, USA
| | - Michael Wallace
- Department of Medicine, Mayo Clinic, Director, Digestive Diseases Research Program, Editor in Chief Gastrointestinal Endoscopy, President, Florida Gastroenterology Society, Jacksonville, Florida, USA
| | - Ulas Bagci
- Artificial Intelligence in Medicine (AIM), Center for Research in Computer Vision, University of Central Florida, Orlando, Florida, USA
| | - Mark Antonino
- Gastroenterology and Endoscopy Devices Team, Division of Renal, Gastrointestinal, Obesity and Transplant Devices, Office of Gastrorenal, ObGyn, General Hospital and Urology Devices, Office of Product Evaluation and Quality, Center for Devices and Radiological Health, U.S. Food and Drug Administration, Silver Spring, Maryland, USA
| | - Tyler Berzin
- Department of Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
| | - Michael Byrne
- Division of Gastroenterology, Vancouver General Hospital/University of British Columbia, Vancouver, British Columbia, Canada
| | - Haydar Celik
- Clinical Center, National Institutes of Health, Bethesda, Maryland, USA; George Washington University, Washington, DC, USA
| | - Keyvan Farahani
- Image-Guided Interventions and Imaging Informatics, National Cancer Institute, National Institutes of Health, Rockville, Maryland, USA
| | - Martin Golding
- Gastroenterology and Endoscopy Devices Team, Division of Renal, Gastrointestinal, Obesity and Transplant Devices, Office of Gastrorenal, ObGyn, General Hospital and Urology Devices, Office of Product Evaluation and Quality, Center for Devices and Radiological Health, U.S. Food and Drug Administration, Silver Spring, Maryland, USA
| | - Seth Gross
- Department of Medicine, Division of Gastroenterology, Clinical Care and Quality, NYU Langone Health, New York, New York, USA
| | - Vafa Jamali
- Respiratory, Gastrointestinal & Informatics, Medtronic Inc, Boulder, Colorado, USA
| | - Paulo Mendonca
- Digestive Disease Center, Showa University, Northern Yokohama Hospital, Yokohama, Japan
| | | | | | - Alessandro Repici
- Digestive Endoscopy Unit, Humanitas, Research Hospital, Milan, Italy
| | - Douglas Rex
- Departments of Medicine, Endoscopy, and Gastroenterology, Indiana University of School of Medicine, Indianapolis, Indiana, USA
| | - Kris Skrinak
- Global Machine Learning Segment Lead, Amazon Web Services, New York, New York, USA
| | - Shyam J Thakkar
- Department of Endoscopy, Allegheny Health Network, Department of Medicine, Temple University, Philadelphia, Pennsylvania, USA; Department of Biomedical Engineering, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA
| | | | - John Vargo
- Department of Medicine, Gastroenterology, Hepatology & Nutrition, Cleveland Clinic, Cleveland, Ohio, USA
| | - Honggang Yu
- Division of Gastroenterology, Renmin Hospital, Wuhan University, Wuhan, China
| | - Ziyue Xu
- Medical Image Analysis, NVIDIA, Bethesda, Maryland, USA
| | - Prateek Sharma
- Division of Gastroenterology and Hepatology, University of Kansas School of Medicine, Kansas City, Kansas, USA
| |
Collapse
|
29
|
Afshar P, Oikonomou A, Naderkhani F, Tyrrell PN, Plataniotis KN, Farahani K, Mohammadi A. 3D-MCN: A 3D Multi-scale Capsule Network for Lung Nodule Malignancy Prediction. Sci Rep 2020; 10:7948. [PMID: 32409715 PMCID: PMC7224210 DOI: 10.1038/s41598-020-64824-5] [Citation(s) in RCA: 19] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2020] [Accepted: 04/17/2020] [Indexed: 12/29/2022] Open
Abstract
Despite the advances in automatic lung cancer malignancy prediction, achieving high accuracy remains challenging. Existing solutions are mostly based on Convolutional Neural Networks (CNNs), which require a large amount of training data. Most of the developed CNN models are based only on the main nodule region, without considering the surrounding tissues. Obtaining high sensitivity is challenging with lung nodule malignancy prediction. Moreover, the interpretability of the proposed techniques should be a consideration when the end goal is to utilize the model in a clinical setting. Capsule networks (CapsNets) are new and revolutionary machine learning architectures proposed to overcome shortcomings of CNNs. Capitalizing on the success of CapsNet in biomedical domains, we propose a novel model for lung tumor malignancy prediction. The proposed framework, referred to as the 3D Multi-scale Capsule Network (3D-MCN), is uniquely designed to benefit from: (i) 3D inputs, providing information about the nodule in 3D; (ii) Multi-scale input, capturing the nodule's local features, as well as the characteristics of the surrounding tissues, and; (iii) CapsNet-based design, being capable of dealing with a small number of training samples. The proposed 3D-MCN architecture predicted lung nodule malignancy with a high accuracy of 93.12%, sensitivity of 94.94%, area under the curve (AUC) of 0.9641, and specificity of 90% when tested on the LIDC-IDRI dataset. When classifying patients as having a malignant condition (i.e., at least one malignant nodule is detected) or not, the proposed model achieved an accuracy of 83%, and a sensitivity and specificity of 84% and 81% respectively.
Collapse
Affiliation(s)
- Parnian Afshar
- Concordia Institute for Information Systems Engineering, Montreal, QC, Canada
| | - Anastasia Oikonomou
- Department of Medical Imaging, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, ON, Canada
| | - Farnoosh Naderkhani
- Concordia Institute for Information Systems Engineering, Montreal, QC, Canada
| | - Pascal N Tyrrell
- Department of Medical Imaging, Department of Statistical Sciences, University of Toronto, Toronto, ON, Canada
| | | | - Keyvan Farahani
- Center for Biomedical Informatics and Information Technology, National Cancer Institute (NCI), Rockville, MD, USA
| | - Arash Mohammadi
- Concordia Institute for Information Systems Engineering, Montreal, QC, Canada.
| |
Collapse
|
30
|
Armato SG, Farahani K, Zaidi H. Biomedical image analysis challenges should be considered as an academic exercise, not an instrument that will move the field forward in a real, practical way. Med Phys 2020; 47:2325-2328. [PMID: 32040865 DOI: 10.1002/mp.14081] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2020] [Revised: 02/05/2020] [Accepted: 02/05/2020] [Indexed: 11/09/2022] Open
Affiliation(s)
- Samuel G Armato
- Department of Radiology, The University of Chicago, 5841 S. Maryland Ave., Chicago, IL, 60637, USA
| | - Keyvan Farahani
- Center for Biomedical Imaging and Information Technology, National Cancer Institute, Bethesda, Maryland, USA
| | | |
Collapse
|
31
|
Kurc T, Bakas S, Ren X, Bagari A, Momeni A, Huang Y, Zhang L, Kumar A, Thibault M, Qi Q, Wang Q, Kori A, Gevaert O, Zhang Y, Shen D, Khened M, Ding X, Krishnamurthi G, Kalpathy-Cramer J, Davis J, Zhao T, Gupta R, Saltz J, Farahani K. Segmentation and Classification in Digital Pathology for Glioma Research: Challenges and Deep Learning Approaches. Front Neurosci 2020; 14:27. [PMID: 32153349 PMCID: PMC7046596 DOI: 10.3389/fnins.2020.00027] [Citation(s) in RCA: 29] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2019] [Accepted: 01/10/2020] [Indexed: 12/12/2022] Open
Abstract
Biomedical imaging Is an important source of information in cancer research. Characterizations of cancer morphology at onset, progression, and in response to treatment provide complementary information to that gleaned from genomics and clinical data. Accurate extraction and classification of both visual and latent image features Is an increasingly complex challenge due to the increased complexity and resolution of biomedical image data. In this paper, we present four deep learning-based image analysis methods from the Computational Precision Medicine (CPM) satellite event of the 21st International Medical Image Computing and Computer Assisted Intervention (MICCAI 2018) conference. One method Is a segmentation method designed to segment nuclei in whole slide tissue images (WSIs) of adult diffuse glioma cases. It achieved a Dice similarity coefficient of 0.868 with the CPM challenge datasets. Three methods are classification methods developed to categorize adult diffuse glioma cases into oligodendroglioma and astrocytoma classes using radiographic and histologic image data. These methods achieved accuracy values of 0.75, 0.80, and 0.90, measured as the ratio of the number of correct classifications to the number of total cases, with the challenge datasets. The evaluations of the four methods indicate that (1) carefully constructed deep learning algorithms are able to produce high accuracy in the analysis of biomedical image data and (2) the combination of radiographic with histologic image information improves classification performance.
Collapse
Affiliation(s)
- Tahsin Kurc
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, United States
| | - Spyridon Bakas
- Center for Biomedical Image Computing and Analytics, University of Pennsylvania, Philadelphia, PA, United States
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
- Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
| | - Xuhua Ren
- Institute for Medical Imaging Technology, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Aditya Bagari
- Department of Engineering Design, Indian Institute of Technology Madras, Chennai, India
| | - Alexandre Momeni
- Department of Medicine and Biomedical Data Science, Stanford University, Stanford, CA, United States
| | - Yue Huang
- School of Informatics, Xiamen University, Xiamen, China
| | - Lichi Zhang
- Institute for Medical Imaging Technology, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Ashish Kumar
- Department of Engineering Design, Indian Institute of Technology Madras, Chennai, India
| | - Marc Thibault
- Department of Medicine and Biomedical Data Science, Stanford University, Stanford, CA, United States
| | - Qi Qi
- School of Informatics, Xiamen University, Xiamen, China
| | - Qian Wang
- Institute for Medical Imaging Technology, School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China
| | - Avinash Kori
- Department of Engineering Design, Indian Institute of Technology Madras, Chennai, India
| | - Olivier Gevaert
- Department of Medicine and Biomedical Data Science, Stanford University, Stanford, CA, United States
| | - Yunlong Zhang
- School of Informatics, Xiamen University, Xiamen, China
| | - Dinggang Shen
- Department of Radiology and BRIC, The University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
- Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea
| | - Mahendra Khened
- Department of Engineering Design, Indian Institute of Technology Madras, Chennai, India
| | - Xinghao Ding
- School of Informatics, Xiamen University, Xiamen, China
| | | | - Jayashree Kalpathy-Cramer
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, United States
| | - James Davis
- Department of Pathology, Stony Brook University, Stony Brook, NY, United States
| | - Tianhao Zhao
- Department of Pathology, Stony Brook University, Stony Brook, NY, United States
| | - Rajarsi Gupta
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, United States
- Department of Pathology, Stony Brook University, Stony Brook, NY, United States
| | - Joel Saltz
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, United States
| | - Keyvan Farahani
- Cancer Imaging Program, National Cancer Institute, National Institutes of Health, Bethesda, MD, United States
| |
Collapse
|
32
|
Abstract
The Quantitative Imaging Network of the National Cancer Institute is in its 10th year of operation, and research teams within the network are developing and validating clinical decision support software tools to measure or predict the response of cancers to various therapies. As projects progress from development activities to validation of quantitative imaging tools and methods, it is important to evaluate the performance and clinical readiness of the tools before committing them to prospective clinical trials. A variety of tests, including special challenges and tool benchmarking, have been instituted within the network to prepare the quantitative imaging tools for service in clinical trials. This article highlights the benchmarking process and provides a current evaluation of several tools in their transition from development to validation.
Collapse
Affiliation(s)
- Keyvan Farahani
- Cancer Imaging Program, National Cancer Institute of NIH, Bethesda, MD
| | - Darrell Tata
- Cancer Imaging Program, National Cancer Institute of NIH, Bethesda, MD
| | | |
Collapse
|
33
|
Vu QD, Graham S, Kurc T, To MNN, Shaban M, Qaiser T, Koohbanani NA, Khurram SA, Kalpathy-Cramer J, Zhao T, Gupta R, Kwak JT, Rajpoot N, Saltz J, Farahani K. Methods for Segmentation and Classification of Digital Microscopy Tissue Images. Front Bioeng Biotechnol 2019; 7:53. [PMID: 31001524 PMCID: PMC6454006 DOI: 10.3389/fbioe.2019.00053] [Citation(s) in RCA: 86] [Impact Index Per Article: 17.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2018] [Accepted: 03/01/2019] [Indexed: 12/12/2022] Open
Abstract
High-resolution microscopy images of tissue specimens provide detailed information about the morphology of normal and diseased tissue. Image analysis of tissue morphology can help cancer researchers develop a better understanding of cancer biology. Segmentation of nuclei and classification of tissue images are two common tasks in tissue image analysis. Development of accurate and efficient algorithms for these tasks is a challenging problem because of the complexity of tissue morphology and tumor heterogeneity. In this paper we present two computer algorithms; one designed for segmentation of nuclei and the other for classification of whole slide tissue images. The segmentation algorithm implements a multiscale deep residual aggregation network to accurately segment nuclear material and then separate clumped nuclei into individual nuclei. The classification algorithm initially carries out patch-level classification via a deep learning method, then patch-level statistical and morphological features are used as input to a random forest regression model for whole slide image classification. The segmentation and classification algorithms were evaluated in the MICCAI 2017 Digital Pathology challenge. The segmentation algorithm achieved an accuracy score of 0.78. The classification algorithm achieved an accuracy score of 0.81. These scores were the highest in the challenge.
Collapse
Affiliation(s)
- Quoc Dang Vu
- Department of Computer Science and Engineering, Sejong University, Seoul, South Korea
| | - Simon Graham
- Department of Computer Science, University of Warwick, Coventry, United Kingdom
| | - Tahsin Kurc
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, United States
| | - Minh Nguyen Nhat To
- Department of Computer Science and Engineering, Sejong University, Seoul, South Korea
| | - Muhammad Shaban
- Department of Computer Science, University of Warwick, Coventry, United Kingdom
| | - Talha Qaiser
- Department of Computer Science, University of Warwick, Coventry, United Kingdom
| | | | - Syed Ali Khurram
- School of Clinical Dentistry, The University of Sheffield, Sheffield, United Kingdom
| | - Jayashree Kalpathy-Cramer
- Department of Radiology, Harvard Medical School and Mass General Hospital, Boston, MA, United States
| | - Tianhao Zhao
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, United States
- Department of Pathology, Stony Brook University, Stony Brook, NY, United States
| | - Rajarsi Gupta
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, United States
- Department of Pathology, Stony Brook University, Stony Brook, NY, United States
| | - Jin Tae Kwak
- Department of Computer Science and Engineering, Sejong University, Seoul, South Korea
| | - Nasir Rajpoot
- Department of Computer Science, University of Warwick, Coventry, United Kingdom
| | - Joel Saltz
- Department of Biomedical Informatics, Stony Brook University, Stony Brook, NY, United States
| | - Keyvan Farahani
- Cancer Imaging Program, National Cancer Institute, National Institutes of Health, Bethesda, MD, United States
| |
Collapse
|
34
|
Mohammadi A, Afshar P, Asif A, Farahani K, Kirby J, Oikonomou A, Plataniotis KN. Lung Cancer Radiomics: Highlights from the IEEE Video and Image Processing Cup 2018 Student Competition. IEEE Signal Process Mag 2019; 36:164-173. [PMID: 31543691 PMCID: PMC6753949 DOI: 10.1109/msp.2018.2877123] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
|
35
|
Armato SG, Huisman H, Drukker K, Hadjiiski L, Kirby JS, Petrick N, Redmond G, Giger ML, Cha K, Mamonov A, Kalpathy-Cramer J, Farahani K. PROSTATEx Challenges for computerized classification of prostate lesions from multiparametric magnetic resonance images. J Med Imaging (Bellingham) 2018; 5:044501. [PMID: 30840739 DOI: 10.1117/1.jmi.5.4.044501] [Citation(s) in RCA: 66] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2018] [Accepted: 10/10/2018] [Indexed: 12/18/2022] Open
Abstract
Grand challenges stimulate advances within the medical imaging research community; within a competitive yet friendly environment, they allow for a direct comparison of algorithms through a well-defined, centralized infrastructure. The tasks of the two-part PROSTATEx Challenges (the PROSTATEx Challenge and the PROSTATEx-2 Challenge) are (1) the computerized classification of clinically significant prostate lesions and (2) the computerized determination of Gleason Grade Group in prostate cancer, both based on multiparametric magnetic resonance images. The challenges incorporate well-vetted cases for training and testing, a centralized performance assessment process to evaluate results, and an established infrastructure for case dissemination, communication, and result submission. In the PROSTATEx Challenge, 32 groups apply their computerized methods (71 methods total) to 208 prostate lesions in the test set. The area under the receiver operating characteristic curve for these methods in the task of differentiating between lesions that are and are not clinically significant ranged from 0.45 to 0.87; statistically significant differences in performance among the top-performing methods, however, are not observed. In the PROSTATEx-2 Challenge, 21 groups apply their computerized methods (43 methods total) to 70 prostate lesions in the test set. When compared with the reference standard, the quadratic-weighted kappa values for these methods in the task of assigning a five-point Gleason Grade Group to each lesion range from - 0.24 to 0.27; superiority to random guessing can be established for only two methods. When approached with a sense of commitment and scientific rigor, challenges foster interest in the designated task and encourage innovation in the field.
Collapse
Affiliation(s)
- Samuel G Armato
- The University of Chicago, Department of Radiology, Chicago, Illinois, United States
| | - Henkjan Huisman
- Radboud University Medical Center, Department of Radiology and Nuclear Medicine, Nijmegen, The Netherlands
| | - Karen Drukker
- The University of Chicago, Department of Radiology, Chicago, Illinois, United States
| | - Lubomir Hadjiiski
- University of Michigan, Department of Radiology, Ann Arbor, Michigan, United States
| | - Justin S Kirby
- Frederick National Laboratory for Cancer Research, Cancer Imaging Program, Frederick, Maryland, United States
| | - Nicholas Petrick
- U.S. Food and Drug Administration, Center for Devices and Radiological Health, Silver Spring, Maryland, United States
| | - George Redmond
- National Cancer Institute, Cancer Imaging Program, Division of Cancer Treatment and Diagnosis, Bethesda, Maryland, United States
| | - Maryellen L Giger
- The University of Chicago, Department of Radiology, Chicago, Illinois, United States
| | - Kenny Cha
- University of Michigan, Department of Radiology, Ann Arbor, Michigan, United States.,U.S. Food and Drug Administration, Center for Devices and Radiological Health, Silver Spring, Maryland, United States
| | - Artem Mamonov
- MGH/Harvard Medical School, Boston, Massachusetts, United States
| | | | - Keyvan Farahani
- National Cancer Institute, Cancer Imaging Program, Division of Cancer Treatment and Diagnosis, Bethesda, Maryland, United States
| |
Collapse
|
36
|
Yang J, Veeraraghavan H, Armato SG, Farahani K, Kirby JS, Kalpathy-Kramer J, van Elmpt W, Dekker A, Han X, Feng X, Aljabar P, Oliveira B, van der Heyden B, Zamdborg L, Lam D, Gooding M, Sharp GC. Autosegmentation for thoracic radiation treatment planning: A grand challenge at AAPM 2017. Med Phys 2018; 45:4568-4581. [PMID: 30144101 DOI: 10.1002/mp.13141] [Citation(s) in RCA: 127] [Impact Index Per Article: 21.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/12/2018] [Revised: 08/15/2018] [Accepted: 08/15/2018] [Indexed: 12/25/2022] Open
Abstract
PURPOSE This report presents the methods and results of the Thoracic Auto-Segmentation Challenge organized at the 2017 Annual Meeting of American Association of Physicists in Medicine. The purpose of the challenge was to provide a benchmark dataset and platform for evaluating performance of autosegmentation methods of organs at risk (OARs) in thoracic CT images. METHODS Sixty thoracic CT scans provided by three different institutions were separated into 36 training, 12 offline testing, and 12 online testing scans. Eleven participants completed the offline challenge, and seven completed the online challenge. The OARs were left and right lungs, heart, esophagus, and spinal cord. Clinical contours used for treatment planning were quality checked and edited to adhere to the RTOG 1106 contouring guidelines. Algorithms were evaluated using the Dice coefficient, Hausdorff distance, and mean surface distance. A consolidated score was computed by normalizing the metrics against interrater variability and averaging over all patients and structures. RESULTS The interrater study revealed highest variability in Dice for the esophagus and spinal cord, and in surface distances for lungs and heart. Five out of seven algorithms that participated in the online challenge employed deep-learning methods. Although the top three participants using deep learning produced the best segmentation for all structures, there was no significant difference in the performance among them. The fourth place participant used a multi-atlas-based approach. The highest Dice scores were produced for lungs, with averages ranging from 0.95 to 0.98, while the lowest Dice scores were produced for esophagus, with a range of 0.55-0.72. CONCLUSION The results of the challenge showed that the lungs and heart can be segmented fairly accurately by various algorithms, while deep-learning methods performed better on the esophagus. Our dataset together with the manual contours for all training cases continues to be available publicly as an ongoing benchmarking resource.
Collapse
Affiliation(s)
- Jinzhong Yang
- Department of Radiation Physics, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | | | - Samuel G Armato
- Department of Radiology, The University of Chicago, Chicago, IL, USA
| | - Keyvan Farahani
- Cancer Imaging Program, National Cancer Institute, Bethesda, MD, USA
| | - Justin S Kirby
- Cancer Imaging Program, Frederick National Laboratory for Cancer Research sponsored by the National Cancer Institute, Frederick, MD, USA
| | | | - Wouter van Elmpt
- Department of Radiation Oncology (MAASTRO), GROW - School for Oncology and Developmental Biology, Maastricht University Medical Center, Maastricht, The Netherlands
| | - Andre Dekker
- Department of Radiation Oncology (MAASTRO), GROW - School for Oncology and Developmental Biology, Maastricht University Medical Center, Maastricht, The Netherlands
| | - Xiao Han
- Elekta Inc., Maryland Heights, MO, USA
| | - Xue Feng
- Department of Biomedical Engineering, University of Virginia, Charlottesville, VA, USA
| | | | - Bruno Oliveira
- Life and Health Sciences Research Institute (ICVS), School of Medicine, University of Minho, Braga, Portugal.,ICVS/3Bs - PT Government Associaste Laboratory, Braga/Guimares, Portugal
| | - Brent van der Heyden
- Department of Radiation Oncology (MAASTRO), GROW - School for Oncology and Developmental Biology, Maastricht University Medical Center, Maastricht, The Netherlands
| | - Leonid Zamdborg
- Department of Radiation Oncology, Beaumont Health, Royal Oak, MI, USA
| | - Dao Lam
- Department of Radiation Oncology, Washington University School of Medicine in St. Louis, St. Louis, MO, USA
| | | | | |
Collapse
|
37
|
Elhalawani H, Lin TA, Volpe S, Mohamed ASR, White AL, Zafereo J, Wong AJ, Berends JE, AboHashem S, Williams B, Aymard JM, Kanwar A, Perni S, Rock CD, Cooksey L, Campbell S, Yang P, Nguyen K, Ger RB, Cardenas CE, Fave XJ, Sansone C, Piantadosi G, Marrone S, Liu R, Huang C, Yu K, Li T, Yu Y, Zhang Y, Zhu H, Morris JS, Baladandayuthapani V, Shumway JW, Ghosh A, Pöhlmann A, Phoulady HA, Goyal V, Canahuate G, Marai GE, Vock D, Lai SY, Mackin DS, Court LE, Freymann J, Farahani K, Kaplathy-Cramer J, Fuller CD. Machine Learning Applications in Head and Neck Radiation Oncology: Lessons From Open-Source Radiomics Challenges. Front Oncol 2018; 8:294. [PMID: 30175071 PMCID: PMC6107800 DOI: 10.3389/fonc.2018.00294] [Citation(s) in RCA: 30] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2018] [Accepted: 07/16/2018] [Indexed: 12/13/2022] Open
Abstract
Radiomics leverages existing image datasets to provide non-visible data extraction via image post-processing, with the aim of identifying prognostic, and predictive imaging features at a sub-region of interest level. However, the application of radiomics is hampered by several challenges such as lack of image acquisition/analysis method standardization, impeding generalizability. As of yet, radiomics remains intriguing, but not clinically validated. We aimed to test the feasibility of a non-custom-constructed platform for disseminating existing large, standardized databases across institutions for promoting radiomics studies. Hence, University of Texas MD Anderson Cancer Center organized two public radiomics challenges in head and neck radiation oncology domain. This was done in conjunction with MICCAI 2016 satellite symposium using Kaggle-in-Class, a machine-learning and predictive analytics platform. We drew on clinical data matched to radiomics data derived from diagnostic contrast-enhanced computed tomography (CECT) images in a dataset of 315 patients with oropharyngeal cancer. Contestants were tasked to develop models for (i) classifying patients according to their human papillomavirus status, or (ii) predicting local tumor recurrence, following radiotherapy. Data were split into training, and test sets. Seventeen teams from various professional domains participated in one or both of the challenges. This review paper was based on the contestants' feedback; provided by 8 contestants only (47%). Six contestants (75%) incorporated extracted radiomics features into their predictive model building, either alone (n = 5; 62.5%), as was the case with the winner of the "HPV" challenge, or in conjunction with matched clinical attributes (n = 2; 25%). Only 23% of contestants, notably, including the winner of the "local recurrence" challenge, built their model relying solely on clinical data. In addition to the value of the integration of machine learning into clinical decision-making, our experience sheds light on challenges in sharing and directing existing datasets toward clinical applications of radiomics, including hyper-dimensionality of the clinical/imaging data attributes. Our experience may help guide researchers to create a framework for sharing and reuse of already published data that we believe will ultimately accelerate the pace of clinical applications of radiomics; both in challenge or clinical settings.
Collapse
Affiliation(s)
- Hesham Elhalawani
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Timothy A. Lin
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX, United States
- Baylor College of Medicine, Houston, TX, United States
| | - Stefania Volpe
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX, United States
- Università degli Studi di Milano, Milan, Italy
| | - Abdallah S. R. Mohamed
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX, United States
- Department of Clinical Oncology and Nuclear Medicine, Alexandria University, Alexandria, Egypt
| | - Aubrey L. White
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX, United States
- McGovern Medical School, University of Texas, Houston, TX, United States
| | - James Zafereo
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX, United States
- McGovern Medical School, University of Texas, Houston, TX, United States
| | - Andrew J. Wong
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX, United States
- School of Medicine, The University of Texas Health Science Center San Antonio, San Antonio, TX, United States
| | - Joel E. Berends
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX, United States
- School of Medicine, The University of Texas Health Science Center San Antonio, San Antonio, TX, United States
| | - Shady AboHashem
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX, United States
- Department of Cardiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, United States
| | - Bowman Williams
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX, United States
- Furman University, Greenville, SC, United States
| | - Jeremy M. Aymard
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX, United States
- Abilene Christian University, Abilene, TX, United States
| | - Aasheesh Kanwar
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX, United States
- Department of Radiation Oncology, Oregon Health and Science University, Portland, OR, United States
| | - Subha Perni
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX, United States
- Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, NY, United States
| | - Crosby D. Rock
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX, United States
- Texas Tech University Health Sciences Center El Paso, El Paso, TX, United States
| | - Luke Cooksey
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX, United States
- University of North Texas Health Science Center, Fort Worth, TX, United States
| | - Shauna Campbell
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX, United States
- Department of Radiation Oncology, Cleveland Clinic, Cleveland, OH, United States
| | - Pei Yang
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX, United States
- Baylor College of Medicine, Houston, TX, United States
| | - Khahn Nguyen
- Colgate University, Hamilton City, CA, United States
| | - Rachel B. Ger
- Graduate School of Biomedical Sciences, MD Anderson Cancer Center, Houston, TX, United States
- Department of Radiation Physics, Graduate School of Biomedical Sciences, MD Anderson Cancer Center, Houston, TX, United States
| | - Carlos E. Cardenas
- Graduate School of Biomedical Sciences, MD Anderson Cancer Center, Houston, TX, United States
- Department of Radiation Physics, Graduate School of Biomedical Sciences, MD Anderson Cancer Center, Houston, TX, United States
| | - Xenia J. Fave
- Moores Cancer Center, University of California, La Jolla, San Diego, CA, United States
| | - Carlo Sansone
- Dipartimento di Ingegneria Elettrica e delle Tecnologie dell'Informazione, Università Degli Studi di Napoli Federico II, Naples, Italy
| | - Gabriele Piantadosi
- Dipartimento di Ingegneria Elettrica e delle Tecnologie dell'Informazione, Università Degli Studi di Napoli Federico II, Naples, Italy
| | - Stefano Marrone
- Dipartimento di Ingegneria Elettrica e delle Tecnologie dell'Informazione, Università Degli Studi di Napoli Federico II, Naples, Italy
| | - Rongjie Liu
- Baylor College of Medicine, Houston, TX, United States
- Department of Biostatistics, University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Chao Huang
- Baylor College of Medicine, Houston, TX, United States
- Department of Biostatistics, University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Kaixian Yu
- Baylor College of Medicine, Houston, TX, United States
- Department of Biostatistics, University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Tengfei Li
- Baylor College of Medicine, Houston, TX, United States
- Department of Biostatistics, University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Yang Yu
- Baylor College of Medicine, Houston, TX, United States
- Department of Biostatistics, University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Youyi Zhang
- Baylor College of Medicine, Houston, TX, United States
- Department of Biostatistics, University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Hongtu Zhu
- Baylor College of Medicine, Houston, TX, United States
- Department of Biostatistics, University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Jeffrey S. Morris
- Baylor College of Medicine, Houston, TX, United States
- Department of Biostatistics, University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Veerabhadran Baladandayuthapani
- Baylor College of Medicine, Houston, TX, United States
- Department of Biostatistics, University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - John W. Shumway
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Alakonanda Ghosh
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Andrei Pöhlmann
- Fraunhofer-Institut für Fabrikbetrieb und Automatisierung (IFF), Magdeburg, Germany
| | - Hady A. Phoulady
- Department of Computer Science, University of Southern Maine, Portland, OR, United States
| | - Vibhas Goyal
- Indian Institute of Technology Hyderabad, Sangareddy, India
| | | | | | - David Vock
- Department of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, MN, United States
| | - Stephen Y. Lai
- Department of Head and Neck Surgery, University of Texas MD Anderson Cancer Center, Houston, TX, United States
| | - Dennis S. Mackin
- Colgate University, Hamilton City, CA, United States
- Department of Radiation Physics, Graduate School of Biomedical Sciences, MD Anderson Cancer Center, Houston, TX, United States
| | - Laurence E. Court
- Colgate University, Hamilton City, CA, United States
- Department of Radiation Physics, Graduate School of Biomedical Sciences, MD Anderson Cancer Center, Houston, TX, United States
| | - John Freymann
- Frederick National Laboratory for Cancer Research, Leidos Biomedical Research, Inc., Frederick, MD, United States
| | - Keyvan Farahani
- National Cancer Institute, Rockville, MD, United States
- The Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins Medicine, Baltimore, MD, United States
| | - Jayashree Kaplathy-Cramer
- Department of Radiology and Athinoula A. Martinos Center for Biomedical Imaging, MGH/Harvard Medical School, Boston, MA, United States
| | - Clifton D. Fuller
- Department of Radiation Oncology, University of Texas MD Anderson Cancer Center, Houston, TX, United States
- Baylor College of Medicine, Houston, TX, United States
- Department of Radiation Physics, Graduate School of Biomedical Sciences, MD Anderson Cancer Center, Houston, TX, United States
| |
Collapse
|
38
|
Airan RD, Foss CA, Ellens NPK, Wang Y, Mease RC, Farahani K, Pomper MG. MR-Guided Delivery of Hydrophilic Molecular Imaging Agents Across the Blood-Brain Barrier Through Focused Ultrasound. Mol Imaging Biol 2017; 19:24-30. [PMID: 27481359 DOI: 10.1007/s11307-016-0985-2] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
PURPOSE A wide variety of hydrophilic imaging and therapeutic agents are unable to gain access to the central nervous system (CNS) due to the blood-brain barrier (BBB). In particular, unless a particular transporter exists that may transport the agent across the BBB, most agents that are larger than 500 Da or that are hydrophilic will be excluded by the BBB. Glutamate carboxypeptidase II (GCPII), also known as the prostate-specific membrane antigen (PSMA) in the periphery, has been implicated in various neuropsychiatric conditions. As all agents that target GCPII are hydrophilic and thereby excluded from the CNS, we used GCPII as a platform for demonstrating our MR-guided focused ultrasound (MRgFUS) technique for delivery of GCPII/PSMA-specific imaging agents to the brain. PROCEDURES Female rats underwent MRgFUS-mediated opening of the BBB. After opening of the BBB, either a radio- or fluorescently labeled ureido-based ligand for GCPII/PSMA was administered intravenously. Brain uptake was assessed for 2-(3-{1-carboxy-5-[(6-[18F]fluoro-pyridine-3-carbonyl)-amino]-pentyl}-ureido)-pentanedioic acid ([18F]DCFPyL) and YC-27, two compounds known to bind GCPII/PSMA with high affinity, using positron emission tomography (PET) and near-infrared fluorescence (NIRF) imaging, respectively. Specificity of ligand binding to GCPII/PSMA in the brain was determined with co-administration of a molar excess of ZJ-43, a compound of the same chemical class but different structure from either [18F]DCFPyL or YC-27, which competes for GCPII/PSMA binding. RESULTS Dynamic PET imaging using [18F]DCFPyL demonstrated that target uptake reached a plateau by ∼1 h after radiotracer administration, with target/background ratios continuing to increase throughout the course of imaging, from a ratio of ∼4:1 at 45 min to ∼7:1 by 80 min. NIRF imaging likewise demonstrated delivery of YC-27 to the brain, with clear visualization of tracer in the brain at 24 h. Tissue uptake of both ligands was greatly diminished by ZJ-43 co-administration, establishing specificity of binding of each to GCPII/PSMA. On gross and histological examination, animals showed no evidence for hemorrhage or other deleterious consequences of MRgFUS. CONCLUSIONS MRgFUS provided safe opening of the BBB to enable specific delivery of two hydrophilic agents to target tissues within the brain. This platform might facilitate imaging and therapy using a variety of agents that have heretofore been excluded from the CNS.
Collapse
Affiliation(s)
- Raag D Airan
- The Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins Medical Institutions, Baltimore, MD, USA
| | - Catherine A Foss
- The Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins Medical Institutions, Baltimore, MD, USA
| | - Nicholas P K Ellens
- The Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins Medical Institutions, Baltimore, MD, USA
| | - Yuchuan Wang
- The Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins Medical Institutions, Baltimore, MD, USA
| | - Ronnie C Mease
- The Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins Medical Institutions, Baltimore, MD, USA
| | - Keyvan Farahani
- The Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins Medical Institutions, Baltimore, MD, USA.,National Cancer Institute, National Institutes of Health, Bethesda, MD, USA
| | - Martin G Pomper
- The Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins Medical Institutions, Baltimore, MD, USA.
| |
Collapse
|
39
|
Armato SG, Drukker K, Li F, Hadjiiski L, Tourassi GD, Engelmann RM, Giger ML, Redmond G, Farahani K, Kirby JS, Petrick NA. Letter to the Editor: Use of Publicly Available Image Resources. Acad Radiol 2017; 24:916-917. [PMID: 28506513 DOI: 10.1016/j.acra.2017.03.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2017] [Accepted: 03/16/2017] [Indexed: 10/19/2022]
|
40
|
Fowlkes B, Ghanouni P, Sanghvi N, Coussios C, Lyon PC, Gray M, Mannaris C, Victor MDS, Stride E, Cleveland R, Carlisle R, Wu F, Middleton M, Gleeson F, Aubry JF, Pauly KB, Moonen C, Vortman J, Ghanouni P, Sharabi S, Daniels D, Last D, Guez D, Levy Y, Volovick A, Grinfeld J, Rachmilevich I, Amar T, Zibly Z, Mardor Y, Harnof S, Plaksin M, Weissler Y, Shoham S, Kimmel E, Naor O, Farah N, Shoham S, Paeng DG, Xu Z, Snell J, Quigg AH, Eames M, Jin C, Everstine AC, Sheehan JP, Lopes BS, Kassell N, Looi T, Khokhlova V, Mougenot C, Hynynen K, Drake J, Slayton M, Amodei RC, Compton K, McNelly A, Latt D, Slayton M, Amodei RC, Compton K, Kearney J, Melodelima D, Dupre A, Chen Y, Perol D, Vincenot J, Chapelon JY, Rivoire M, Guo W, Ren G, Shen G, Neidrauer M, Zubkov L, Weingarten MS, Margolis DJ, Lewin PA, McDannold N, Sutton J, Vykhodtseva N, Livingstone M, Kobus T, Zhang YZ, Vykhodtseva N, McDannold N, Schwartz M, Huang Y, Lipsman N, Jain J, Chapman M, Sankar T, Lozano A, Hynynen K, Schwartz M, Yeung R, Huang Y, Lipsman N, Jain J, Chapman M, Lozano A, Hynynen K, Damianou C, Papadopoulos N, Volovick A, Grinfeld J, Levy Y, Brokman O, Zadicario E, Brenner O, Castel D, Wu SY, Grondin J, Zheng W, Heidmann M, Karakatsani ME, Sánchez CJS, Ferrera V, Konofagou EE, Damianou C, Yiannakou M, Cho H, Lee H, Han M, Choi JR, Lee T, Ahn S, Chang Y, Park J, Ellens N, Partanen A, Farahani K, Airan R, Carpentier A, Canney M, Vignot A, Lafon C, Chapelon JY, Delattre JY, Idbaih A, Odéen H, Bolster B, Jeong EK, Parker DL, Gaur P, Feng X, Fielden S, Meyer C, Werner B, Grissom W, Marx M, Ghanouni P, Pauly KB, Weber H, Taviani V, Pauly KB, Ghanouni P, Hargreaves B, Tanaka J, Kikuchi K, Ishijima A, Azuma T, Minamihata K, Yamaguchi S, Nagamune T, Sakuma I, Takagi S, Santin MD, Marsac L, Maimbourg G, Monfort M, Larrat B, François C, Lehéricy S, Tanter M, Aubry JF, Karakatsani ME, Samiotaki G, Wang S, Acosta C, Feinberg ER, Konofagou EE, Kovacs ZI, Tu TW, Papadakis GZ, Reid WC, Hammoud DA, Frank JA, Kovacs ZI, Kim S, Jikaria N, Bresler M, Qureshi F, Frank JA, Xia J, Tsui PS, Liu HL, Plata JC, Fielden S, Sveinsson B, Hargreaves B, Meyer C, Pauly KB, Plata JC, Salgaonkar VA, Adams M, Diederich C, Ozhinsky E, Bucknor MD, Rieke V, Partanen A, Mikhail A, Severance L, Negussie AH, Wood B, de Greef M, Schubert G, Moonen C, Ries M, Poorman ME, Dockery M, Chaplin V, Dudzinski SO, Spears R, Caskey C, Giorgio T, Grissom W, Costa MM, Papaevangelou E, Shah A, Rivens I, Box C, Bamber J, ter Haar G, Burks SR, Nagle M, Nguyen B, Bresler M, Frank JA, Burks SR, Nagle M, Nguyen B, Bresler M, Kim S, Milo B, Frank JA, Le NM, Song S, Zhou K, Nabi G, Huang Z, Ben-Ezra S, Rosen S, Mihcin S, Strehlow J, Karakitsios I, Le N, Schwenke M, Demedts D, Prentice P, Haase S, Preusser T, Melzer A, Mestas JL, Chettab K, Gomez GS, Dumontet C, Werle B, Lafon C, Marquet F, Bour P, Vaillant F, Amraoui S, Dubois R, Ritter P, Haïssaguerre M, Hocini M, Bernus O, Quesson B, Livneh A, Kimmel E, Adam D, Robin J, Arnal B, Fink M, Tanter M, Pernot M, Khokhlova TD, Schade GR, Wang YN, Kreider W, Simon J, Starr F, Karzova M, Maxwell A, Bailey MR, Khokhlova V, Lundt JE, Allen SP, Sukovich JR, Hall T, Xu Z, Schade GR, Wang YN, Khokhlova TD, May P, Lin DW, Bailey MR, Khokhlova V, Constans C, Deffieux T, Tanter M, Aubry JF, Park EJ, Ahn YD, Kang SY, Park DH, Lee JY, Vidal-Jove J, Perich E, Ruiz A, Jaen A, Eres N, del Castillo MA, Myers R, Kwan J, Coviello C, Rowe C, Crake C, Finn S, Jackson E, Carlisle R, Coussios C, Pouliopoulos A, Li C, Tinguely M, Tang MX, Garbin V, Choi JJ, Lyon PC, Mannaris C, Gray M, Folkes L, Stratford M, Carlisle R, Wu F, Middleton M, Gleeson F, Coussios C, Nwokeoha S, Carlisle R, Cleveland R, Wang YN, Khokhlova TD, Li T, Farr N, D’Andrea S, Starr F, Gravelle K, Chen H, Partanen A, Lee D, Hwang JH, Tardoski S, Ngo J, Gineyts E, Roux JP, Clézardin P, Melodelima D, Conti A, Magnin R, Gerstenmayer M, Lux F, Tillement O, Mériaux S, Penna SD, Romani GL, Dumont E, Larrat B, Sun T, Power C, Zhang YZ, Sutton J, Miller E, McDannold N, Sapozhnikov O, Tsysar S, Yuldashev PV, Khokhlova V, Svet V, Kreider W, Li D, Pellegrino A, Petrinic N, Siviour C, Jerusalem A, Cleveland R, Yuldashev PV, Karzova M, Cunitz BW, Dunmire B, Kreider W, Sapozhnikov O, Bailey MR, Khokhlova V, Inserra C, Guedra M, Mauger C, Gilles B, Solovchuk M, Sheu TWH, Thiriet M, Zhou Y, Neufeld E, Baumgartner C, Payne D, Kyriakou A, Kuster N, Xiao X, McLeod H, Melzer A, Dillon C, Rieke V, Ghanouni P, Parker DL, Payne A, Khokhova VA, Yuldashev PV, Sinilshchikov I, Andriyakhina Y, Khokhlova TD, Kreider W, Maxwell A, Sapozhnikov O, Partanen A, Rybyanets A, Shvetsova N, Berkovich A, Shvetsov I, Sapozhnikov O, Khokhlova V, Shaw CJ, Rivens I, Civale J, Giussani D, ter Haar G, Lees C, Bour P, Marquet F, Ozenne V, Toupin S, Quesson B, Dumont E, Ozhinsky E, Salgaonkar V, Diederich C, Rieke V, Kaye E, Monette S, Maybody M, Srimathveeravalli G, Solomon S, Gulati A, Preusser T, Haase S, Bezzi M, Jenne JW, Lango T, Levy Y, Müller M, Sat G, Tanner C, Zangos S, Günther M, Melzer A, Lafon C, Dinh AH, Niaf E, Bratan F, Guillen N, Souchon R, Lartizien C, Crouzet S, Rouviere O, Chapelon JY, Han Y, Wang S, Konofagou EE, Payen T, Palermo C, Sastra S, Chen H, Han Y, Olive K, Konofagou EE, van Breugel JM, de Greef M, Mougenot C, van den Bosch MA, Moonen C, Ries M, Gerstenmayer M, Magnin R, Fellah B, Le Bihan D, Larrat B, Gerstenmayer M, Magnin R, Mériaux S, Le Bihan D, Larrat B, Allen SP, Hernandez-Garcia L, Cain CA, Hall T, Lyka E, Elbes D, Coviello C, Cleveland R, Coussios C, Zhou K, Le NM, Li C, Huang Z, Tamano S, Jimbo H, Azuma T, Yoshizawa S, Fujiwara K, Itani K, Umemura SI, Damianou C, Yiannakou M, Ellens N, Partanen A, Stoianovici D, Farahani K, Zaini Z, Takagi R, Yoshizawa S, Umemura SI, Zong S, Shen G, Watkins R, Pascal-Tenorio A, Adams M, Plata JC, Salgaonkar V, Jones P, Butts-Pauly K, Diederich C, Bouley D, Rybyanets A, Ren G, Guo W, Shen G, Chen Y, Lin CY, Hsieh HY, Wei KC, Liu HL, Garnier C, Renault G, Farr N, Partanen A, Negussie AH, Mikhail A, Seifabadi R, Wilson E, Eranki A, Kim P, Wood B, Lübke D, Jenne JW, Huber P, Günther M, Lübke D, Georgii J, Schwenke M, Dresky CV, Haller J, Günther M, Preusser T, Jenne JW, Eranki A, Farr N, Partanen A, Yarmolenko P, Negussie AH, Sharma K, Celik H, Wood B, Kim P, Li G, Qiu W, Zheng H, Tsai MY, Chu PC, Liu HL, Webb T, Vyas U, Pauly KB, Walker M, Zhong J, Looi T, Waspe AC, Drake J, Hodaie M, Yang FY, Huang SL, Zur Y, Volovick A, Assif B, Aurup C, Kamimura H, Wang S, Chen H, Acosta C, Carneiro AA, Konofagou EE, Volovick A, Grinfeld J, Castel D, Rothlübbers S, Schwaab J, Tanner C, Mihcin S, Houston G, Günther M, Jenne JW, Ozhinsky E, Bucknor MD, Rieke V, Azhari H, Weiss N, Sosna J, Goldberg SN, Barrere V, Melodelima D, Jang KW, Burks SR, Kovacs ZI, Tu TW, Lewis B, Kim S, Nagle M, Jikaria N, Frank JA, Zhou Y, Wang X, Ahn YD, Park EJ, Park DH, Kang SY, Lee JY, Suomi V, Konofagou EE, Edwards D, Cleveland R, Larrabee Z, Eames M, Hananel A, Aubry JF, Rafaely B, Volovick A, Grinfeld J, Kimmel E, Debbiny RE, Dekel CZ, Assa M, Kimmel E, Menikou G, Damianou C, Mouratidis P, Rivens I, ter Haar G, Pineda-Pardo JA, de Pedro MDÁ, Martinez R, Hernandez F, Casas S, Oliver C, Pastor P, Vela L, Obeso J, Greillier P, Zorgani A, Souchon R, Melodelima D, Catheline S, Lafon C, Solovov V, Vozdvizhenskiy MO, Orlov AE, Wu CH, Sun MK, Shih TT, Chen WS, Prieur F, Pillon A, Mestas JL, Cartron V, Cebe P, Chansard N, Lafond M, Lafon C, Inserra C, Seya PM, Chen WS, Bera JC, Boissenot T, Larrat B, Fattal E, Bordat A, Chacun H, Guetin C, Tsapis N, Maruyama K, Unga J, Suzuki R, Fant C, Lafond M, Rogez B, Ngo J, Lafon C, Mestas JL, Afadzi M, Myhre OF, Vea S, Bjørkøy A, Yemane PT, van Wamel A, Berg S, Hansen R, Angelsen B, Davies C. International Society for Therapeutic Ultrasound Conference 2016. J Ther Ultrasound 2017. [PMCID: PMC5374646 DOI: 10.1186/s40349-016-0079-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/02/2022] Open
|
41
|
Airan RD, Meyer RA, Ellens NPK, Rhodes KR, Farahani K, Pomper MG, Kadam SD, Green JJ. Noninvasive Targeted Transcranial Neuromodulation via Focused Ultrasound Gated Drug Release from Nanoemulsions. Nano Lett 2017; 17:652-659. [PMID: 28094959 PMCID: PMC5362146 DOI: 10.1021/acs.nanolett.6b03517] [Citation(s) in RCA: 109] [Impact Index Per Article: 15.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/21/2016] [Revised: 01/05/2017] [Indexed: 05/19/2023]
Abstract
Targeted, noninvasive neuromodulation of the brain of an otherwise awake subject could revolutionize both basic and clinical neuroscience. Toward this goal, we have developed nanoparticles that allow noninvasive uncaging of a neuromodulatory drug, in this case the small molecule anesthetic propofol, upon the application of focused ultrasound. These nanoparticles are composed of biodegradable and biocompatible constituents and are activated using sonication parameters that are readily achievable by current clinical transcranial focused ultrasound systems. These particles are potent enough that their activation can silence seizures in an acute rat seizure model. Notably, there is no evidence of brain parenchymal damage or blood-brain barrier opening with their use. Further development of these particles promises noninvasive, focal, and image-guided clinical neuromodulation along a variety of pharmacological axes.
Collapse
Affiliation(s)
- Raag D. Airan
- Department of Radiology
and Radiological Science, Johns Hopkins
University School of Medicine, Baltimore, Maryland 21231, United States
- Department
of Biomedical Engineering and the Translational Tissue Engineering
Center, Johns Hopkins University School
of Medicine, Baltimore, Maryland 21231, United
States
- Department of Radiology, Stanford
University, Stanford, California 94305, United States
| | - Randall A. Meyer
- Department
of Biomedical Engineering and the Translational Tissue Engineering
Center, Johns Hopkins University School
of Medicine, Baltimore, Maryland 21231, United
States
- Institute for NanoBioTechnology, Johns Hopkins University, Baltimore, Maryland 21231, United States
| | - Nicholas P. K. Ellens
- Department of Radiology
and Radiological Science, Johns Hopkins
University School of Medicine, Baltimore, Maryland 21231, United States
| | - Kelly R. Rhodes
- Department
of Biomedical Engineering and the Translational Tissue Engineering
Center, Johns Hopkins University School
of Medicine, Baltimore, Maryland 21231, United
States
| | - Keyvan Farahani
- Department of Radiology
and Radiological Science, Johns Hopkins
University School of Medicine, Baltimore, Maryland 21231, United States
- National
Cancer Institute/National Institutes of Health, Bethesda, Maryland 20892, United States
| | - Martin G. Pomper
- Department of Radiology
and Radiological Science, Johns Hopkins
University School of Medicine, Baltimore, Maryland 21231, United States
- Institute for NanoBioTechnology, Johns Hopkins University, Baltimore, Maryland 21231, United States
- Department
of Oncology, Johns Hopkins University School
of Medicine, Baltimore, Maryland 21231, United
States
| | - Shilpa D. Kadam
- Neuroscience Laboratory, Hugo Moser Research Institute, Kennedy Krieger Institute, Baltimore, Maryland 21287, United States
- Department
of Neurology, Johns Hopkins Medical Institutions, Baltimore, Maryland 21287, United States
| | - Jordan J. Green
- Department
of Biomedical Engineering and the Translational Tissue Engineering
Center, Johns Hopkins University School
of Medicine, Baltimore, Maryland 21231, United
States
- Institute for NanoBioTechnology, Johns Hopkins University, Baltimore, Maryland 21231, United States
- Department
of Oncology, Johns Hopkins University School
of Medicine, Baltimore, Maryland 21231, United
States
- Departments of Neurosurgery, Ophthalmology, and Materials Science
and Engineering, Johns Hopkins University
School of Medicine, Baltimore, Maryland 21231, United States
| |
Collapse
|
42
|
Farahani K, Kalpathy-Cramer J, Chenevert TL, Rubin DL, Sunderland JJ, Nordstrom RJ, Buatti J, Hylton N. Computational Challenges and Collaborative Projects in the NCI Quantitative Imaging Network. ACTA ACUST UNITED AC 2016; 2:242-249. [PMID: 28798963 PMCID: PMC5548142 DOI: 10.18383/j.tom.2016.00265] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
Abstract
The Quantitative Imaging Network (QIN) of the National Cancer Institute (NCI) conducts research in development and validation of imaging tools and methods for predicting and evaluating clinical response to cancer therapy. Members of the network are involved in examining various imaging and image assessment parameters through network-wide cooperative projects. To more effectively use the cooperative power of the network in conducting computational challenges in benchmarking of tools and methods and collaborative projects in analytical assessment of imaging technologies, the QIN Challenge Task Force has developed policies and procedures to enhance the value of these activities by developing guidelines and leveraging NCI resources to help their administration and manage dissemination of results. Challenges and Collaborative Projects (CCPs) are further divided into technical and clinical CCPs. As the first NCI network to engage in CCPs, we anticipate a variety of CCPs to be conducted by QIN teams in the coming years. These will be aimed to benchmark advanced software tools for clinical decision support, explore new imaging biomarkers for therapeutic assessment, and establish consensus on a range of methods and protocols in support of the use of quantitative imaging to predict and assess response to cancer therapy.
Collapse
Affiliation(s)
- Keyvan Farahani
- Cancer Imaging Program, National Cancer Institute, Bethesda, Maryland
| | | | | | - Daniel L Rubin
- Department of Radiology, Biomedical Data Science, and Medicine (Biomedical Informatics Research), Stanford University, Palo Alto, California
| | | | | | - John Buatti
- Department of Radiation Oncology, University of Iowa, Iowa City, Iowa
| | - Nola Hylton
- Department of Radiology, University of California San Francisco, San Francisco, California
| |
Collapse
|
43
|
Zaaroor M, Sinai A, Goldsher D, Eran A, Nassar M, Schlesinger I, Parker J, Ravikumar V, Ghanouni P, Stein S, Halpern C, Krishna V, Hargrove A, Agrawal P, Changizi B, Bourekas E, Knopp M, Rezai A, Mead B, Kim N, Mastorakos P, Suk JS, Miller W, Klibanov A, Hanes J, Price R, Wang S, Olumolade O, Kugelman T, Jackson-Lewis V, Karakatsani ME, Han Y, Przedborski S, Konofagou E, Hynynen K, Aubert I, Leinenga G, Nisbet R, Hatch R, Van der Jeugd A, Evans H, Götz J, Götz J, Nisbet R, Van der Jeugd A, Evans H, Leinenga G, Fishman P, Yarowsky P, Frenkel V, Wei-Bin S, Nguyen B, Sanchez CS, Acosta C, Chen C, Wu SY, Karakatsani ME, Konofagou E, Aryal M, Papademetriou IT, Zhang YZ, Power C, McDannold N, Porter T, Kovacs Z, Kim S, Jikaria N, Qureshi F, Bresler M, Frank J, Odéen H, Chiou G, Snell J, Todd N, Madore B, Parker D, Pauly KB, Marx M, Ghanouni P, Jonathan S, Grissom W, Arvanitis C, McDannold N, Clement G, Parker D, de Bever J, Odéen H, Payne A, Christensen D, Maimbourg G, Santin MD, Houdouin A, Lehericy S, Tanter M, Aubry JF, Pauly KB, Federau C, Werner B, Halpern C, Ghanouni P, Preusser T, McLeod H, Abraham C, Pichardo S, Curiel L, Ramaekers P, de Greef M, Berriet R, Moonen C, Ries M, Paeng DG, Dillon C, Janát-Amsbury M, Payne A, Corea J, Ye PP, Arias AC, Pauly KB, Lustig M, Svedin B, Payne A, Xu Z, Parker D, Snell J, Quigg A, Eames M, Jin C, Everstine A, Sheehan J, Lopes MB, Kassell N, Snell J, Quigg A, Drake J, Price K, Lustgarten L, Sin V, Mougenot C, Donner E, Tam E, Hodaie M, Waspe A, Looi T, Pichardo S, Lee W, Chung YA, Jung Y, Song IU, Yoo SS, Lee W, Kim HC, Jung Y, Chung YA, Song IU, Lee JH, Yoo SS, Caskey C, Zinke W, Cosman J, Shuman J, Schall J, Aurup C, Wang S, Chen H, Acosta C, Konofagou E, Kamimura H, Carneiro A, Todd N, Sun T, Zhang YZ, Power C, Nazai N, Patz S, Livingstone M, McDannold N, Mainprize T, Huang Y, Alkins R, Chapman M, Perry J, Lipsman N, Bethune A, Sahgal A, Trudeau M, Hynynen K, Liu HL, Hsu PH, Wei KC, Sun T, Power C, Zhang YZ, Sutton J, Alexander P, Aryal M, Miller E, McDannold N, Kobus T, Zhang YZ, McDannold N, Carpentier A, Canney M, Vignot A, Beccaria K, Leclercq D, Lafon C, Chapelon JY, Hoang-Xuan K, Delattre JY, Idbaih A, Xu Z, Moore D, Xu A, Schmitt P, Snell J, Foley J, Eames M, Sheehan J, Kassell N, Sukovich J, Cain C, Xu Z, Pandey A, Snell J, Chaudhary N, Camelo-Piragua S, Allen S, Paeng DG, Cannata J, Teofilovic D, Bertolina J, Kassell N, Hall T, Xu Z, Wu SY, Karakatsani ME, Grondin J, Sanchez CS, Ferrera V, Konofagou E, ter Haar G, Mouratidis P, Repasky E, Timbie K, Badr L, Campbell B, McMichael J, Buckner A, Prince J, Stevens A, Bullock T, Price R, Skalina K, Guha C, Orsi F, Bonomo G, Vigna PD, Mauri G, Varano G, Schade G, Wang YN, Pillarisetty V, Hwang JH, Khokhlova V, Bailey M, Khokhlova T, Khokhlova V, Sinilshchikov I, Yuldashev P, Andriyakhina Y, Kreider W, Maxwell A, Khokhlova T, Sapozhnikov O, Partanen A, Lundt J, Allen S, Sukovich J, Hall T, Cain C, Xu Z, Preusser T, Haase S, Bezzi M, Jenne J, Langø T, Midiri M, Mueller M, Sat G, Tanner C, Zangos S, Guenther M, Melzer A, Menciassi A, Tognarelli S, Cafarelli A, Diodato A, Ciuti G, Rothluebbers S, Schwaab J, Strehlow J, Mihcin S, Tanner C, Tretbar S, Preusser T, Guenther M, Jenne J, Payen T, Palermo C, Sastra S, Chen H, Han Y, Olive K, Konofagou E, Adams M, Salgaonkar V, Scott S, Sommer G, Diederich C, Vidal-Jove J, Perich E, Ruiz A, Velat M, Melodelima D, Dupre A, Vincenot J, Yao C, Perol D, Rivoire M, Tucci S, Mahakian L, Fite B, Ingham E, Tam S, Hwang CI, Tuveson D, Ferrara K, Scionti S, Chen L, Cvetkovic D, Chen X, Gupta R, Wang B, Ma C, Bader K, Haworth K, Maxwell A, Holland C, Sanghvi N, Carlson R, Chen W, Chaussy C, Thueroff S, Cesana C, Bellorofonte C, Wang Q, Wang H, Wang S, Zhang J, Bazzocchi A, Napoli A, Staruch R, Bing C, Shaikh S, Nofiele J, Szczepanski D, Staruch MW, Williams N, Laetsch T, Chopra R, Ghanouni P, Rosenberg J, Bitton R, Napoli A, LeBlang S, Meyer J, Hurwitz M, Pauly KB, Partanen A, Yarmolenko P, Partanen A, Celik H, Eranki A, Beskin V, Santos D, Patel J, Oetgen M, Kim A, Kim P, Sharma K, Chisholm A, Drake J, Aleman D, Waspe A, Looi T, Pichardo S, Napoli A, Bazzocchi A, Scipione R, Temple M, Waspe A, Amaral JG, Huang Y, Endre R, Lamberti-Pasculli M, de Ruiter J, Campbell F, Stimec J, Gupta S, Singh M, Mougenot C, Hopyan S, Hynynen K, Czarnota G, Drake J, Brenin D, Rochman C, Kovatcheva R, Vlahov J, Zaletel K, Stoinov J, Han Y, Wang S, Konofagou E, Bucknor M, Rieke V, Shim J, Staruch R, Koral K, Chopra R, Laetsch T, Lang B, Wong C, Lam H, Kovatcheva R, Vlahov J, Zaletel K, Stoinov J, Shinkov A, Hu J, Sharma K, Zhang X, Macoskey J, Ives K, Owens G, Gurm H, Shi J, Pizzuto M, Cain C, Xu Z, Payne A, Dillon C, Christofferson I, Hilas E, Shea J, Greillier P, Ankou B, Bessière F, Zorgani A, Pioche M, Kwiecinski W, Magat J, Melot-Dusseau S, Lacoste R, Quesson B, Pernot M, Catheline S, Chevalier P, Lafon C, Marquet F, Bour P, Vaillant F, Amraoui S, Dubois R, Ritter P, Haïssaguerre M, Hocini M, Bernus O, Quesson B, Tebebi P, Burks S, Kim S, Milo B, Frank J, Gertner M, Zhang J, Wong A, Fite B, Liu Y, Kheirolomoom A, Seo J, Watson K, Mahakian L, Tam S, Zhang H, Foiret J, Borowsky A, Ferrara K, Xu D, Melzer A, Thanou M, Centelles M, Wright M, Amrahli M, So PW, Gedroyc W, Centelles M, Wright M, Gedroyc W, Thanou M, Kneepkens E, Heijman E, Keupp J, Weiss S, Nicolay K, Grüll H, Fite B, Wong A, Liu Y, Kheirolomoom A, Mahakian L, Tam S, Foiret J, Ferrara K, Burks S, Nagle M, Kim S, Milo B, Frank J, Sapozhnikov O, Nikolaeva AV, Terzi ME, Tsysar SA, Maxwell A, Cunitz B, Bailey M, Mourad P, Downs M, Yang G, Wang Q, Konofagou E, Burks S, Nagle M, Nguyen B, Bresler M, Kim S, Milo B, Frank J, Burks S, Nagle M, Kim S, Milo B, Frank J, Chen J, Farry J, Dixon A, Du Z, Dhanaliwala A, Hossack J, Klibanov A, Ranjan A, Maples D, Chopra R, Bing C, Staruch R, Wardlow R, Staruch MW, Malayer J, Ramachandran A, Nofiele J, Namba H, Kawasaki M, Izumi M, Kiyasu K, Takemasa R, Ikeuchi M, Ushida T, Crake C, Papademetriou IT, Zhang YZ, Porter T, McDannold N, Kothapalli SVVN, Leighton W, Wang Z, Partanen A, Gach HM, Straube W, Altman M, Chen H, Kim YS, Lim HK, Rhim H, Kim YS, Lim HK, Rhim H, van Breugel J, Braat M, Moonen C, van den Bosch M, Ries M, Marrocchio C, Dababou S, Bitton R, Pauly KB, Ghanouni P, Lee JY, Lee JY, Chung HH, Kang SY, Kang KJ, Son KH, Zhang D, Adams M, Salgaonkar V, Plata J, Jones P, Pascal-Tenorio A, Bouley D, Sommer G, Pauly KB, Diederich C, Bond A, Dallapiazza R, Huss D, Warren A, Sperling S, Gwinn R, Shah B, Elias WJ, Curley C, Zhang Y, Negron K, Miller W, Klibanov A, Abounader R, Suk JS, Hanes J, Price R, Karakatsani ME, Samiotaki G, Wang S, Kugelman T, Acosta C, Konofagou E, Kovacs Z, Tu TW, Papadakis G, Hammoud D, Frank J, Silvestrini M, Wolfram F, Güllmar D, Reichenbach J, Hofmann D, Böttcher J, Schubert H, Lesser TG, Almquist S, Parker D, Christensen D, Camarena F, Jiménez-Gambín S, Jiménez N, Konofagou E, Chang JW, Chaplin V, Griesenauer R, Miga M, Caskey C, Ellens N, Airan R, Quinones-Hinojosa A, Farahani K, Partanen A, Feng X, Fielden S, Zhao L, Miller W, Wintermark M, Pauly KB, Meyer C, Guo S, Lu X, Zhuo J, Xu S, Gullapalli R, Gandhi D, Jin C, Brokman O, Eames M, Snell J, Paeng DG, Baek H, Kim H, Leung S, Webb T, Pauly KB, McDannold N, Zhang YZ, Vykhodtseva N, Nguyen TS, Sukovich J, Hall T, Xu Z, Cain C, Park CK, Park SM, Jung NY, Kim MS, Chang WS, Jung HH, Chang JW, Pichardo S, Hynynen K, Plaksin M, Weissler Y, Shoham S, Kimmel E, Quigg A, Snell J, Paeng DG, Eames M, Sapozhnikov O, Rosnitskiy PB, Khokhlova V, Shoham S, Krupa S, Hazan E, Naor O, Levy Y, Maimon N, Brosh I, Kimmel E, Kahn I, Sukovich J, Xu Z, Hall T, Allen S, Cain C, Cahill J, Sun T, Zhang YZ, Power C, Livingstone M, McDannold N, Todd N, Colas EC, Wydra A, Waspe A, Looi T, Maev R, Pichardo S, Drake J, Aly A, Sun T, Zhang YZ, Sesenoglu-Laird O, Padegimas L, Cooper M, McDannold N, Waszczak B, Tehrani S, Miller W, Slingluff C, Larner J, Andarawewa K, Bucknor M, Ozhinsky E, Shah R, Krug R, Rieke V, Deckers R, Linn S, Suelmann B, Braat M, Witkamp A, Vaessen P, van Diest P, Bartels LW, Bos C, van den Bosch M, Borys N, Storm G, Van der Wall E, Moonen C, Farr N, Alnazeer M, Yarmolenko P, Katti P, Partanen A, Eranki A, Kim P, Wood B, Farrer A, Almquist S, Dillon C, Parker D, Christensen D, Payne A, Ferrer C, Bartels LW, de Senneville BD, van Stralen M, Moonen C, Bos C, Liu Y, Liu J, Fite B, Foiret J, Leach JK, Ferrara K, Gupta R, Cvetkovic D, Ma C, Chen L, Haase S, Zidowitz S, Melzer A, Preusser T, Lee HL, Hsu FC, Kuo CC, Jeng SC, Chen TH, Yang NY, Chiou JF, Jeng SC, Kao YT, Pan CH, Wu JF, Chen TH, Hsu FC, Lee HL, Chiou JF, Hsu FC, Tsai YC, Lee HL, Chiou JF, Johnson S, Parker D, Payne A, Li D, He Y, Mihcin S, Karakitsios I, Strehlow J, Schwenke M, Haase S, Demedts D, Levy Y, Preusser T, Melzer A, Mihcin S, Rothluebbers S, Karakitsios I, Xiao X, Strehlow J, Demedts D, Cavin I, Sat G, Preusser T, Melzer A, Minalga E, Payne A, Merrill R, Parker D, Hadley R, Ramaekers P, Ries M, Moonen C, de Greef M, Shahriari K, Parvizi MH, Asadnia K, Chamanara M, Kamrava SK, Chabok HR, Schwenke M, Strehlow J, Demedts D, Tanner C, Rothluebbers S, Preusser T, Strehlow J, Stein R, Demedts D, Schwenke M, Rothluebbers S, Preusser T, Demedts D, Haase S, Muller S, Strehlow J, Langø T, Preusser T, Tan J, Zachiu C, Ramaekers P, Moonen C, Ries M, Wolfram F, Güllmar D, Schubert H, Lesser TG, Erasmus HP, Colas EC, Waspe A, Mougenot C, Looi T, Van Arsdell G, Benson L, Drake J, Jang KW, Tu TW, Jikaria N, Nagle M, Angstadt M, Lewis B, Qureshi F, Burks S, Frank J, McLean H, Payne A, Hoogenboom M, Eikelenboom D, den Brok M, Wesseling P, Heerschap A, Fütterer J, Adema G, Wang K, Zhang Y, Zhong P, Xiao X, Joy J, McLeod H, Melzer A, Bing C, Staruch R, Nofiele J, Szczepanski D, Staruch MW, Laetsch T, Chopra R, Bing C, Staruch R, Yarmolenko P, Celik H, Nofiele J, Szczepanski D, Kim P, Kim H, Lewis M, Chopra R, Shah R, Ozhinsky E, Rieke V, Bucknor M, Diederich C, Salgaonkar V, Jones P, Adams M, Ozilgen A, Zahos P, Coughlin D, Tang X, Lotz J, Jedruszczuk K, Gulati A, Solomon S, Kaye E, Fielden S, Mugler J, Miller W, Pauly KB, Meyer C, Barbato G, Scoarughi GL, Corso C, Gorgone A, Migliore IG, Larrabee Z, Hananel A, Eames M, Aubry JF, Eranki A, Farr N, Partanen A, Sharma K, Yarmolenko P, Wood B, Kim P, Farr N, Kothapalli SVVN, Eranki A, Negussie A, Wilson E, Seifabadi R, Kim P, Chen H, Wood B, Partanen A, Moon H, Kang J, Sim C, Chang JH, Kim H, Lee HJ, Sasaki N, Takiguchi M, Sebeke L, Luo X, de Jager B, Heemels M, Heijman E, Grüll H, Strehlow J, Schwenke M, Demedts D. 5th International Symposium on Focused Ultrasound. J Ther Ultrasound 2016. [PMCID: PMC5123388 DOI: 10.1186/s40349-016-0076-5] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022] Open
|
44
|
Armato SG, Drukker K, Li F, Hadjiiski L, Tourassi GD, Engelmann RM, Giger ML, Redmond G, Farahani K, Kirby JS, Clarke LP. LUNGx Challenge for computerized lung nodule classification. J Med Imaging (Bellingham) 2016; 3:044506. [PMID: 28018939 PMCID: PMC5166709 DOI: 10.1117/1.jmi.3.4.044506] [Citation(s) in RCA: 57] [Impact Index Per Article: 7.1] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2016] [Accepted: 11/17/2016] [Indexed: 11/14/2022] Open
Abstract
The purpose of this work is to describe the LUNGx Challenge for the computerized classification of lung nodules on diagnostic computed tomography (CT) scans as benign or malignant and report the performance of participants' computerized methods along with that of six radiologists who participated in an observer study performing the same Challenge task on the same dataset. The Challenge provided sets of calibration and testing scans, established a performance assessment process, and created an infrastructure for case dissemination and result submission. Ten groups applied their own methods to 73 lung nodules (37 benign and 36 malignant) that were selected to achieve approximate size matching between the two cohorts. Area under the receiver operating characteristic curve (AUC) values for these methods ranged from 0.50 to 0.68; only three methods performed statistically better than random guessing. The radiologists' AUC values ranged from 0.70 to 0.85; three radiologists performed statistically better than the best-performing computer method. The LUNGx Challenge compared the performance of computerized methods in the task of differentiating benign from malignant lung nodules on CT scans, placed in the context of the performance of radiologists on the same task. The continued public availability of the Challenge cases will provide a valuable resource for the medical imaging research community.
Collapse
Affiliation(s)
- Samuel G. Armato
- The University of Chicago, Department of Radiology, 5841 South Maryland Avenue, MC 2026, Chicago, Illinois 60637, United States
| | - Karen Drukker
- The University of Chicago, Department of Radiology, 5841 South Maryland Avenue, MC 2026, Chicago, Illinois 60637, United States
| | - Feng Li
- The University of Chicago, Department of Radiology, 5841 South Maryland Avenue, MC 2026, Chicago, Illinois 60637, United States
| | - Lubomir Hadjiiski
- University of Michigan, Department of Radiology, 1500 East Medical Center Drive, Ann Arbor, Michigan 48109, United States
| | - Georgia D. Tourassi
- Health Data Sciences Institute, Biomedical Science and Engineering Center, Oak Ridge National Laboratory, P.O. Box 2008 MS6085 Oak Ridge, Tennessee 37831-6085, United States
| | - Roger M. Engelmann
- The University of Chicago, Department of Radiology, 5841 South Maryland Avenue, MC 2026, Chicago, Illinois 60637, United States
| | - Maryellen L. Giger
- The University of Chicago, Department of Radiology, 5841 South Maryland Avenue, MC 2026, Chicago, Illinois 60637, United States
| | - George Redmond
- National Cancer Institute, Cancer Imaging Program, Division of Cancer Treatment and Diagnosis, 9609 Medical Center Drive, Bethesda, Maryland 20892, United States
| | - Keyvan Farahani
- National Cancer Institute, Cancer Imaging Program, Division of Cancer Treatment and Diagnosis, 9609 Medical Center Drive, Bethesda, Maryland 20892, United States
| | - Justin S. Kirby
- Leidos Biomedical Research, Inc., Frederick National Laboratory for Cancer Research, Cancer Imaging Program, 8560 Progress Drive, Frederick, Maryland 21702, United States
| | - Laurence P. Clarke
- National Cancer Institute, Cancer Imaging Program, Division of Cancer Treatment and Diagnosis, 9609 Medical Center Drive, Bethesda, Maryland 20892, United States
| |
Collapse
|
45
|
Farahani K. TU-FG-207B-04: NCI Funding Opportunities in Imaging and Image-Guided Therapy. Med Phys 2016. [DOI: 10.1118/1.4957587] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022] Open
|
46
|
Farahani K. TU-FG-207B-00: Federal Funding Opportunities and Grantsmanship. Med Phys 2016. [DOI: 10.1118/1.4957583] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022] Open
|
47
|
Farahani K. MO-DE-202-01: Image-Guided Focused Ultrasound Surgery and Therapy. Med Phys 2016. [DOI: 10.1118/1.4957225] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022] Open
|
48
|
Sammet S, Partanen A, Yousuf A, Sammet CL, Ward EV, Wardrip C, Niekrasz M, Antic T, Razmaria A, Farahani K, Sokka S, Karczmar G, Oto A. Cavernosal nerve functionality evaluation after magnetic resonance imaging-guided transurethral ultrasound treatment of the prostate. World J Radiol 2015; 7:521-530. [PMID: 26753067 PMCID: PMC4697126 DOI: 10.4329/wjr.v7.i12.521] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/07/2015] [Revised: 06/15/2015] [Accepted: 11/25/2015] [Indexed: 02/06/2023] Open
Abstract
AIM: To evaluate the feasibility of using therapeutic ultrasound as an alternative treatment option for organ-confined prostate cancer.
METHODS: In this study, a trans-urethral therapeutic ultrasound applicator in combination with 3T magnetic resonance imaging (MRI) guidance was used for real-time multi-planar MRI-based temperature monitoring and temperature feedback control of prostatic tissue thermal ablation in vivo. We evaluated the feasibility and safety of MRI-guided trans-urethral ultrasound to effectively and accurately ablate prostate tissue while minimizing the damage to surrounding tissues in eight canine prostates. MRI was used to plan sonications, monitor temperature changes during therapy, and to evaluate treatment outcome. Real-time temperature and thermal dose maps were calculated using the proton resonance frequency shift technique and were displayed as two-dimensional color-coded overlays on top of the anatomical images. After ultrasound treatment, an evaluation of the integrity of cavernosal nerves was performed during prostatectomy with a nerve stimulator that measured tumescence response quantitatively and indicated intact cavernous nerve functionality. Planned sonication volumes were visually correlated to MRI ablation volumes and corresponding histo-pathological sections after prostatectomy.
RESULTS: A total of 16 sonications were performed in 8 canines. MR images acquired before ultrasound treatment were used to localize the prostate and to prescribe sonication targets in all canines. Temperature elevations corresponded within 1 degree of the targeted sonication angle, as well as with the width and length of the active transducer elements. The ultrasound treatment procedures were automatically interrupted when the temperature in the target zone reached 56 °C. In all canines erectile responses were evaluated with a cavernous nerve stimulator post-treatment and showed a tumescence response after stimulation with an electric current. These results indicated intact cavernous nerve functionality. In all specimens, regions of thermal ablation were limited to areas within the prostate capsule and no damage was observed in periprostatic tissues. Additionally, a visual analysis of the ablation zones on contrast-enhanced MR images acquired post ultrasound treatment correlated excellent with the ablation zones on thermal dose maps. All of the ablation zones received a consensus score of 3 (excellent) for the location and size of the correlation between the histologic ablation zone and MRI based ablation zone. During the prostatectomy and histologic examination, no damage was noted in the bladder or rectum.
CONCLUSION: Trans-urethral ultrasound treatment of the prostate with MRI guidance has potential to safely, reliably, and accurately ablate prostatic regions, while minimizing the morbidities associated with conventional whole-gland resection or therapy.
Collapse
|
49
|
Menze BH, Jakab A, Bauer S, Kalpathy-Cramer J, Farahani K, Kirby J, Burren Y, Porz N, Slotboom J, Wiest R, Lanczi L, Gerstner E, Weber MA, Arbel T, Avants BB, Ayache N, Buendia P, Collins DL, Cordier N, Corso JJ, Criminisi A, Das T, Delingette H, Demiralp Ç, Durst CR, Dojat M, Doyle S, Festa J, Forbes F, Geremia E, Glocker B, Golland P, Guo X, Hamamci A, Iftekharuddin KM, Jena R, John NM, Konukoglu E, Lashkari D, Mariz JA, Meier R, Pereira S, Precup D, Price SJ, Raviv TR, Reza SMS, Ryan M, Sarikaya D, Schwartz L, Shin HC, Shotton J, Silva CA, Sousa N, Subbanna NK, Szekely G, Taylor TJ, Thomas OM, Tustison NJ, Unal G, Vasseur F, Wintermark M, Ye DH, Zhao L, Zhao B, Zikic D, Prastawa M, Reyes M, Van Leemput K. The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS). IEEE Trans Med Imaging 2015; 34:1993-2024. [PMID: 25494501 PMCID: PMC4833122 DOI: 10.1109/tmi.2014.2377694] [Citation(s) in RCA: 1616] [Impact Index Per Article: 179.6] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/04/2023]
Abstract
In this paper we report the set-up and results of the Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) organized in conjunction with the MICCAI 2012 and 2013 conferences. Twenty state-of-the-art tumor segmentation algorithms were applied to a set of 65 multi-contrast MR scans of low- and high-grade glioma patients-manually annotated by up to four raters-and to 65 comparable scans generated using tumor image simulation software. Quantitative evaluations revealed considerable disagreement between the human raters in segmenting various tumor sub-regions (Dice scores in the range 74%-85%), illustrating the difficulty of this task. We found that different algorithms worked best for different sub-regions (reaching performance comparable to human inter-rater variability), but that no single algorithm ranked in the top for all sub-regions simultaneously. Fusing several good algorithms using a hierarchical majority vote yielded segmentations that consistently ranked above all individual algorithms, indicating remaining opportunities for further methodological improvements. The BRATS image data and manual annotations continue to be publicly available through an online evaluation system as an ongoing benchmarking resource.
Collapse
|
50
|
Armato SG, Hadjiiski L, Tourassi GD, Drukker K, Giger ML, Li F, Redmond G, Farahani K, Kirby JS, Clarke LP. LUNGx Challenge for computerized lung nodule classification: reflections and lessons learned. J Med Imaging (Bellingham) 2015; 2:020103. [PMID: 26158094 DOI: 10.1117/1.jmi.2.2.020103] [Citation(s) in RCA: 33] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Affiliation(s)
- Samuel G Armato
- University of Chicago , Department of Radiology , MC 2026 , 5841 S. Maryland Avenue , Chicago, Illinois 60637, United States
| | - Lubomir Hadjiiski
- University of Michigan , Department of Radiology , 1500 E. Medical Center Drive , Ann Arbor, Michigan 48109, United States
| | - Georgia D Tourassi
- Biomedical Science and Engineering Center , Health Data Sciences Institute , Oak Ridge National Laboratory , Oak Ridge, Tennessee 37831, United States
| | - Karen Drukker
- University of Chicago , Department of Radiology , MC 2026 , 5841 S. Maryland Avenue , Chicago, Illinois 60637, United States
| | - Maryellen L Giger
- University of Chicago , Department of Radiology , MC 2026 , 5841 S. Maryland Avenue , Chicago, Illinois 60637, United States
| | - Feng Li
- University of Chicago , Department of Radiology , MC 2026 , 5841 S. Maryland Avenue , Chicago, Illinois 60637, United States
| | - George Redmond
- National Cancer Institute , Division of Cancer Treatment and Diagnosis , Cancer Imaging Program , 9609 Medical Center Drive , Bethesda, Maryland 20892, United States
| | - Keyvan Farahani
- National Cancer Institute , Division of Cancer Treatment and Diagnosis , Cancer Imaging Program , 9609 Medical Center Drive , Bethesda, Maryland 20892, United States
| | - Justin S Kirby
- Frederick National Laboratory for Cancer Research , Leidos Biomedical Research, Inc. , Cancer Imaging Program , Frederick, Maryland 21702, United States
| | - Laurence P Clarke
- National Cancer Institute , Division of Cancer Treatment and Diagnosis , Cancer Imaging Program , 9609 Medical Center Drive , Bethesda, Maryland 20892, United States
| |
Collapse
|