2
|
Dayan I, Roth HR, Zhong A, Harouni A, Gentili A, Abidin AZ, Liu A, Costa AB, Wood BJ, Tsai CS, Wang CH, Hsu CN, Lee CK, Ruan P, Xu D, Wu D, Huang E, Kitamura FC, Lacey G, de Antônio Corradi GC, Nino G, Shin HH, Obinata H, Ren H, Crane JC, Tetreault J, Guan J, Garrett JW, Kaggie JD, Park JG, Dreyer K, Juluru K, Kersten K, Rockenbach MABC, Linguraru MG, Haider MA, AbdelMaseeh M, Rieke N, Damasceno PF, E Silva PMC, Wang P, Xu S, Kawano S, Sriswasdi S, Park SY, Grist TM, Buch V, Jantarabenjakul W, Wang W, Tak WY, Li X, Lin X, Kwon YJ, Quraini A, Feng A, Priest AN, Turkbey B, Glicksberg B, Bizzo B, Kim BS, Tor-Díez C, Lee CC, Hsu CJ, Lin C, Lai CL, Hess CP, Compas C, Bhatia D, Oermann EK, Leibovitz E, Sasaki H, Mori H, Yang I, Sohn JH, Murthy KNK, Fu LC, de Mendonça MRF, Fralick M, Kang MK, Adil M, Gangai N, Vateekul P, Elnajjar P, Hickman S, Majumdar S, McLeod SL, Reed S, Gräf S, Harmon S, Kodama T, Puthanakit T, Mazzulli T, de Lavor VL, Rakvongthai Y, Lee YR, Wen Y, Gilbert FJ, Flores MG, Li Q. Federated learning for predicting clinical outcomes in patients with COVID-19. Nat Med 2021; 27:1735-1743. [PMID: 34526699 PMCID: PMC9157510 DOI: 10.1038/s41591-021-01506-3] [Citation(s) in RCA: 152] [Impact Index Per Article: 50.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2020] [Accepted: 08/13/2021] [Indexed: 02/08/2023]
Abstract
Federated learning (FL) is a method used for training artificial intelligence models with data from multiple sources while maintaining data anonymity, thus removing many barriers to data sharing. Here we used data from 20 institutes across the globe to train a FL model, called EXAM (electronic medical record (EMR) chest X-ray AI model), that predicts the future oxygen requirements of symptomatic patients with COVID-19 using inputs of vital signs, laboratory data and chest X-rays. EXAM achieved an average area under the curve (AUC) >0.92 for predicting outcomes at 24 and 72 h from the time of initial presentation to the emergency room, and it provided 16% improvement in average AUC measured across all participating sites and an average increase in generalizability of 38% when compared with models trained at a single site using that site's data. For prediction of mechanical ventilation treatment or death at 24 h at the largest independent test site, EXAM achieved a sensitivity of 0.950 and specificity of 0.882. In this study, FL facilitated rapid data science collaboration without data exchange and generated a model that generalized across heterogeneous, unharmonized datasets for prediction of clinical outcomes in patients with COVID-19, setting the stage for the broader use of FL in healthcare.
Collapse
Affiliation(s)
- Ittai Dayan
- MGH Radiology and Harvard Medical School, Boston, MA, USA
| | | | - Aoxiao Zhong
- Center for Advanced Medical Computing and Analysis, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
- School of Engineering and Applied Sciences, Harvard University, Boston, MA, USA
| | | | | | | | | | | | - Bradford J Wood
- Radiology & Imaging Sciences/Clinical Center, National Institutes of Health, Bethesda, MD, USA
- National Cancer Institute, National Institutes of Health, Bethesda, MD, USA
| | - Chien-Sung Tsai
- Division of Cardiovascular Surgery, Department of Surgery, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan
| | - Chih-Hung Wang
- Department of Otolaryngology-Head and Neck Surgery, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan
- Graduate Institute of Medical Sciences, National Defense Medical Center, Taipei, Taiwan
| | - Chun-Nan Hsu
- Center for Research in Biological Systems, University of California, San Diego, CA, USA
| | - C K Lee
- NVIDIA, Santa Clara, CA, USA
| | | | | | - Dufan Wu
- Center for Advanced Medical Computing and Analysis, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | | | | | | | | | - Gustavo Nino
- Division of Pediatric Pulmonary and Sleep Medicine, Children's National Hospital, Washington, DC, USA
| | - Hao-Hsin Shin
- Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | | | - Hui Ren
- Center for Advanced Medical Computing and Analysis, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Jason C Crane
- Center for Intelligent Imaging, Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA, USA
| | | | | | - John W Garrett
- Departments of Radiology and Medical Physics, The University of Wisconsin-Madison School of Medicine and Public Health, Madison, WI, USA
| | - Joshua D Kaggie
- Department of Radiology, NIHR Cambridge Biomedical Resource Centre, University of Cambridge, Cambridge, UK
| | - Jung Gil Park
- Department of Internal Medicine, Yeungnam University College of Medicine, Daegu, South Korea
| | - Keith Dreyer
- MGH Radiology and Harvard Medical School, Boston, MA, USA
- Center for Clinical Data Science, Massachusetts General Brigham, Boston, MA, USA
| | - Krishna Juluru
- Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | | | | | - Marius George Linguraru
- Sheikh Zayed Institute for Pediatric Surgical Innovation, Children's National Hospital, Washington, DC, USA
- Departments of Radiology and Pediatrics, The George Washington University School of Medicine and Health Sciences, Washington, DC, USA
| | - Masoom A Haider
- Joint Dept. of Medical Imaging, Sinai Health System, University of Toronto, Toronto, Ontario, Canada
- Lunenfeld-Tanenbaum Research Institute, Toronto, Ontario, Canada
| | | | | | - Pablo F Damasceno
- Center for Intelligent Imaging, Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA, USA
| | | | - Pochuan Wang
- MeDA Lab Institute of Applied Mathematical Sciences, National Taiwan University, Taipei, Taiwan
- Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan
| | - Sheng Xu
- Radiology & Imaging Sciences/Clinical Center, National Institutes of Health, Bethesda, MD, USA
- National Cancer Institute, National Institutes of Health, Bethesda, MD, USA
| | | | - Sira Sriswasdi
- Research Affairs, Faculty of Medicine, Chulalongkorn University, Bangkok, Thailand
- Center for Artificial Intelligence in Medicine, Faculty of Medicine, Chulalongkorn University, Bangkok, Thailand
| | - Soo Young Park
- Department of Internal Medicine, School of Medicine, Kyungpook National University, Daegu, South Korea
| | - Thomas M Grist
- Departments of Radiology, Medical Physics, and Biomedical Engineering, The University of Wisconsin-Madison School of Medicine and Public Health, Madison, WI, USA
| | - Varun Buch
- Center for Clinical Data Science, Massachusetts General Brigham, Boston, MA, USA
| | - Watsamon Jantarabenjakul
- Department of Pediatrics, Faculty of Medicine, Chulalongkorn University, Bangkok, Thailand
- Thai Red Cross Emerging Infectious Diseases Clinical Center, King Chulalongkorn Memorial Hospital, Bangkok, Thailand
| | - Weichung Wang
- MeDA Lab Institute of Applied Mathematical Sciences, National Taiwan University, Taipei, Taiwan
- Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan
| | - Won Young Tak
- Department of Internal Medicine, School of Medicine, Kyungpook National University, Daegu, South Korea
| | - Xiang Li
- Center for Advanced Medical Computing and Analysis, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Xihong Lin
- Harvard T.H. Chan School of Public Health, Harvard University, Boston, MA, USA
| | - Young Joon Kwon
- Department of Neurosurgery, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | | | | | - Andrew N Priest
- Department of Radiology, NIHR Cambridge Biomedical Resource Centre, Cambridge University Hospital, Cambridge, UK
| | - Baris Turkbey
- National Cancer Institute, National Institutes of Health, Bethesda, MD, USA
- Department of Radiology and Imaging Sciences, National Institutes of Health, Bethesda, MD, USA
| | - Benjamin Glicksberg
- Hasso Plattner Institute for Digital Health at Mount Sinai, Icahn School of Medicine at Mount Sinai and Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Bernardo Bizzo
- Center for Clinical Data Science, Massachusetts General Brigham, Boston, MA, USA
| | - Byung Seok Kim
- Department of Internal Medicine, Catholic University of Daegu School of Medicine, Daegu, South Korea
| | - Carlos Tor-Díez
- Sheikh Zayed Institute for Pediatric Surgical Innovation, Children's National Hospital, Washington, DC, USA
| | - Chia-Cheng Lee
- Planning and Management Office, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan
| | - Chia-Jung Hsu
- Planning and Management Office, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan
| | - Chin Lin
- School of Medicine, National Defense Medical Center, Taipei, Taiwan
- School of Public Health, National Defense Medical Center, Taipei, Taiwan
- Graduate Institute of Life Sciences, National Defense Medical Center, Taipei, Taiwan
| | - Chiu-Ling Lai
- Medical Review and Pharmaceutical Benefits Division, National Health Insurance Administration, Taipei, Taiwan
| | - Christopher P Hess
- Center for Intelligent Imaging, Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA, USA
| | | | | | - Eric K Oermann
- Department of Neurosurgery, NYU Grossman School of Medicine, New York, NY, USA
| | - Evan Leibovitz
- Center for Clinical Data Science, Massachusetts General Brigham, Boston, MA, USA
| | | | - Hitoshi Mori
- Self-Defense Forces Central Hospital, Tokyo, Japan
| | | | - Jae Ho Sohn
- Center for Intelligent Imaging, Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA, USA
| | | | - Li-Chen Fu
- MOST/NTU All Vista Healthcare Center, Center for Artificial Intelligence and Advanced Robotics, National Taiwan University, Taipei, Taiwan
| | | | - Mike Fralick
- Division of General Internal Medicine and Geriatrics (Fralick), Sinai Health System, Toronto, Ontario, Canada
| | - Min Kyu Kang
- Department of Internal Medicine, Yeungnam University College of Medicine, Daegu, South Korea
| | | | - Natalie Gangai
- Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Peerapon Vateekul
- Department of Computer Engineering, Faculty of Engineering, Chulalongkorn University, Bangkok, Thailand
| | | | - Sarah Hickman
- Department of Radiology, NIHR Cambridge Biomedical Resource Centre, University of Cambridge, Cambridge, UK
| | - Sharmila Majumdar
- Center for Intelligent Imaging, Department of Radiology and Biomedical Imaging, University of California, San Francisco, CA, USA
| | - Shelley L McLeod
- Schwartz/Reisman Emergency Medicine Institute, Sinai Health, Toronto, Ontario, Canada
- Department of Family and Community Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Sheridan Reed
- Radiology & Imaging Sciences/Clinical Center, National Institutes of Health, Bethesda, MD, USA
- National Cancer Institute, National Institutes of Health, Bethesda, MD, USA
| | - Stefan Gräf
- Department of Medicine and NIHR BioResource for Translational Research, NIHR Cambridge Biomedical Research Centre, University of Cambridge, Cambridge, UK
| | - Stephanie Harmon
- National Cancer Institute, National Institutes of Health, Bethesda, MD, USA
- Clinical Research Directorate, Frederick National Laboratory for Cancer, National Cancer Institute, Frederick, MD, USA
| | | | - Thanyawee Puthanakit
- Department of Pediatrics, Faculty of Medicine, Chulalongkorn University, Bangkok, Thailand
- Thai Red Cross Emerging Infectious Diseases Clinical Center, King Chulalongkorn Memorial Hospital, Bangkok, Thailand
| | - Tony Mazzulli
- Department of Microbiology, Sinai Health/University Health Network, Toronto, Ontario, Canada
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Ontario, Canada
- Public Health Ontario Laboratories, Toronto, Ontario, Canada
| | | | - Yothin Rakvongthai
- Chulalongkorn University Biomedical Imaging Group and Division of Nuclear Medicine, Department of Radiology, Faculty of Medicine, Chulalongkorn University, Bangkok, Thailand
| | - Yu Rim Lee
- Department of Internal Medicine, School of Medicine, Kyungpook National University, Daegu, South Korea
| | | | - Fiona J Gilbert
- Department of Radiology, NIHR Cambridge Biomedical Resource Centre, University of Cambridge, Cambridge, UK
| | | | - Quanzheng Li
- Center for Advanced Medical Computing and Analysis, Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| |
Collapse
|