1
|
Liu F, Li Z, Yin Q, Huang J, Luo J, Thakur A, Branson K, Schwab P, Yin B, Wu X, Zheng Y, Clifton DA. A multimodal multidomain multilingual medical foundation model for zero shot clinical diagnosis. NPJ Digit Med 2025; 8:86. [PMID: 39915635 PMCID: PMC11802893 DOI: 10.1038/s41746-024-01339-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2024] [Accepted: 11/11/2024] [Indexed: 02/09/2025] Open
Abstract
Radiology images are one of the most commonly used in daily clinical diagnosis. Typically, clinical diagnosis using radiology images involves disease reporting and classification, where the former is a multimodal task whereby textual reports are generated to describe clinical findings in images, as are common in various domains, e.g., chest X-ray or computed tomography. Existing approaches are mainly supervised, the quality of which heavily depends on the volume and quality of available labeled data. However, for rarer or more novel diseases, enrolling patients to collect data is both time-consuming and expensive. For non-English languages, sufficient quantities of labeled data are typically not available. We propose the Multimodal Multidomain Multilingual Foundation Model. It is useful for rare diseases and non-English languages, where the labeled data are frequently much more scarce, and may even be absent. Our approach achieves encouraging performances on nine datasets, including 2 infectious and 14 non-infectious diseases.
Collapse
Affiliation(s)
- Fenglin Liu
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK.
| | | | | | - Jinfa Huang
- Department of Computer Science, University of Rochester, Rochester, NY, USA
| | - Jiebo Luo
- Department of Computer Science, University of Rochester, Rochester, NY, USA
| | - Anshul Thakur
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK
| | | | | | | | - Xian Wu
- Jarvis Research Center, Tencent YouTu Lab, Beijing, China.
| | - Yefeng Zheng
- Medical Artificial Intelligence Laboratory, Westlake University, Hangzhou, China.
| | - David A Clifton
- Institute of Biomedical Engineering, Department of Engineering Science, University of Oxford, Oxford, UK.
- Oxford-Suzhou Centre for Advanced Research, Suzhou, China.
| |
Collapse
|
2
|
Ngo B, Nguyen D, vanSonnenberg E. Artificial Intelligence: Has Its Time Come for Inclusion in Medical School Education? Maybe…Maybe Not. MEDEDPUBLISH 2021; 10:131. [PMID: 38486566 PMCID: PMC10939546 DOI: 10.15694/mep.2021.000131.2] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/17/2024] Open
Abstract
This article was migrated. The article was marked as recommended. Artificial intelligence (AI) has the potential to strongly modify or even transform the landscape of medicine. Judicious utilization of AI can further propel progress in medical research, facilitate precision medicine, and optimize clinical workflow-the applications are limitless. Although technology and AI algorithms are evolving rapidly and have important implications for future physicians, there is a dearth of literature available for medical students and their educators to learn about AI. While scientific journals provide information regarding AI, they often are written for and by scientists, engineers, and physicians who are well-versed in technology. Currently, medical students must navigate the technical jargon and decipher AI literature without any foundational exposure. It is difficult for students to understand the implications of AI if they do not have basic knowledge of AI and its current capabilities. A fear about AI is pervasive amongst medical students. There is virtually no literature that provides a fundamental and easily digestible overview of AI for medical students and educators, while also offering suggestions about how to integrate AI into medical school curricula. Our goal is to address the lack of formal AI instruction by presenting an informative primer with curricular suggestions for each medical school year, from a U.S. perspective, tailored to medical students and their educators. We seek to present a balanced approach, as there are pros and cons about incorporating AI in undergraduate medical education.
Collapse
Affiliation(s)
- Brandon Ngo
- University of Arizona College of Medicine - Phoenix
| | - Diep Nguyen
- University of Arizona College of Medicine - Phoenix
| | | |
Collapse
|
3
|
Ahmed Z, Mohamed K, Zeeshan S, Dong X. Artificial intelligence with multi-functional machine learning platform development for better healthcare and precision medicine. Database (Oxford) 2020; 2020:baaa010. [PMID: 32185396 PMCID: PMC7078068 DOI: 10.1093/database/baaa010] [Citation(s) in RCA: 234] [Impact Index Per Article: 46.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2019] [Revised: 01/05/2020] [Accepted: 01/21/2020] [Indexed: 02/06/2023]
Abstract
Precision medicine is one of the recent and powerful developments in medical care, which has the potential to improve the traditional symptom-driven practice of medicine, allowing earlier interventions using advanced diagnostics and tailoring better and economically personalized treatments. Identifying the best pathway to personalized and population medicine involves the ability to analyze comprehensive patient information together with broader aspects to monitor and distinguish between sick and relatively healthy people, which will lead to a better understanding of biological indicators that can signal shifts in health. While the complexities of disease at the individual level have made it difficult to utilize healthcare information in clinical decision-making, some of the existing constraints have been greatly minimized by technological advancements. To implement effective precision medicine with enhanced ability to positively impact patient outcomes and provide real-time decision support, it is important to harness the power of electronic health records by integrating disparate data sources and discovering patient-specific patterns of disease progression. Useful analytic tools, technologies, databases, and approaches are required to augment networking and interoperability of clinical, laboratory and public health systems, as well as addressing ethical and social issues related to the privacy and protection of healthcare data with effective balance. Developing multifunctional machine learning platforms for clinical data extraction, aggregation, management and analysis can support clinicians by efficiently stratifying subjects to understand specific scenarios and optimize decision-making. Implementation of artificial intelligence in healthcare is a compelling vision that has the potential in leading to the significant improvements for achieving the goals of providing real-time, better personalized and population medicine at lower costs. In this study, we focused on analyzing and discussing various published artificial intelligence and machine learning solutions, approaches and perspectives, aiming to advance academic solutions in paving the way for a new data-centric era of discovery in healthcare.
Collapse
Affiliation(s)
- Zeeshan Ahmed
- Institute for Health, Health Care Policy and Aging Research, Rutgers, The State University of New Jersey, 112 Paterson Street, New Brunswick, NJ, USA
- Department of Medicine, Rutgers Robert Wood Johnson Medical School, Rutgers Biomedical and Health Sciences, 125 Paterson Street, New Brunswick, NJ, USA
- Department of Genetics and Genome Sciences, School of Medicine, University of Connecticut Health Center, 263 Farmington Ave., Farmington, CT, USA
- Institute for Systems Genomics, University of Connecticut, 67 North Eagleville Road, Storrs, CT, USA
| | - Khalid Mohamed
- Department of Genetics and Genome Sciences, School of Medicine, University of Connecticut Health Center, 263 Farmington Ave., Farmington, CT, USA
| | - Saman Zeeshan
- The Jackson Laboratory for Genomic Medicine, 10 Discovery Drive, Farmington, CT, USA
| | - XinQi Dong
- Institute for Health, Health Care Policy and Aging Research, Rutgers, The State University of New Jersey, 112 Paterson Street, New Brunswick, NJ, USA
- Department of Medicine, Rutgers Robert Wood Johnson Medical School, Rutgers Biomedical and Health Sciences, 125 Paterson Street, New Brunswick, NJ, USA
| |
Collapse
|