1
|
Franco D’Souza R, Mathew M, Mishra V, Surapaneni KM. Twelve tips for addressing ethical concerns in the implementation of artificial intelligence in medical education. MEDICAL EDUCATION ONLINE 2024; 29:2330250. [PMID: 38566608 PMCID: PMC10993743 DOI: 10.1080/10872981.2024.2330250] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/30/2024] [Accepted: 03/08/2024] [Indexed: 04/04/2024]
Abstract
Artificial Intelligence (AI) holds immense potential for revolutionizing medical education and healthcare. Despite its proven benefits, the full integration of AI faces hurdles, with ethical concerns standing out as a key obstacle. Thus, educators should be equipped to address the ethical issues that arise and ensure the seamless integration and sustainability of AI-based interventions. This article presents twelve essential tips for addressing the major ethical concerns in the use of AI in medical education. These include emphasizing transparency, addressing bias, validating content, prioritizing data protection, obtaining informed consent, fostering collaboration, training educators, empowering students, regularly monitoring, establishing accountability, adhering to standard guidelines, and forming an ethics committee to address the issues that arise in the implementation of AI. By adhering to these tips, medical educators and other stakeholders can foster a responsible and ethical integration of AI in medical education, ensuring its long-term success and positive impact.
Collapse
Affiliation(s)
- Russell Franco D’Souza
- Department of Education, UNESCO Chair in Bioethics, Melbourne, Australia
- Department of Organisational Psychological Medicine, International Institute of Organisational Psychological Medicine, Melbourne, Australia
| | - Mary Mathew
- Department of Pathology, Kasturba Medical College, Manipal, Manipal Academy of Higher Education (MAHE), Manipal, India
| | - Vedprakash Mishra
- School of Hogher Education and Research, Datta Meghe Institute of Higher Education and Research (Deemed to be University), Nagpur, India
| | - Krishna Mohan Surapaneni
- Department of Biochemistry, Panimalar Medical College Hospital & Research Institute, Chennai, India
- Department of Medical Education, Panimalar Medical College Hospital & Research Institute, Chennai, India
| |
Collapse
|
2
|
Cunha-Oliveira T, Ioannidis JPA, Oliveira PJ. Best practices for data management and sharing in experimental biomedical research. Physiol Rev 2024; 104:1387-1408. [PMID: 38451234 DOI: 10.1152/physrev.00043.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2023] [Revised: 02/07/2024] [Accepted: 02/29/2024] [Indexed: 03/08/2024] Open
Abstract
Effective data management is crucial for scientific integrity and reproducibility, a cornerstone of scientific progress. Well-organized and well-documented data enable validation and building on results. Data management encompasses activities including organization, documentation, storage, sharing, and preservation. Robust data management establishes credibility, fostering trust within the scientific community and benefiting researchers' careers. In experimental biomedicine, comprehensive data management is vital due to the typically intricate protocols, extensive metadata, and large datasets. Low-throughput experiments, in particular, require careful management to address variations and errors in protocols and raw data quality. Transparent and accountable research practices rely on accurate documentation of procedures, data collection, and analysis methods. Proper data management ensures long-term preservation and accessibility of valuable datasets. Well-managed data can be revisited, contributing to cumulative knowledge and potential new discoveries. Publicly funded research has an added responsibility for transparency, resource allocation, and avoiding redundancy. Meeting funding agency expectations increasingly requires rigorous methodologies, adherence to standards, comprehensive documentation, and widespread sharing of data, code, and other auxiliary resources. This review provides critical insights into raw and processed data, metadata, high-throughput versus low-throughput datasets, a common language for documentation, experimental and reporting guidelines, efficient data management systems, sharing practices, and relevant repositories. We systematically present available resources and optimal practices for wide use by experimental biomedical researchers.
Collapse
Affiliation(s)
- Teresa Cunha-Oliveira
- Center for Neuroscience and Cell Biology, University of Coimbra, Coimbra, Portugal
- Center for Innovative Biomedicine and Biotechnology, University of Coimbra, Coimbra, Portugal
| | - John P A Ioannidis
- Meta-Research Innovation Center at Stanford (METRICS), Stanford, California, United States
- Department of Statistics, Stanford University, Stanford, California, United States
| | - Paulo J Oliveira
- Center for Neuroscience and Cell Biology, University of Coimbra, Coimbra, Portugal
- Center for Innovative Biomedicine and Biotechnology, University of Coimbra, Coimbra, Portugal
| |
Collapse
|
3
|
Tai K, Zhao R, Rameau A. Artificial Intelligence in Otolaryngology: Topics in Epistemology & Ethics. Otolaryngol Clin North Am 2024:S0030-6665(24)00066-5. [PMID: 38839555 DOI: 10.1016/j.otc.2024.04.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/07/2024]
Abstract
To fuel artificial intelligence (AI) potential in clinical practice in otolaryngology, researchers must understand its epistemic limitations, which are tightly linked to ethical dilemmas requiring careful consideration. AI tools are fundamentally opaque systems, though there are methods to increase explainability and transparency. Reproducibility and replicability limitations can be overcomed by sharing computing code, raw data, and data processing methodology. The risk of bias can be mitigated via algorithmic auditing, careful consideration of the training data, and advocating for a diverse AI workforce to promote algorithmic pluralism, reflecting our population's diverse values and preferences.
Collapse
Affiliation(s)
- Katie Tai
- New York Presbyterian Hospital, 1300 York Avenue, New York, NY 10065, USA
| | - Robin Zhao
- Department of Otolaryngology-Head & Neck Surgery, Sean Parker Institute for the Voice, Weill Cornell Medical College, 240 East 59th Street, New York, NY 10022, USA
| | - Anaïs Rameau
- Department of Otolaryngology-Head & Neck Surgery, Sean Parker Institute for the Voice, Weill Cornell Medical College, 240 East 59th Street, New York, NY 10022, USA.
| |
Collapse
|
4
|
Jackson A, Hua CH, Olch A, Yorke ED, Rancati T, Milano MT, Constine LS, Marks LB, Bentzen SM. Reporting Standards for Complication Studies of Radiation Therapy for Pediatric Cancer: Lessons From PENTEC. Int J Radiat Oncol Biol Phys 2024; 119:697-707. [PMID: 38760117 DOI: 10.1016/j.ijrobp.2024.02.018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2023] [Revised: 01/14/2024] [Accepted: 02/08/2024] [Indexed: 05/19/2024]
Abstract
The major aim of Pediatric Normal Tissue Effects in the Clinic (PENTEC) was to synthesize quantitative published dose/-volume/toxicity data in pediatric radiation therapy. Such systematic reviews are often challenging because of the lack of standardization and difficulty of reporting outcomes, clinical factors, and treatment details in journal articles. This has clinical consequences: optimization of treatment plans must balance between the risks of toxicity and local failure; counseling patients and their parents requires knowledge of the excess risks encountered after a specific treatment. Studies addressing outcomes after pediatric radiation therapy are particularly challenging because: (a) survivors may live for decades after treatment, and the latency time to toxicity can be very long; (b) children's maturation can be affected by radiation, depending on the developmental status of the organs involved at time of treatment; and (c) treatment regimens frequently involve chemotherapies, possibly modifying and adding to the toxicity of radiation. Here we discuss: basic reporting strategies to account for the actuarial nature of the complications; the reporting of modeling of abnormal development; and the need for standardized, comprehensively reported data sets and multivariate models (ie, accounting for the simultaneous effects of radiation dose, age, developmental status at time of treatment, and chemotherapy dose). We encourage the use of tools that facilitate comprehensive reporting, for example, electronic supplements for journal articles. Finally, we stress the need for clinicians to be able to trust artificial intelligence models of outcome of radiation therapy, which requires transparency, rigor, reproducibility, and comprehensive reporting. Adopting the reporting methods discussed here and in the individual PENTEC articles will increase the clinical and scientific usefulness of individual reports and associated pooled analyses.
Collapse
Affiliation(s)
- Andrew Jackson
- Department of Medical Physics, Memorial Sloan-Kettering Cancer Center, New York, New York.
| | - Chia-Ho Hua
- Department of Radiation Oncology, St. Jude Children's Research Hospital, Memphis, Tennessee
| | - Arthur Olch
- Radiation Oncology Department, University of Southern California and Children's Hospital, Los Angeles, California
| | - Ellen D Yorke
- Department of Medical Physics, Memorial Sloan-Kettering Cancer Center, New York, New York
| | - Tiziana Rancati
- Data Science Unit, Fondazione IRCCS Istituto Nazionale dei Tumori, Milan, Italy
| | - Michael T Milano
- Department of Radiation Oncology, University of Rochester Medical Center, Wilmot Cancer Institute, Rochester, New York
| | - Louis S Constine
- Department of Radiation Oncology, University of Rochester Medical Center, Wilmot Cancer Institute, Rochester, New York; Pediatrics, University of Rochester Medical Center, Wilmot Cancer Institute, Rochester, New York
| | - Lawrence B Marks
- Department of Radiation Oncology and Lineberger Cancer Center, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| | - Soren M Bentzen
- Department of Epidemiology and Public Health, University of Maryland, Baltimore, Maryland
| |
Collapse
|
5
|
Kapoor S, Cantrell EM, Peng K, Pham TH, Bail CA, Gundersen OE, Hofman JM, Hullman J, Lones MA, Malik MM, Nanayakkara P, Poldrack RA, Raji ID, Roberts M, Salganik MJ, Serra-Garcia M, Stewart BM, Vandewiele G, Narayanan A. REFORMS: Consensus-based Recommendations for Machine-learning-based Science. SCIENCE ADVANCES 2024; 10:eadk3452. [PMID: 38691601 PMCID: PMC11092361 DOI: 10.1126/sciadv.adk3452] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Accepted: 03/29/2024] [Indexed: 05/03/2024]
Abstract
Machine learning (ML) methods are proliferating in scientific research. However, the adoption of these methods has been accompanied by failures of validity, reproducibility, and generalizability. These failures can hinder scientific progress, lead to false consensus around invalid claims, and undermine the credibility of ML-based science. ML methods are often applied and fail in similar ways across disciplines. Motivated by this observation, our goal is to provide clear recommendations for conducting and reporting ML-based science. Drawing from an extensive review of past literature, we present the REFORMS checklist (recommendations for machine-learning-based science). It consists of 32 questions and a paired set of guidelines. REFORMS was developed on the basis of a consensus of 19 researchers across computer science, data science, mathematics, social sciences, and biomedical sciences. REFORMS can serve as a resource for researchers when designing and implementing a study, for referees when reviewing papers, and for journals when enforcing standards for transparency and reproducibility.
Collapse
Affiliation(s)
- Sayash Kapoor
- Department of Computer Science, Princeton University, Princeton, NJ 08544, USA
- Center for Information Technology Policy, Princeton University, Princeton, NJ 08544, USA
| | - Emily M. Cantrell
- Department of Sociology, Princeton University, Princeton, NJ 08544, USA
- School of Public and International Affairs, Princeton University, Princeton, NJ 08544, USA
| | - Kenny Peng
- Department of Computer Science, Cornell University, Ithaca, NY 14850, USA
| | - Thanh Hien Pham
- Department of Computer Science, Princeton University, Princeton, NJ 08544, USA
- Center for Information Technology Policy, Princeton University, Princeton, NJ 08544, USA
| | - Christopher A. Bail
- Department of Sociology, Duke University, Durham, NC 27708, USA
- Department of Political Science, Duke University, Durham, NC 27708, USA
- Sanford School of Public Policy, Duke University, Durham, NC 27708, USA
| | - Odd Erik Gundersen
- Department of Computer Science, Norwegian University of Science and Technology, Trondheim, Norway
- Aneo AS, Trondheim, Norway
| | | | - Jessica Hullman
- Department of Computer Science, Northwestern University, Evanston, IL 60208, USA
| | - Michael A. Lones
- School of Mathematical and Computer Sciences, Heriot-Watt University, Edinburgh, UK
| | - Momin M. Malik
- Center for Digital Health, Mayo Clinic, Rochester, MN 55905, USA
- School of Social Policy & Practice, University of Pennsylvania, Philadelphia, PA 19104, USA
- Institute in Critical Quantitative, Computational, & Mixed Methodologies, Johns Hopkins University, Baltimore, MD 21218, USA
| | - Priyanka Nanayakkara
- Department of Computer Science, Northwestern University, Evanston, IL 60208, USA
- Department of Communication Studies, Northwestern University, Evanston, IL 60208, USA
| | | | - Inioluwa Deborah Raji
- Department of Computer Science, University of California, Berkeley, Berkeley, CA 94720, USA
| | - Michael Roberts
- Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, UK
- Department of Medicine, University of Cambridge, Cambridge, UK
| | - Matthew J. Salganik
- Center for Information Technology Policy, Princeton University, Princeton, NJ 08544, USA
- Department of Sociology, Princeton University, Princeton, NJ 08544, USA
- Office of Population Research, Princeton University, Princeton, NJ 08544, USA
| | - Marta Serra-Garcia
- Rady School of Management, University of California, San Diego, La Jolla, CA 92093, USA
| | - Brandon M. Stewart
- Center for Information Technology Policy, Princeton University, Princeton, NJ 08544, USA
- Department of Sociology, Princeton University, Princeton, NJ 08544, USA
- Office of Population Research, Princeton University, Princeton, NJ 08544, USA
- Department of Politics, Princeton University, Princeton, NJ 08544, USA
| | - Gilles Vandewiele
- Department of Information Technology, Ghent University, Ghent, Belgium
| | - Arvind Narayanan
- Department of Computer Science, Princeton University, Princeton, NJ 08544, USA
- Center for Information Technology Policy, Princeton University, Princeton, NJ 08544, USA
| |
Collapse
|
6
|
Lotter W, Hassett MJ, Schultz N, Kehl KL, Van Allen EM, Cerami E. Artificial Intelligence in Oncology: Current Landscape, Challenges, and Future Directions. Cancer Discov 2024; 14:711-726. [PMID: 38597966 PMCID: PMC11131133 DOI: 10.1158/2159-8290.cd-23-1199] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2023] [Revised: 01/29/2024] [Accepted: 02/28/2024] [Indexed: 04/11/2024]
Abstract
Artificial intelligence (AI) in oncology is advancing beyond algorithm development to integration into clinical practice. This review describes the current state of the field, with a specific focus on clinical integration. AI applications are structured according to cancer type and clinical domain, focusing on the four most common cancers and tasks of detection, diagnosis, and treatment. These applications encompass various data modalities, including imaging, genomics, and medical records. We conclude with a summary of existing challenges, evolving solutions, and potential future directions for the field. SIGNIFICANCE AI is increasingly being applied to all aspects of oncology, where several applications are maturing beyond research and development to direct clinical integration. This review summarizes the current state of the field through the lens of clinical translation along the clinical care continuum. Emerging areas are also highlighted, along with common challenges, evolving solutions, and potential future directions for the field.
Collapse
Affiliation(s)
- William Lotter
- Department of Data Science, Dana-Farber Cancer Institute, Boston, MA, USA
- Department of Pathology, Brigham and Women’s Hospital, Boston, MA, USA
- Harvard Medical School, Boston, MA, USA
| | - Michael J. Hassett
- Harvard Medical School, Boston, MA, USA
- Division of Population Sciences, Dana-Farber Cancer Institute, Boston, MA, USA
- Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Nikolaus Schultz
- Marie-Josée and Henry R. Kravis Center for Molecular Oncology, Memorial Sloan Kettering Cancer Center; New York, NY, USA
- Department of Epidemiology and Biostatistics, Memorial Sloan Kettering Cancer Center, New York, NY, USA
| | - Kenneth L. Kehl
- Harvard Medical School, Boston, MA, USA
- Division of Population Sciences, Dana-Farber Cancer Institute, Boston, MA, USA
- Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA, USA
| | - Eliezer M. Van Allen
- Harvard Medical School, Boston, MA, USA
- Division of Population Sciences, Dana-Farber Cancer Institute, Boston, MA, USA
- Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA, USA
- Cancer Program, Broad Institute of MIT and Harvard, Cambridge, MA, USA
| | - Ethan Cerami
- Department of Data Science, Dana-Farber Cancer Institute, Boston, MA, USA
- Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, MA, USA
| |
Collapse
|
7
|
Zhang W, Song LN, You YF, Qi FN, Cui XH, Yi MX, Zhu G, Chang RA, Zhang HJ. Application of artificial intelligence in the prediction of immunotherapy efficacy in hepatocellular carcinoma: Current status and prospects. Artif Intell Gastroenterol 2024; 5:90096. [DOI: 10.35712/aig.v5.i1.90096] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/23/2023] [Revised: 01/28/2024] [Accepted: 03/12/2024] [Indexed: 04/29/2024] Open
Abstract
Artificial Intelligence (AI) has increased as a potent tool in medicine, with promising oncology applications. The emergence of immunotherapy has transformed the treatment terrain for hepatocellular carcinoma (HCC), offering new hope to patients with this challenging malignancy. This article examines the role and future of AI in forecasting the effectiveness of immunotherapy in HCC. We highlight the potential of AI to revolutionize the prediction of therapy response, thus improving patient selection and clinical outcomes. The article further outlines the challenges and future research directions in this emerging field.
Collapse
Affiliation(s)
- Wei Zhang
- Research Center of Clinical Medicine and Department of General Surgery, The Affiliated Hospital of Nantong University, Nantong 226001, Jiangsu Province, China
| | - Li-Ning Song
- Research Center of Clinical Medicine and Department of General Surgery, The Affiliated Hospital of Nantong University, Nantong 226001, Jiangsu Province, China
| | - Yun-Fei You
- Research Center of Clinical Medicine and Department of General Surgery, The Affiliated Hospital of Nantong University, Nantong 226001, Jiangsu Province, China
| | - Feng-Nan Qi
- Research Center of Clinical Medicine and Department of General Surgery, The Affiliated Hospital of Nantong University, Nantong 226001, Jiangsu Province, China
| | - Xiao-Hong Cui
- Department of General Surgery, Shanghai Electric Power Hospital, Shanghai 200050, China
| | - Ming-Xun Yi
- Research Center of Clinical Medicine and Department of General Surgery, The Affiliated Hospital of Nantong University, Nantong 226001, Jiangsu Province, China
| | - Guang Zhu
- Division of Life Science, The Hong Kong University of Science and Technology, Hong Kong, China
| | - Ren-An Chang
- Research Center of Clinical Medicine and Department of General Surgery, The Affiliated Hospital of Nantong University, Nantong 226001, Jiangsu Province, China
| | - Hai-Jian Zhang
- Division of Life Science, The Hong Kong University of Science and Technology, Hong Kong, China
- Research Center of Clinical Medicine, The Affiliated Hospital of Nantong University, Nantong 226001, Jiangsu Province, China
| |
Collapse
|
8
|
Yan Y, Huang X, Jiang X, Gao Z, Liu X, Jin K, Ye J. Clinical evaluation of deep learning systems for assisting in the diagnosis of the epiretinal membrane grade in general ophthalmologists. Eye (Lond) 2024; 38:730-736. [PMID: 37848677 PMCID: PMC10920879 DOI: 10.1038/s41433-023-02765-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2023] [Revised: 08/28/2023] [Accepted: 09/18/2023] [Indexed: 10/19/2023] Open
Abstract
BACKGROUND Epiretinal membrane (ERM) is a common age-related retinal disease detected by optical coherence tomography (OCT), with a prevalence of 34.1% among people over 60 years old. This study aims to develop artificial intelligence (AI) systems to assist in the diagnosis of ERM grade using OCT images and to clinically evaluate the potential benefits and risks of our AI systems with a comparative experiment. METHODS A segmentation deep learning (DL) model that segments retinal features associated with ERM severity and a classification DL model that grades the severity of ERM were developed based on an OCT dataset obtained from three hospitals. A comparative experiment was conducted to compare the performance of four general ophthalmologists with and without assistance from the AI in diagnosing ERM severity. RESULTS The segmentation network had a pixel accuracy (PA) of 0.980 and a mean intersection over union (MIoU) of 0.873, while the six-classification network had a total accuracy of 81.3%. The diagnostic accuracy scores of the four ophthalmologists increased with AI assistance from 81.7%, 80.7%, 78.0%, and 80.7% to 87.7%, 86.7%, 89.0%, and 91.3%, respectively, while the corresponding time expenditures were reduced. The specific results of the study as well as the misinterpretations of the AI systems were analysed. CONCLUSION Through our comparative experiment, the AI systems proved to be valuable references for medical diagnosis and demonstrated the potential to accelerate clinical workflows. Systematic efforts are needed to ensure the safe and rapid integration of AI systems into ophthalmic practice.
Collapse
Affiliation(s)
- Yan Yan
- Eye Center, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, 310009, China
| | - Xiaoling Huang
- Eye Center, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, 310009, China
| | - Xiaoyu Jiang
- College of Control Science and Engineering, Zhejiang University, Hangzhou, 310027, China
| | - Zhiyuan Gao
- Eye Center, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, 310009, China
| | - Xindi Liu
- Eye Center, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, 310009, China
| | - Kai Jin
- Eye Center, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, 310009, China.
| | - Juan Ye
- Eye Center, The Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, 310009, China.
| |
Collapse
|
9
|
Wei Y, Wang H, Chen Z, Zhu Y, Li Y, Lu B, Pan K, Wen C, Cao G, He Y, Zhou J, Pan Z, Wang M. Deep Learning-Based Multiparametric MRI Model for Preoperative T-Stage in Rectal Cancer. J Magn Reson Imaging 2024; 59:1083-1092. [PMID: 37367938 DOI: 10.1002/jmri.28856] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2023] [Revised: 05/24/2023] [Accepted: 05/29/2023] [Indexed: 06/28/2023] Open
Abstract
BACKGROUND Conventional MRI staging can be challenging in the preoperative assessment of rectal cancer. Deep learning methods based on MRI have shown promise in cancer diagnosis and prognostication. However, the value of deep learning in rectal cancer T-staging is unclear. PURPOSE To develop a deep learning model based on preoperative multiparametric MRI for evaluation of rectal cancer and to investigate its potential to improve T-staging accuracy. STUDY TYPE Retrospective. POPULATION After cross-validation, 260 patients (123 with T-stage T1-2 and 134 with T-stage T3-4) with histopathologically confirmed rectal cancer were randomly divided to the training (N = 208) and test sets (N = 52). FIELD STRENGTH/SEQUENCE 3.0 T/Dynamic contrast enhanced (DCE), T2-weighted imaging (T2W), and diffusion-weighted imaging (DWI). ASSESSMENT The deep learning (DL) model of multiparametric (DCE, T2W, and DWI) convolutional neural network were constructed for evaluating preoperative diagnosis. The pathological findings served as the reference standard for T-stage. For comparison, the single parameter DL-model, a logistic regression model composed of clinical features and subjective assessment of radiologists were used. STATISTICAL TESTS The receiver operating characteristic curve (ROC) was used to evaluate the models, the Fleiss' kappa for the intercorrelation coefficients, and DeLong test for compare the diagnostic performance of ROCs. P-values less than 0.05 were considered statistically significant. RESULTS The Area Under Curve (AUC) of the multiparametric DL-model was 0.854, which was significantly higher than the radiologist's assessment (AUC = 0.678), clinical model (AUC = 0.747), and the single parameter DL-models including T2W-model (AUC = 0.735), DWI-model (AUC = 0.759), and DCE-model (AUC = 0.789). DATA CONCLUSION In the evaluation of rectal cancer patients, the proposed multiparametric DL-model outperformed the radiologist's assessment, the clinical model as well as the single parameter models. The multiparametric DL-model has the potential to assist clinicians by providing more reliable and precise preoperative T staging diagnosis. EVIDENCE LEVEL 3 TECHNICAL EFFICACY: Stage 2.
Collapse
Affiliation(s)
- Yaru Wei
- Department of Radiology, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, China
- School of Laboratory Medicine and Life Sciences, Wenzhou Medical University, Wenzhou, China
| | - Haojie Wang
- Cixi Biomedical Research Institute, Wenzhou Medical University, Zhejiang, China
- Key Laboratory of Intelligent Medical Imaging of Wenzhou, First Affiliated Hospital of Wenzhou Medical University, Wenzhou, China
| | - Zhongwei Chen
- Department of Radiology, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, China
| | - Ying Zhu
- Department of Radiology, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, China
| | - Yingfa Li
- Key Laboratory of Intelligent Medical Imaging of Wenzhou, First Affiliated Hospital of Wenzhou Medical University, Wenzhou, China
| | - Beichen Lu
- Key Laboratory of Intelligent Medical Imaging of Wenzhou, First Affiliated Hospital of Wenzhou Medical University, Wenzhou, China
| | - Kehua Pan
- Department of Radiology, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, China
| | - Caiyun Wen
- Department of Radiology, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, China
| | - Guoquan Cao
- Department of Radiology, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, China
| | - Yun He
- Department of Radiology, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, China
| | - Jiejie Zhou
- Department of Radiology, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, China
| | - Zhifang Pan
- Key Laboratory of Intelligent Medical Imaging of Wenzhou, First Affiliated Hospital of Wenzhou Medical University, Wenzhou, China
| | - Meihao Wang
- Department of Radiology, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, China
- School of Laboratory Medicine and Life Sciences, Wenzhou Medical University, Wenzhou, China
| |
Collapse
|
10
|
Zhu Y, Liu W, Zhang L, Hu B. Endoscopic measurement of lesion size: An unmet clinical need. Chin Med J (Engl) 2024; 137:379-381. [PMID: 38053310 PMCID: PMC10876249 DOI: 10.1097/cm9.0000000000002882] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2023] [Indexed: 12/07/2023] Open
Affiliation(s)
| | | | | | - Bing Hu
- Department of Gastroenterology, West China Hospital, Sichuan University, Chengdu, Sichuan 610041, China
| |
Collapse
|
11
|
Fehr J, Citro B, Malpani R, Lippert C, Madai VI. A trustworthy AI reality-check: the lack of transparency of artificial intelligence products in healthcare. Front Digit Health 2024; 6:1267290. [PMID: 38455991 PMCID: PMC10919164 DOI: 10.3389/fdgth.2024.1267290] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2023] [Accepted: 02/05/2024] [Indexed: 03/09/2024] Open
Abstract
Trustworthy medical AI requires transparency about the development and testing of underlying algorithms to identify biases and communicate potential risks of harm. Abundant guidance exists on how to achieve transparency for medical AI products, but it is unclear whether publicly available information adequately informs about their risks. To assess this, we retrieved public documentation on the 14 available CE-certified AI-based radiology products of the II b risk category in the EU from vendor websites, scientific publications, and the European EUDAMED database. Using a self-designed survey, we reported on their development, validation, ethical considerations, and deployment caveats, according to trustworthy AI guidelines. We scored each question with either 0, 0.5, or 1, to rate if the required information was "unavailable", "partially available," or "fully available." The transparency of each product was calculated relative to all 55 questions. Transparency scores ranged from 6.4% to 60.9%, with a median of 29.1%. Major transparency gaps included missing documentation on training data, ethical considerations, and limitations for deployment. Ethical aspects like consent, safety monitoring, and GDPR-compliance were rarely documented. Furthermore, deployment caveats for different demographics and medical settings were scarce. In conclusion, public documentation of authorized medical AI products in Europe lacks sufficient public transparency to inform about safety and risks. We call on lawmakers and regulators to establish legally mandated requirements for public and substantive transparency to fulfill the promise of trustworthy AI for health.
Collapse
Affiliation(s)
- Jana Fehr
- Digital Health & Machine Learning, Hasso Plattner Institute, Potsdam, Germany
- Digital Engineering Faculty, University of Potsdam, Potsdam, Germany
- QUEST Center for Responsible Research, Berlin Institute of Health (BIH), Charité Universitätsmedizin Berlin, Berlin, Germany
| | - Brian Citro
- Independent Researcher, Chicago, IL, United States
| | | | - Christoph Lippert
- Digital Health & Machine Learning, Hasso Plattner Institute, Potsdam, Germany
- Digital Engineering Faculty, University of Potsdam, Potsdam, Germany
- Hasso Plattner Institute for Digital Health at Mount Sinai, Icahn School of Medicine at Mount Sinai, New York, NY, United States
| | - Vince I. Madai
- QUEST Center for Responsible Research, Berlin Institute of Health (BIH), Charité Universitätsmedizin Berlin, Berlin, Germany
- Faculty of Computing, Engineering and the Built Environment, School of Computing and Digital Technology, Birmingham City University, Birmingham, United Kingdom
| |
Collapse
|
12
|
Christopher M, Gonzalez R, Huynh J, Walker E, Radha Saseendrakumar B, Bowd C, Belghith A, Goldbaum MH, Fazio MA, Girkin CA, De Moraes CG, Liebmann JM, Weinreb RN, Baxter SL, Zangwill LM. Proactive Decision Support for Glaucoma Treatment: Predicting Surgical Interventions with Clinically Available Data. Bioengineering (Basel) 2024; 11:140. [PMID: 38391627 PMCID: PMC10886033 DOI: 10.3390/bioengineering11020140] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2023] [Revised: 01/06/2024] [Accepted: 01/27/2024] [Indexed: 02/24/2024] Open
Abstract
A longitudinal ophthalmic dataset was used to investigate multi-modal machine learning (ML) models incorporating patient demographics and history, clinical measurements, optical coherence tomography (OCT), and visual field (VF) testing in predicting glaucoma surgical interventions. The cohort included 369 patients who underwent glaucoma surgery and 592 patients who did not undergo surgery. The data types used for prediction included patient demographics, history of systemic conditions, medication history, ophthalmic measurements, 24-2 VF results, and thickness measurements from OCT imaging. The ML models were trained to predict surgical interventions and evaluated on independent data collected at a separate study site. The models were evaluated based on their ability to predict surgeries at varying lengths of time prior to surgical intervention. The highest performing predictions achieved an AUC of 0.93, 0.92, and 0.93 in predicting surgical intervention at 1 year, 2 years, and 3 years, respectively. The models were also able to achieve high sensitivity (0.89, 0.77, 0.86 at 1, 2, and 3 years, respectively) and specificity (0.85, 0.90, and 0.91 at 1, 2, and 3 years, respectively) at an 0.80 level of precision. The multi-modal models trained on a combination of data types predicted surgical interventions with high accuracy up to three years prior to surgery and could provide an important tool to predict the need for glaucoma intervention.
Collapse
Affiliation(s)
- Mark Christopher
- Hamilton Glaucoma Center and Division of Ophthalmology Informatics and Data Science, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, CA 92037, USA
| | - Ruben Gonzalez
- Hamilton Glaucoma Center and Division of Ophthalmology Informatics and Data Science, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, CA 92037, USA
| | - Justin Huynh
- Hamilton Glaucoma Center and Division of Ophthalmology Informatics and Data Science, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, CA 92037, USA
| | - Evan Walker
- Hamilton Glaucoma Center and Division of Ophthalmology Informatics and Data Science, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, CA 92037, USA
| | - Bharanidharan Radha Saseendrakumar
- Hamilton Glaucoma Center and Division of Ophthalmology Informatics and Data Science, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, CA 92037, USA
| | - Christopher Bowd
- Hamilton Glaucoma Center and Division of Ophthalmology Informatics and Data Science, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, CA 92037, USA
| | - Akram Belghith
- Hamilton Glaucoma Center and Division of Ophthalmology Informatics and Data Science, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, CA 92037, USA
| | - Michael H Goldbaum
- Hamilton Glaucoma Center and Division of Ophthalmology Informatics and Data Science, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, CA 92037, USA
| | - Massimo A Fazio
- Department of Ophthalmology and Vision Sciences, Heersink School of Medicine, University of Alabama at Birmingham, Birmingham, AL 35233, USA
| | - Christopher A Girkin
- Department of Ophthalmology and Vision Sciences, Heersink School of Medicine, University of Alabama at Birmingham, Birmingham, AL 35233, USA
| | - Carlos Gustavo De Moraes
- Bernard and Shirlee Brown Glaucoma Research Laboratory, Department of Ophthalmology, Edward S. Harkness Eye Institute, Columbia University Medical Center, New York, NY 10032, USA
| | - Jeffrey M Liebmann
- Bernard and Shirlee Brown Glaucoma Research Laboratory, Department of Ophthalmology, Edward S. Harkness Eye Institute, Columbia University Medical Center, New York, NY 10032, USA
| | - Robert N Weinreb
- Hamilton Glaucoma Center and Division of Ophthalmology Informatics and Data Science, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, CA 92037, USA
| | - Sally L Baxter
- Hamilton Glaucoma Center and Division of Ophthalmology Informatics and Data Science, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, CA 92037, USA
| | - Linda M Zangwill
- Hamilton Glaucoma Center and Division of Ophthalmology Informatics and Data Science, Shiley Eye Institute, Viterbi Family Department of Ophthalmology, University of California, San Diego, CA 92037, USA
| |
Collapse
|
13
|
Barberis A, Aerts HJWL, Buffa FM. Robustness and reproducibility for AI learning in biomedical sciences: RENOIR. Sci Rep 2024; 14:1933. [PMID: 38253545 PMCID: PMC10810363 DOI: 10.1038/s41598-024-51381-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Accepted: 01/04/2024] [Indexed: 01/24/2024] Open
Abstract
Artificial intelligence (AI) techniques are increasingly applied across various domains, favoured by the growing acquisition and public availability of large, complex datasets. Despite this trend, AI publications often suffer from lack of reproducibility and poor generalisation of findings, undermining scientific value and contributing to global research waste. To address these issues and focusing on the learning aspect of the AI field, we present RENOIR (REpeated random sampliNg fOr machIne leaRning), a modular open-source platform for robust and reproducible machine learning (ML) analysis. RENOIR adopts standardised pipelines for model training and testing, introducing elements of novelty, such as the dependence of the performance of the algorithm on the sample size. Additionally, RENOIR offers automated generation of transparent and usable reports, aiming to enhance the quality and reproducibility of AI studies. To demonstrate the versatility of our tool, we applied it to benchmark datasets from health, computer science, and STEM (Science, Technology, Engineering, and Mathematics) domains. Furthermore, we showcase RENOIR's successful application in recently published studies, where it identified classifiers for SET2D and TP53 mutation status in cancer. Finally, we present a use case where RENOIR was employed to address a significant pharmacological challenge-predicting drug efficacy. RENOIR is freely available at https://github.com/alebarberis/renoir .
Collapse
Affiliation(s)
- Alessandro Barberis
- Nuffield Department of Surgical Sciences, Medical Sciences Division, University of Oxford, Old Road Campus Research Building, Roosevelt Drive, Oxford, OX3 7DQ, UK.
- Computational Biology and Integrative Genomics Lab, Department of Oncology, Medical Sciences Division, University of Oxford, Oxford, OX3 7DQ, UK.
| | - Hugo J W L Aerts
- Artificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, MA, USA
- Radiation Oncology and Radiology, Dana-Farber Cancer Institute, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
- Radiology and Nuclear Medicine, GROW & CARIM, Maastricht University, Maastricht, The Netherlands
- Cardiovascular Imaging Research Center, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Francesca M Buffa
- Computational Biology and Integrative Genomics Lab, Department of Oncology, Medical Sciences Division, University of Oxford, Oxford, OX3 7DQ, UK.
- AI and Systems Biology, IFOM ETS, 20139, Milan, Italy.
- Department of Computing Sciences and Bocconi Institute for Data Science and Analytics (BIDSA), Bocconi University, 20100, Milan, Italy.
| |
Collapse
|
14
|
Tripathi S, Tabari A, Mansur A, Dabbara H, Bridge CP, Daye D. From Machine Learning to Patient Outcomes: A Comprehensive Review of AI in Pancreatic Cancer. Diagnostics (Basel) 2024; 14:174. [PMID: 38248051 PMCID: PMC10814554 DOI: 10.3390/diagnostics14020174] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2023] [Revised: 12/28/2023] [Accepted: 12/29/2023] [Indexed: 01/23/2024] Open
Abstract
Pancreatic cancer is a highly aggressive and difficult-to-detect cancer with a poor prognosis. Late diagnosis is common due to a lack of early symptoms, specific markers, and the challenging location of the pancreas. Imaging technologies have improved diagnosis, but there is still room for improvement in standardizing guidelines. Biopsies and histopathological analysis are challenging due to tumor heterogeneity. Artificial Intelligence (AI) revolutionizes healthcare by improving diagnosis, treatment, and patient care. AI algorithms can analyze medical images with precision, aiding in early disease detection. AI also plays a role in personalized medicine by analyzing patient data to tailor treatment plans. It streamlines administrative tasks, such as medical coding and documentation, and provides patient assistance through AI chatbots. However, challenges include data privacy, security, and ethical considerations. This review article focuses on the potential of AI in transforming pancreatic cancer care, offering improved diagnostics, personalized treatments, and operational efficiency, leading to better patient outcomes.
Collapse
Affiliation(s)
- Satvik Tripathi
- Department of Radiology, Massachusetts General Hospital, Boston, MA 02114, USA; (S.T.); (A.T.); (A.M.); (C.P.B.)
- Athinoula A. Martinos Center for Biomedical Imaging, Charlestown, MA 02129, USA
- Harvard Medical School, Boston, MA 02115, USA
| | - Azadeh Tabari
- Department of Radiology, Massachusetts General Hospital, Boston, MA 02114, USA; (S.T.); (A.T.); (A.M.); (C.P.B.)
- Harvard Medical School, Boston, MA 02115, USA
| | - Arian Mansur
- Department of Radiology, Massachusetts General Hospital, Boston, MA 02114, USA; (S.T.); (A.T.); (A.M.); (C.P.B.)
- Harvard Medical School, Boston, MA 02115, USA
| | - Harika Dabbara
- Boston University Chobanian & Avedisian School of Medicine, Boston, MA 02118, USA;
| | - Christopher P. Bridge
- Department of Radiology, Massachusetts General Hospital, Boston, MA 02114, USA; (S.T.); (A.T.); (A.M.); (C.P.B.)
- Athinoula A. Martinos Center for Biomedical Imaging, Charlestown, MA 02129, USA
- Harvard Medical School, Boston, MA 02115, USA
| | - Dania Daye
- Department of Radiology, Massachusetts General Hospital, Boston, MA 02114, USA; (S.T.); (A.T.); (A.M.); (C.P.B.)
- Athinoula A. Martinos Center for Biomedical Imaging, Charlestown, MA 02129, USA
- Harvard Medical School, Boston, MA 02115, USA
| |
Collapse
|
15
|
Cau R, Pisu F, Suri JS, Montisci R, Gatti M, Mannelli L, Gong X, Saba L. Artificial Intelligence in the Differential Diagnosis of Cardiomyopathy Phenotypes. Diagnostics (Basel) 2024; 14:156. [PMID: 38248033 DOI: 10.3390/diagnostics14020156] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2023] [Revised: 01/03/2024] [Accepted: 01/08/2024] [Indexed: 01/23/2024] Open
Abstract
Artificial intelligence (AI) is rapidly being applied to the medical field, especially in the cardiovascular domain. AI approaches have demonstrated their applicability in the detection, diagnosis, and management of several cardiovascular diseases, enhancing disease stratification and typing. Cardiomyopathies are a leading cause of heart failure and life-threatening ventricular arrhythmias. Identifying the etiologies is fundamental for the management and diagnostic pathway of these heart muscle diseases, requiring the integration of various data, including personal and family history, clinical examination, electrocardiography, and laboratory investigations, as well as multimodality imaging, making the clinical diagnosis challenging. In this scenario, AI has demonstrated its capability to capture subtle connections from a multitude of multiparametric datasets, enabling the discovery of hidden relationships in data and handling more complex tasks than traditional methods. This review aims to present a comprehensive overview of the main concepts related to AI and its subset. Additionally, we review the existing literature on AI-based models in the differential diagnosis of cardiomyopathy phenotypes, and we finally examine the advantages and limitations of these AI approaches.
Collapse
Affiliation(s)
- Riccardo Cau
- Department of Radiology, Azienda Ospedaliero Universitaria (A.O.U.), di Cagliari-Polo di Monserrato s.s. 554 Monserrato, 09045 Cagliari, Italy
| | - Francesco Pisu
- Department of Radiology, Azienda Ospedaliero Universitaria (A.O.U.), di Cagliari-Polo di Monserrato s.s. 554 Monserrato, 09045 Cagliari, Italy
| | - Jasjit S Suri
- Stroke Monitoring and Diagnostic Division, AtheroPoin™, Roseville, CA 95661, USA
| | - Roberta Montisci
- Department of Cardiology, Azienda Ospedaliero Universitaria (A.O.U.), di Cagliari-Polo di Monserrato s.s. 554 Monserrato, 09045 Cagliari, Italy
| | - Marco Gatti
- Department of Radiology, Università degli Studi di Torino, 10129 Turin, Italy
| | | | - Xiangyang Gong
- Radiology Department, Zhejiang Provincial People's Hospital, Affiliated People's Hospital, Hangzhou Medical College, Hangzhou 310014, China
| | - Luca Saba
- Department of Radiology, Azienda Ospedaliero Universitaria (A.O.U.), di Cagliari-Polo di Monserrato s.s. 554 Monserrato, 09045 Cagliari, Italy
| |
Collapse
|
16
|
Ricker CA, Meli K, Van Allen EM. Historical perspective and future directions: computational science in immuno-oncology. J Immunother Cancer 2024; 12:e008306. [PMID: 38191244 PMCID: PMC10826578 DOI: 10.1136/jitc-2023-008306] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/07/2023] [Indexed: 01/10/2024] Open
Abstract
Immuno-oncology holds promise for transforming patient care having achieved durable clinical response rates across a variety of advanced and metastatic cancers. Despite these achievements, only a minority of patients respond to immunotherapy, underscoring the importance of elucidating molecular mechanisms responsible for response and resistance to inform the development and selection of treatments. Breakthroughs in molecular sequencing technologies have led to the generation of an immense amount of genomic and transcriptomic sequencing data that can be mined to uncover complex tumor-immune interactions using computational tools. In this review, we discuss existing and emerging computational methods that contextualize the composition and functional state of the tumor microenvironment, infer the reactivity and clonal dynamics from reconstructed immune cell receptor repertoires, and predict the antigenic landscape for immune cell recognition. We further describe the advantage of multi-omics analyses for capturing multidimensional relationships and artificial intelligence techniques for integrating omics data with histopathological and radiological images to encapsulate patterns of treatment response and tumor-immune biology. Finally, we discuss key challenges impeding their widespread use and clinical application and conclude with future perspectives. We are hopeful that this review will both serve as a guide for prospective researchers seeking to use existing tools for scientific discoveries and inspire the optimization or development of novel tools to enhance precision, ultimately expediting advancements in immunotherapy that improve patient survival and quality of life.
Collapse
Affiliation(s)
- Cora A Ricker
- Medical Oncology, Dana-Farber Cancer Institute, Boston, Massachusetts, USA
| | - Kevin Meli
- Medical Oncology, Dana-Farber Cancer Institute, Boston, Massachusetts, USA
- Harvard Medical School, Boston, Massachusetts, USA
| | | |
Collapse
|
17
|
Wagner SJ, Matek C, Shetab Boushehri S, Boxberg M, Lamm L, Sadafi A, Winter DJE, Marr C, Peng T. Built to Last? Reproducibility and Reusability of Deep Learning Algorithms in Computational Pathology. Mod Pathol 2024; 37:100350. [PMID: 37827448 DOI: 10.1016/j.modpat.2023.100350] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 10/02/2023] [Accepted: 10/03/2023] [Indexed: 10/14/2023]
Abstract
Recent progress in computational pathology has been driven by deep learning. While code and data availability are essential to reproduce findings from preceding publications, ensuring a deep learning model's reusability is more challenging. For that, the codebase should be well-documented and easy to integrate into existing workflows and models should be robust toward noise and generalizable toward data from different sources. Strikingly, only a few computational pathology algorithms have been reused by other researchers so far, let alone employed in a clinical setting. To assess the current state of reproducibility and reusability of computational pathology algorithms, we evaluated peer-reviewed articles available in PubMed, published between January 2019 and March 2021, in 5 use cases: stain normalization; tissue type segmentation; evaluation of cell-level features; genetic alteration prediction; and inference of grading, staging, and prognostic information. We compiled criteria for data and code availability and statistical result analysis and assessed them in 160 publications. We found that only one-quarter (41 of 160 publications) made code publicly available. Among these 41 studies, three-quarters (30 of 41) analyzed their results statistically, half of them (20 of 41) released their trained model weights, and approximately a third (16 of 41) used an independent cohort for evaluation. Our review is intended for both pathologists interested in deep learning and researchers applying algorithms to computational pathology challenges. We provide a detailed overview of publications with published code in the field, list reusable data handling tools, and provide criteria for reproducibility and reusability.
Collapse
Affiliation(s)
- Sophia J Wagner
- Helmholtz AI, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany; School of Computation, Information and Technology, Technical University of Munich, Garching, Germany
| | - Christian Matek
- Institute of AI for Health, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany; Institute of Pathology, University Hospital Erlangen, Erlangen, Germany
| | - Sayedali Shetab Boushehri
- School of Computation, Information and Technology, Technical University of Munich, Garching, Germany; Institute of AI for Health, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany; Data & Analytics (D&A), Roche Pharma Research and Early Development (pRED), Roche Innovation Center Munich, Germany
| | - Melanie Boxberg
- Institute of Pathology, Technical University Munich, Munich, Germany; Institute of Pathology Munich-North, Munich, Germany
| | - Lorenz Lamm
- Helmholtz AI, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany; Helmholtz Pioneer Campus, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany
| | - Ario Sadafi
- School of Computation, Information and Technology, Technical University of Munich, Garching, Germany; Institute of AI for Health, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany
| | - Dominik J E Winter
- Institute of AI for Health, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany; School of Life Sciences, Technical University of Munich, Weihenstephan, Germany
| | - Carsten Marr
- Institute of AI for Health, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany.
| | - Tingying Peng
- Helmholtz AI, Helmholtz Munich-German Research Center for Environmental Health, Neuherberg, Germany.
| |
Collapse
|
18
|
Rahrooh A, Garlid AO, Bartlett K, Coons W, Petousis P, Hsu W, Bui AAT. Towards a framework for interoperability and reproducibility of predictive models. J Biomed Inform 2024; 149:104551. [PMID: 38000765 DOI: 10.1016/j.jbi.2023.104551] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2023] [Revised: 08/28/2023] [Accepted: 11/19/2023] [Indexed: 11/26/2023]
Abstract
The development and deployment of machine learning (ML) models for biomedical research and healthcare currently lacks standard methodologies. Although tools for model replication are numerous, without a unifying blueprint it remains difficult to scientifically reproduce predictive ML models for any number of reasons (e.g., assumptions regarding data distributions and preprocessing, unclear test metrics, etc.) and ultimately, questions around generalizability and transportability are not readily answered. To facilitate scientific reproducibility, we built upon the Predictive Model Markup Language (PMML) to capture essential information. As a key component of the PREdictive Model Index and Exchange REpository (PREMIERE) platform, we present the Automated Metadata Pipeline (AMP) for conversion of a given predictive ML model into an extended PMML file that autocompletes an ML-based checklist, assessing model elements for interoperability and reproducibility. We demonstrate this pipeline on multiple test cases with three different ML algorithms and health-related datasets, providing a foundation for future predictive model reproducibility, sharing, and comparison.
Collapse
Affiliation(s)
- Al Rahrooh
- Medical & Imaging Informatics (MII) Group, University of California Los Angeles (UCLA), Los Angeles, CA, USA.
| | - Anders O Garlid
- Medical & Imaging Informatics (MII) Group, University of California Los Angeles (UCLA), Los Angeles, CA, USA
| | - Kelly Bartlett
- Medical & Imaging Informatics (MII) Group, University of California Los Angeles (UCLA), Los Angeles, CA, USA
| | - Warren Coons
- Medical & Imaging Informatics (MII) Group, University of California Los Angeles (UCLA), Los Angeles, CA, USA
| | - Panayiotis Petousis
- Clinical and Translational Science Institute (CTSI), University of California Los Angeles (UCLA), Los Angeles, CA, USA
| | - William Hsu
- Medical & Imaging Informatics (MII) Group, University of California Los Angeles (UCLA), Los Angeles, CA, USA
| | - Alex A T Bui
- Medical & Imaging Informatics (MII) Group, University of California Los Angeles (UCLA), Los Angeles, CA, USA; Clinical and Translational Science Institute (CTSI), University of California Los Angeles (UCLA), Los Angeles, CA, USA
| |
Collapse
|
19
|
Sigala RE, Lagou V, Shmeliov A, Atito S, Kouchaki S, Awais M, Prokopenko I, Mahdi A, Demirkan A. Machine Learning to Advance Human Genome-Wide Association Studies. Genes (Basel) 2023; 15:34. [PMID: 38254924 PMCID: PMC10815885 DOI: 10.3390/genes15010034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2023] [Revised: 12/19/2023] [Accepted: 12/22/2023] [Indexed: 01/24/2024] Open
Abstract
Machine learning, including deep learning, reinforcement learning, and generative artificial intelligence are revolutionising every area of our lives when data are made available. With the help of these methods, we can decipher information from larger datasets while addressing the complex nature of biological systems in a more efficient way. Although machine learning methods have been introduced to human genetic epidemiological research as early as 2004, those were never used to their full capacity. In this review, we outline some of the main applications of machine learning to assigning human genetic loci to health outcomes. We summarise widely used methods and discuss their advantages and challenges. We also identify several tools, such as Combi, GenNet, and GMSTool, specifically designed to integrate these methods for hypothesis-free analysis of genetic variation data. We elaborate on the additional value and limitations of these tools from a geneticist's perspective. Finally, we discuss the fast-moving field of foundation models and large multi-modal omics biobank initiatives.
Collapse
Affiliation(s)
- Rafaella E. Sigala
- Section of Statistical Multi-Omics, Department of Clinical and Experimental Medicine, Guildford GU2 7XH, Surrey, UK; (R.E.S.); (V.L.); (A.S.); (I.P.)
| | - Vasiliki Lagou
- Section of Statistical Multi-Omics, Department of Clinical and Experimental Medicine, Guildford GU2 7XH, Surrey, UK; (R.E.S.); (V.L.); (A.S.); (I.P.)
| | - Aleksey Shmeliov
- Section of Statistical Multi-Omics, Department of Clinical and Experimental Medicine, Guildford GU2 7XH, Surrey, UK; (R.E.S.); (V.L.); (A.S.); (I.P.)
| | - Sara Atito
- Surrey Institute for People-Centred Artificial Intelligence, University of Surrey, Guildford GU2 7XH, Surrey, UK; (S.A.); (S.K.); (M.A.)
- Centre for Vision, Speech and Signal Processing, University of Surrey, Guildford GU2 7XH, Surrey, UK
| | - Samaneh Kouchaki
- Surrey Institute for People-Centred Artificial Intelligence, University of Surrey, Guildford GU2 7XH, Surrey, UK; (S.A.); (S.K.); (M.A.)
- Centre for Vision, Speech and Signal Processing, University of Surrey, Guildford GU2 7XH, Surrey, UK
| | - Muhammad Awais
- Surrey Institute for People-Centred Artificial Intelligence, University of Surrey, Guildford GU2 7XH, Surrey, UK; (S.A.); (S.K.); (M.A.)
- Centre for Vision, Speech and Signal Processing, University of Surrey, Guildford GU2 7XH, Surrey, UK
| | - Inga Prokopenko
- Section of Statistical Multi-Omics, Department of Clinical and Experimental Medicine, Guildford GU2 7XH, Surrey, UK; (R.E.S.); (V.L.); (A.S.); (I.P.)
- Surrey Institute for People-Centred Artificial Intelligence, University of Surrey, Guildford GU2 7XH, Surrey, UK; (S.A.); (S.K.); (M.A.)
| | - Adam Mahdi
- Oxford Internet Institute, University of Oxford, Oxford OX1 3JS, Oxfordshire, UK;
| | - Ayse Demirkan
- Section of Statistical Multi-Omics, Department of Clinical and Experimental Medicine, Guildford GU2 7XH, Surrey, UK; (R.E.S.); (V.L.); (A.S.); (I.P.)
- Surrey Institute for People-Centred Artificial Intelligence, University of Surrey, Guildford GU2 7XH, Surrey, UK; (S.A.); (S.K.); (M.A.)
| |
Collapse
|
20
|
O'Connell S, Cannon DM, Broin PÓ. Predictive modelling of brain disorders with magnetic resonance imaging: A systematic review of modelling practices, transparency, and interpretability in the use of convolutional neural networks. Hum Brain Mapp 2023; 44:6561-6574. [PMID: 37909364 PMCID: PMC10681646 DOI: 10.1002/hbm.26521] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2023] [Revised: 09/28/2023] [Accepted: 10/10/2023] [Indexed: 11/03/2023] Open
Abstract
Brain disorders comprise several psychiatric and neurological disorders which can be characterized by impaired cognition, mood alteration, psychosis, depressive episodes, and neurodegeneration. Clinical diagnoses primarily rely on a combination of life history information and questionnaires, with a distinct lack of discriminative biomarkers in use for psychiatric disorders. Symptoms across brain conditions are associated with functional alterations of cognitive and emotional processes, which can correlate with anatomical variation; structural magnetic resonance imaging (MRI) data of the brain are therefore an important focus of research, particularly for predictive modelling. With the advent of large MRI data consortia (such as the Alzheimer's Disease Neuroimaging Initiative) facilitating a greater number of MRI-based classification studies, convolutional neural networks (CNNs)-deep learning models well suited to image processing tasks-have become increasingly popular for research into brain conditions. This has resulted in a myriad of studies reporting impressive predictive performances, demonstrating the potential clinical value of deep learning systems. However, methodologies can vary widely across studies, making them difficult to compare and/or reproduce, potentially limiting their clinical application. Here, we conduct a qualitative systematic literature review of 55 studies carrying out CNN-based predictive modelling of brain disorders using MRI data and evaluate them based on three principles-modelling practices, transparency, and interpretability. We propose several recommendations to enhance the potential for the integration of CNNs into clinical care.
Collapse
Affiliation(s)
- Shane O'Connell
- School of Mathematical and Statistical Sciences, College of Science and EngineeringUniversity of GalwayGalwayIreland
| | - Dara M. Cannon
- Clinical Neuroimaging Laboratory, Galway Neuroscience Centre, College of MedicineNursing and Health SciencesUniversity of GalwayGalwayIreland
| | - Pilib Ó. Broin
- School of Mathematical and Statistical Sciences, College of Science and EngineeringUniversity of GalwayGalwayIreland
| |
Collapse
|
21
|
Schacherer DP, Herrmann MD, Clunie DA, Höfener H, Clifford W, Longabaugh WJR, Pieper S, Kikinis R, Fedorov A, Homeyer A. The NCI Imaging Data Commons as a platform for reproducible research in computational pathology. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 242:107839. [PMID: 37832430 PMCID: PMC10841477 DOI: 10.1016/j.cmpb.2023.107839] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/15/2023] [Revised: 09/20/2023] [Accepted: 10/01/2023] [Indexed: 10/15/2023]
Abstract
BACKGROUND AND OBJECTIVES Reproducibility is a major challenge in developing machine learning (ML)-based solutions in computational pathology (CompPath). The NCI Imaging Data Commons (IDC) provides >120 cancer image collections according to the FAIR principles and is designed to be used with cloud ML services. Here, we explore its potential to facilitate reproducibility in CompPath research. METHODS Using the IDC, we implemented two experiments in which a representative ML-based method for classifying lung tumor tissue was trained and/or evaluated on different datasets. To assess reproducibility, the experiments were run multiple times with separate but identically configured instances of common ML services. RESULTS The results of different runs of the same experiment were reproducible to a large extent. However, we observed occasional, small variations in AUC values, indicating a practical limit to reproducibility. CONCLUSIONS We conclude that the IDC facilitates approaching the reproducibility limit of CompPath research (i) by enabling researchers to reuse exactly the same datasets and (ii) by integrating with cloud ML services so that experiments can be run in identically configured computing environments.
Collapse
Affiliation(s)
- Daniela P Schacherer
- Fraunhofer Institute for Digital Medicine MEVIS, Max-von-Laue-Straße 2, 28359 Bremen, Germany
| | - Markus D Herrmann
- Department of Pathology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
| | | | - Henning Höfener
- Fraunhofer Institute for Digital Medicine MEVIS, Max-von-Laue-Straße 2, 28359 Bremen, Germany
| | | | | | | | - Ron Kikinis
- Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, Boston, MA, USA
| | - Andrey Fedorov
- Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, Boston, MA, USA
| | - André Homeyer
- Fraunhofer Institute for Digital Medicine MEVIS, Max-von-Laue-Straße 2, 28359 Bremen, Germany.
| |
Collapse
|
22
|
Murray JD, Lange JJ, Bennett-Lenane H, Holm R, Kuentz M, O'Dwyer PJ, Griffin BT. Advancing algorithmic drug product development: Recommendations for machine learning approaches in drug formulation. Eur J Pharm Sci 2023; 191:106562. [PMID: 37562550 DOI: 10.1016/j.ejps.2023.106562] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2023] [Revised: 07/09/2023] [Accepted: 08/07/2023] [Indexed: 08/12/2023]
Abstract
Artificial intelligence is a rapidly expanding area of research, with the disruptive potential to transform traditional approaches in the pharmaceutical industry, from drug discovery and development to clinical practice. Machine learning, a subfield of artificial intelligence, has fundamentally transformed in silico modelling and has the capacity to streamline clinical translation. This paper reviews data-driven modelling methodologies with a focus on drug formulation development. Despite recent advances, there is limited modelling guidance specific to drug product development and a trend towards suboptimal modelling practices, resulting in models that may not give reliable predictions in practice. There is an overwhelming focus on benchtop experimental outcomes obtained for a specific modelling aim, leaving the capabilities of data scraping or the use of combined modelling approaches yet to be fully explored. Moreover, the preference for high accuracy can lead to a reliance on black box methods over interpretable models. This further limits the widespread adoption of machine learning as black boxes yield models that cannot be easily understood for the purposes of enhancing product performance. In this review, recommendations for conducting machine learning research for drug product development to ensure trustworthiness, transparency, and reliability of the models produced are presented. Finally, possible future directions on how research in this area might develop are discussed to aim for models that provide useful and robust guidance to formulators.
Collapse
Affiliation(s)
- Jack D Murray
- School of Pharmacy, University College Cork, Cork, Ireland
| | - Justus J Lange
- School of Pharmacy, University College Cork, Cork, Ireland; Roche Pharmaceutical Research & Early Development, Pre-Clinical CMC, Roche Innovation Center Basel, F. Hoffmann-La Roche Ltd, Grenzacherstrasse 124, Basel, Switzerland
| | | | - René Holm
- Department of Physics, Chemistry and Pharmacy, University of Southern Denmark, Campusvej 55, Odense 5230, Denmark
| | - Martin Kuentz
- School of Life Sciences, University of Applied Sciences and Arts Northwestern Switzerland, Muttenz CH 4132, Switzerland
| | | | | |
Collapse
|
23
|
|
24
|
Ng MY, Youssef A, Miner AS, Sarellano D, Long J, Larson DB, Hernandez-Boussard T, Langlotz CP. Perceptions of Data Set Experts on Important Characteristics of Health Data Sets Ready for Machine Learning: A Qualitative Study. JAMA Netw Open 2023; 6:e2345892. [PMID: 38039004 PMCID: PMC10692863 DOI: 10.1001/jamanetworkopen.2023.45892] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/17/2023] [Accepted: 10/20/2023] [Indexed: 12/02/2023] Open
Abstract
Importance The lack of data quality frameworks to guide the development of artificial intelligence (AI)-ready data sets limits their usefulness for machine learning (ML) research in health care and hinders the diagnostic excellence of developed clinical AI applications for patient care. Objective To discern what constitutes high-quality and useful data sets for health and biomedical ML research purposes according to subject matter experts. Design, Setting, and Participants This qualitative study interviewed data set experts, particularly those who are creators and ML researchers. Semistructured interviews were conducted in English and remotely through a secure video conferencing platform between August 23, 2022, and January 5, 2023. A total of 93 experts were invited to participate. Twenty experts were enrolled and interviewed. Using purposive sampling, experts were affiliated with a diverse representation of 16 health data sets/databases across organizational sectors. Content analysis was used to evaluate survey information and thematic analysis was used to analyze interview data. Main Outcomes and Measures Data set experts' perceptions on what makes data sets AI ready. Results Participants included 20 data set experts (11 [55%] men; mean [SD] age, 42 [11] years), of whom all were health data set creators, and 18 of the 20 were also ML researchers. Themes (3 main and 11 subthemes) were identified and integrated into an AI-readiness framework to show their association within the health data ecosystem. Participants partially determined the AI readiness of data sets using priority appraisal elements of accuracy, completeness, consistency, and fitness. Ethical acquisition and societal impact emerged as appraisal considerations in that participant samples have not been described to date in prior data quality frameworks. Factors that drive creation of high-quality health data sets and mitigate risks associated with data reuse in ML research were also relevant to AI readiness. The state of data availability, data quality standards, documentation, team science, and incentivization were associated with elements of AI readiness and the overall perception of data set usefulness. Conclusions and Relevance In this qualitative study of data set experts, participants contributed to the development of a grounded framework for AI data set quality. Data set AI readiness required the concerted appraisal of many elements and the balancing of transparency and ethical reflection against pragmatic constraints. The movement toward more reliable, relevant, and ethical AI and ML applications for patient care will inevitably require strategic updates to data set creation practices.
Collapse
Affiliation(s)
- Madelena Y. Ng
- Department of Medicine (Biomedical Informatics), Stanford University School of Medicine, Stanford, California
- Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, California
| | - Alaa Youssef
- Department of Radiology, Stanford University School of Medicine, Stanford, California
| | - Adam S. Miner
- Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Stanford, California
| | - Daniela Sarellano
- Department of Radiology, Stanford University School of Medicine, Stanford, California
| | - Jin Long
- Department of Pediatrics, Stanford University School of Medicine, Stanford, California
| | - David B. Larson
- Department of Radiology, Stanford University School of Medicine, Stanford, California
| | - Tina Hernandez-Boussard
- Department of Medicine (Biomedical Informatics), Stanford University School of Medicine, Stanford, California
- Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, California
| | - Curtis P. Langlotz
- Department of Medicine (Biomedical Informatics), Stanford University School of Medicine, Stanford, California
- Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, California
- Department of Radiology, Stanford University School of Medicine, Stanford, California
| |
Collapse
|
25
|
Fedorov A, Longabaugh WJR, Pot D, Clunie DA, Pieper SD, Gibbs DL, Bridge C, Herrmann MD, Homeyer A, Lewis R, Aerts HJWL, Krishnaswamy D, Thiriveedhi VK, Ciausu C, Schacherer DP, Bontempi D, Pihl T, Wagner U, Farahani K, Kim E, Kikinis R. National Cancer Institute Imaging Data Commons: Toward Transparency, Reproducibility, and Scalability in Imaging Artificial Intelligence. Radiographics 2023; 43:e230180. [PMID: 37999984 DOI: 10.1148/rg.230180] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2023]
Abstract
The remarkable advances of artificial intelligence (AI) technology are revolutionizing established approaches to the acquisition, interpretation, and analysis of biomedical imaging data. Development, validation, and continuous refinement of AI tools requires easy access to large high-quality annotated datasets, which are both representative and diverse. The National Cancer Institute (NCI) Imaging Data Commons (IDC) hosts large and diverse publicly available cancer image data collections. By harmonizing all data based on industry standards and colocalizing it with analysis and exploration resources, the IDC aims to facilitate the development, validation, and clinical translation of AI tools and address the well-documented challenges of establishing reproducible and transparent AI processing pipelines. Balanced use of established commercial products with open-source solutions, interconnected by standard interfaces, provides value and performance, while preserving sufficient agility to address the evolving needs of the research community. Emphasis on the development of tools, use cases to demonstrate the utility of uniform data representation, and cloud-based analysis aim to ease adoption and help define best practices. Integration with other data in the broader NCI Cancer Research Data Commons infrastructure opens opportunities for multiomics studies incorporating imaging data to further empower the research community to accelerate breakthroughs in cancer detection, diagnosis, and treatment. Published under a CC BY 4.0 license.
Collapse
Affiliation(s)
- Andrey Fedorov
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - William J R Longabaugh
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - David Pot
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - David A Clunie
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Steven D Pieper
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - David L Gibbs
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Christopher Bridge
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Markus D Herrmann
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - André Homeyer
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Rob Lewis
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Hugo J W L Aerts
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Deepa Krishnaswamy
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Vamsi Krishna Thiriveedhi
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Cosmin Ciausu
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Daniela P Schacherer
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Dennis Bontempi
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Todd Pihl
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Ulrike Wagner
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Keyvan Farahani
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Erika Kim
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| | - Ron Kikinis
- From the Department of Radiology, Brigham and Women's Hospital and Harvard Medical School, 399 Revolution Dr, Somerville, MA 02145 (A.F., D.K., V.K.T., C.C., R.K.); Institute for Systems Biology, Seattle, Wash (W.J.R.L., D.L.G.); General Dynamics Information Technology, Rockville, Md (D.P.); PixelMed Publishing, Bangor, Pa (D.A.C.); Isomics, Cambridge, Mass (S.D.P.); Departments of Radiology (C.B.) and Pathology (M.D.H.), Massachusetts General Hospital and Harvard Medical School, Boston, Mass; Fraunhofer MEVIS, Bremen, Germany (A.H., D.P.S.); Radical Imaging, Boston, Mass (R.L.); Artificial Intelligence in Medicine Program, Mass General Brigham, Harvard Medical School, Boston, Mass (H.J.W.L.A., D.B.); Radiology and Nuclear Medicine, CARIM & GROW, Maastricht University, Maastricht, the Netherlands (H.J.W.L.A., D.B.); Frederick National Laboratory for Cancer Research, Rockville, Md (T.P., U.W.); and National Cancer Institute, Bethesda, Md (K.F., E.K.)
| |
Collapse
|
26
|
Filippi CG, Stein JM, Wang Z, Bakas S, Liu Y, Chang PD, Lui Y, Hess C, Barboriak DP, Flanders AE, Wintermark M, Zaharchuk G, Wu O. Ethical Considerations and Fairness in the Use of Artificial Intelligence for Neuroradiology. AJNR Am J Neuroradiol 2023; 44:1242-1248. [PMID: 37652578 PMCID: PMC10631523 DOI: 10.3174/ajnr.a7963] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2023] [Accepted: 07/07/2023] [Indexed: 09/02/2023]
Abstract
In this review, concepts of algorithmic bias and fairness are defined qualitatively and mathematically. Illustrative examples are given of what can go wrong when unintended bias or unfairness in algorithmic development occurs. The importance of explainability, accountability, and transparency with respect to artificial intelligence algorithm development and clinical deployment is discussed. These are grounded in the concept of "primum no nocere" (first, do no harm). Steps to mitigate unfairness and bias in task definition, data collection, model definition, training, testing, deployment, and feedback are provided. Discussions on the implementation of fairness criteria that maximize benefit and minimize unfairness and harm to neuroradiology patients will be provided, including suggestions for neuroradiologists to consider as artificial intelligence algorithms gain acceptance into neuroradiology practice and become incorporated into routine clinical workflow.
Collapse
Affiliation(s)
- C G Filippi
- From the Department of Radiology (C.G.F.), Tufts University School of Medicine, Boston, Massachusetts
| | - J M Stein
- Department of Radiology (J.M.S., S.B.), University of Pennsylvania, Philadelphia, Pennsylvania
| | - Z Wang
- Athinoula A. Martinos Center for Biomedical Imaging (Z.W., Y. Liu, O.W.), Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts
| | - S Bakas
- Department of Radiology (J.M.S., S.B.), University of Pennsylvania, Philadelphia, Pennsylvania
| | - Y Liu
- Athinoula A. Martinos Center for Biomedical Imaging (Z.W., Y. Liu, O.W.), Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts
| | - P D Chang
- Department of Radiological Sciences (P.D.C.), University of California, Irvine, California
| | - Y Lui
- Department of Neuroradiology (Y. Lui), NYU Langone Health, New York, New York
| | - C Hess
- Department of Radiology and Biomedical Imaging (C.H.), University of California, San Francisco, San Francisco, California
| | - D P Barboriak
- Department of Radiology (D.P.B.), Duke University School of Medicine, Durham, North Carolina
| | - A E Flanders
- Department of Neuroradiology/Otolaryngology (ENT) Radiology (A.E.F.), Thomas Jefferson University, Philadelphia, Pennsylvania
| | - M Wintermark
- Department of Neuroradiology (M.W.), Division of Diagnostic Imaging, MD Anderson Cancer Center, Houston, Texas
| | - G Zaharchuk
- Department of Radiology (G.Z.), Stanford University, Stanford, California
| | - O Wu
- Athinoula A. Martinos Center for Biomedical Imaging (Z.W., Y. Liu, O.W.), Department of Radiology, Massachusetts General Hospital, Boston, Massachusetts
| |
Collapse
|
27
|
Nwosu OI, Crowson MG, Rameau A. Artificial Intelligence Governance and Otolaryngology-Head and Neck Surgery. Laryngoscope 2023; 133:2868-2870. [PMID: 37658749 PMCID: PMC10592089 DOI: 10.1002/lary.31013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Accepted: 08/18/2023] [Indexed: 09/05/2023]
Abstract
This rapid communication highlights components of artificial intelligence governance in healthcare and suggests adopting key governance approaches in otolaryngology – head and neck surgery.
Collapse
Affiliation(s)
- Obinna I. Nwosu
- Department of Otolaryngology-Head & Neck Surgery, Massachusetts Eye & Ear, Boston, Massachusetts, USA
- Department of Otolaryngology-Head & Neck Surgery, Harvard Medical School, Boston, Massachusetts, USA
| | - Matthew G. Crowson
- Department of Otolaryngology-Head & Neck Surgery, Massachusetts Eye & Ear, Boston, Massachusetts, USA
- Department of Otolaryngology-Head & Neck Surgery, Harvard Medical School, Boston, Massachusetts, USA
- Deloitte Consulting, Boston, Massachusetts, USA
| | - Anaïs Rameau
- Department of Otolaryngology–Head and Neck Surgery, Sean Parker Institute for the Voice, Weill Cornell Medical College, New York, New York, USA
| |
Collapse
|
28
|
Waqas A, Bui MM, Glassy EF, El Naqa I, Borkowski P, Borkowski AA, Rasool G. Revolutionizing Digital Pathology With the Power of Generative Artificial Intelligence and Foundation Models. J Transl Med 2023; 103:100255. [PMID: 37757969 DOI: 10.1016/j.labinv.2023.100255] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2023] [Revised: 09/06/2023] [Accepted: 09/19/2023] [Indexed: 09/29/2023] Open
Abstract
Digital pathology has transformed the traditional pathology practice of analyzing tissue under a microscope into a computer vision workflow. Whole-slide imaging allows pathologists to view and analyze microscopic images on a computer monitor, enabling computational pathology. By leveraging artificial intelligence (AI) and machine learning (ML), computational pathology has emerged as a promising field in recent years. Recently, task-specific AI/ML (eg, convolutional neural networks) has risen to the forefront, achieving above-human performance in many image-processing and computer vision tasks. The performance of task-specific AI/ML models depends on the availability of many annotated training datasets, which presents a rate-limiting factor for AI/ML development in pathology. Task-specific AI/ML models cannot benefit from multimodal data and lack generalization, eg, the AI models often struggle to generalize to new datasets or unseen variations in image acquisition, staining techniques, or tissue types. The 2020s are witnessing the rise of foundation models and generative AI. A foundation model is a large AI model trained using sizable data, which is later adapted (or fine-tuned) to perform different tasks using a modest amount of task-specific annotated data. These AI models provide in-context learning, can self-correct mistakes, and promptly adjust to user feedback. In this review, we provide a brief overview of recent advances in computational pathology enabled by task-specific AI, their challenges and limitations, and then introduce various foundation models. We propose to create a pathology-specific generative AI based on multimodal foundation models and present its potentially transformative role in digital pathology. We describe different use cases, delineating how it could serve as an expert companion of pathologists and help them efficiently and objectively perform routine laboratory tasks, including quantifying image analysis, generating pathology reports, diagnosis, and prognosis. We also outline the potential role that foundation models and generative AI can play in standardizing the pathology laboratory workflow, education, and training.
Collapse
Affiliation(s)
- Asim Waqas
- Department of Machine Learning, H. Lee Moffitt Cancer Center and Research Institute, Tampa, Florida; Department of Electrical Engineering, University of South Florida, Tampa, Florida.
| | - Marilyn M Bui
- Department of Machine Learning, H. Lee Moffitt Cancer Center and Research Institute, Tampa, Florida; Department of Pathology, H. Lee Moffitt Cancer Center and Research Institute, Tampa, Florida; University of South Florida, Morsani College of Medicine, Tampa, Florida
| | - Eric F Glassy
- Affiliated Pathologists Medical Group, Inc., Rancho Dominguez, California
| | - Issam El Naqa
- Department of Machine Learning, H. Lee Moffitt Cancer Center and Research Institute, Tampa, Florida
| | - Piotr Borkowski
- Quest Diagnostics/Ameripath, Tampa, Florida; Center of Excellence for Digital and AI-Empowered Pathology, Quest Diagnostics, Tampa, Florida
| | - Andrew A Borkowski
- University of South Florida, Morsani College of Medicine, Tampa, Florida; James A. Haley Veterans' Hospital, Tampa, Florida; National Artificial Intelligence Institute, Washington, District of Columbia
| | - Ghulam Rasool
- Department of Machine Learning, H. Lee Moffitt Cancer Center and Research Institute, Tampa, Florida; Department of Electrical Engineering, University of South Florida, Tampa, Florida; University of South Florida, Morsani College of Medicine, Tampa, Florida; Department of Neuro-Oncology, H. Lee Moffitt Cancer Center and Research Institute, Tampa, Florida
| |
Collapse
|
29
|
Wang YQ, Wang HC, Song YP, Zhou SQ, Li QN, Liang B, Liu WZ, Zhao YW, Wang AJ. Machine learning framework for intelligent aeration control in wastewater treatment plants: Automatic feature engineering based on variation sliding layer. WATER RESEARCH 2023; 246:120676. [PMID: 37806124 DOI: 10.1016/j.watres.2023.120676] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/21/2023] [Revised: 09/08/2023] [Accepted: 09/27/2023] [Indexed: 10/10/2023]
Abstract
Intelligent control of wastewater treatment plants (WWTPs) has the potential to reduce energy consumption and greenhouse gas emissions significantly. Machine learning (ML) provides a promising solution to handle the increasing amount and complexity of generated data. However, relationships between the features of wastewater datasets are generally inconspicuous, which hinders the application of artificial intelligence (AI) in WWTPs intelligent control. In this study, we develop an automatic framework of feature engineering based on variation sliding layer (VSL) to control the air demand precisely. Results demonstrated that using VSL in classic machine learning, deep learning, and ensemble learning could significantly improve the efficiency of aeration intelligent control in WWTPs. Bayesian regression and ensemble learning achieved the highest accuracy for predicting air demand. The developed models with VSL-ML models were also successfully implemented under the full-scale wastewater treatment plant, showing a 16.12 % reduction in demand compared to conventional aeration control of preset dissolved oxygen (DO) and feedback to the blower. The VSL-ML models showed great potential to be applied for the precision air demand prediction and control. The package as a tripartite library of Python is called wwtpai, which is freely accessible on GitHub and CSDN to remove technical barriers to the application of AI technology in WWTPs.
Collapse
Affiliation(s)
- Yu-Qi Wang
- State Key Laboratory of Urban Water Resource and Environment, School of Civil and Environmental Engineering, Harbin Institute of Technology Shenzhen, Shenzhen 518055, China
| | - Hong-Cheng Wang
- State Key Laboratory of Urban Water Resource and Environment, School of Civil and Environmental Engineering, Harbin Institute of Technology Shenzhen, Shenzhen 518055, China.
| | - Yun-Peng Song
- State Key Laboratory of Urban Water Resource and Environment, School of Civil and Environmental Engineering, Harbin Institute of Technology Shenzhen, Shenzhen 518055, China
| | - Shi-Qing Zhou
- Hunan Engineering Research Center of Water Security Technology and Application, College of Civil Engineering, Hunan University, Changsha 410082, China
| | - Qiu-Ning Li
- Department of Chemical and Biological Engineering, Monash University, Clayton, Australia
| | - Bin Liang
- State Key Laboratory of Urban Water Resource and Environment, School of Civil and Environmental Engineering, Harbin Institute of Technology Shenzhen, Shenzhen 518055, China
| | - Wen-Zong Liu
- State Key Laboratory of Urban Water Resource and Environment, School of Civil and Environmental Engineering, Harbin Institute of Technology Shenzhen, Shenzhen 518055, China
| | - Yi-Wei Zhao
- State Key Laboratory of Urban Water Resource and Environment, School of Civil and Environmental Engineering, Harbin Institute of Technology Shenzhen, Shenzhen 518055, China
| | - Ai-Jie Wang
- State Key Laboratory of Urban Water Resource and Environment, School of Civil and Environmental Engineering, Harbin Institute of Technology Shenzhen, Shenzhen 518055, China
| |
Collapse
|
30
|
Oudin A, Maatoug R, Bourla A, Ferreri F, Bonnot O, Millet B, Schoeller F, Mouchabac S, Adrien V. Digital Phenotyping: Data-Driven Psychiatry to Redefine Mental Health. J Med Internet Res 2023; 25:e44502. [PMID: 37792430 PMCID: PMC10585447 DOI: 10.2196/44502] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2022] [Revised: 07/10/2023] [Accepted: 08/21/2023] [Indexed: 10/05/2023] Open
Abstract
The term "digital phenotype" refers to the digital footprint left by patient-environment interactions. It has potential for both research and clinical applications but challenges our conception of health care by opposing 2 distinct approaches to medicine: one centered on illness with the aim of classifying and curing disease, and the other centered on patients, their personal distress, and their lived experiences. In the context of mental health and psychiatry, the potential benefits of digital phenotyping include creating new avenues for treatment and enabling patients to take control of their own well-being. However, this comes at the cost of sacrificing the fundamental human element of psychotherapy, which is crucial to addressing patients' distress. In this viewpoint paper, we discuss the advances rendered possible by digital phenotyping and highlight the risk that this technology may pose by partially excluding health care professionals from the diagnosis and therapeutic process, thereby foregoing an essential dimension of care. We conclude by setting out concrete recommendations on how to improve current digital phenotyping technology so that it can be harnessed to redefine mental health by empowering patients without alienating them.
Collapse
Affiliation(s)
- Antoine Oudin
- Infrastructure for Clinical Research in Neurosciences, Paris Brain Institute, Sorbonne University- Institut national de la santé et de la recherche médicale - Centre national de la recherche scientifique, Paris, France
- Department of Psychiatry, Pitié-Salpêtrière Hospital, Public Hospitals of Sorbonne University, Paris, France
| | - Redwan Maatoug
- Infrastructure for Clinical Research in Neurosciences, Paris Brain Institute, Sorbonne University- Institut national de la santé et de la recherche médicale - Centre national de la recherche scientifique, Paris, France
- Department of Psychiatry, Pitié-Salpêtrière Hospital, Public Hospitals of Sorbonne University, Paris, France
| | - Alexis Bourla
- Infrastructure for Clinical Research in Neurosciences, Paris Brain Institute, Sorbonne University- Institut national de la santé et de la recherche médicale - Centre national de la recherche scientifique, Paris, France
- Department of Psychiatry, Saint-Antoine Hospital, Public Hospitals of Sorbonne University, Paris, France
- Medical Strategy and Innovation Department, Clariane, Paris, France
- NeuroStim Psychiatry Practice, Paris, France
| | - Florian Ferreri
- Infrastructure for Clinical Research in Neurosciences, Paris Brain Institute, Sorbonne University- Institut national de la santé et de la recherche médicale - Centre national de la recherche scientifique, Paris, France
- Department of Psychiatry, Saint-Antoine Hospital, Public Hospitals of Sorbonne University, Paris, France
| | - Olivier Bonnot
- Department of Child and Adolescent Psychiatry, Nantes University Hospital, Nantes, France
- Pays de la Loire Psychology Laboratory, Nantes University, Nantes, France
| | - Bruno Millet
- Infrastructure for Clinical Research in Neurosciences, Paris Brain Institute, Sorbonne University- Institut national de la santé et de la recherche médicale - Centre national de la recherche scientifique, Paris, France
- Department of Psychiatry, Pitié-Salpêtrière Hospital, Public Hospitals of Sorbonne University, Paris, France
| | - Félix Schoeller
- Institute for Advanced Consciousness Studies, Santa Monica, CA, United States
- Media Lab, Massachusetts Institute of Technology, Cambridge, MA, United States
| | - Stéphane Mouchabac
- Infrastructure for Clinical Research in Neurosciences, Paris Brain Institute, Sorbonne University- Institut national de la santé et de la recherche médicale - Centre national de la recherche scientifique, Paris, France
- Department of Psychiatry, Saint-Antoine Hospital, Public Hospitals of Sorbonne University, Paris, France
| | - Vladimir Adrien
- Infrastructure for Clinical Research in Neurosciences, Paris Brain Institute, Sorbonne University- Institut national de la santé et de la recherche médicale - Centre national de la recherche scientifique, Paris, France
- Department of Psychiatry, Saint-Antoine Hospital, Public Hospitals of Sorbonne University, Paris, France
| |
Collapse
|
31
|
Kocak B, Yardimci AH, Yuzkan S, Keles A, Altun O, Bulut E, Bayrak ON, Okumus AA. Transparency in Artificial Intelligence Research: a Systematic Review of Availability Items Related to Open Science in Radiology and Nuclear Medicine. Acad Radiol 2023; 30:2254-2266. [PMID: 36526532 DOI: 10.1016/j.acra.2022.11.030] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2022] [Revised: 11/21/2022] [Accepted: 11/22/2022] [Indexed: 12/15/2022]
Abstract
RATIONALE AND OBJECTIVES Reproducibility of artificial intelligence (AI) research has become a growing concern. One of the fundamental reasons is the lack of transparency in data, code, and model. In this work, we aimed to systematically review the radiology and nuclear medicine papers on AI in terms of transparency and open science. MATERIALS AND METHODS A systematic literature search was performed in PubMed to identify original research studies on AI. The search was restricted to studies published in Q1 and Q2 journals that are also indexed on the Web of Science. A random sampling of the literature was performed. Besides six baseline study characteristics, a total of five availability items were evaluated. Two groups of independent readers including eight readers participated in the study. Inter-rater agreement was analyzed. Disagreements were resolved with consensus. RESULTS Following eligibility criteria, we included a final set of 194 papers. The raw data was available in about one-fifth of the papers (34/194; 18%). However, the authors made their private data available only in one paper (1/161; 1%). About one-tenth of the papers made their pre-modeling (25/194; 13%), modeling (28/194; 14%), or post-modeling files (15/194; 8%) available. Most of the papers (189/194; 97%) did not attempt to create a ready-to-use system for real-world usage. Data origin, use of deep learning, and external validation had statistically significantly different distributions. The use of private data alone was negatively associated with the availability of at least one item (p<0.001). CONCLUSION Overall rates of availability for items were poor, leaving room for substantial improvement.
Collapse
Affiliation(s)
- Burak Kocak
- Department of Radiology, University of Health Sciences, Basaksehir Cam and Sakura City Hospital, Basaksehir, 34480, Istanbul, Turkey.
| | - Aytul Hande Yardimci
- Department of Radiology, University of Health Sciences, Basaksehir Cam and Sakura City Hospital, Basaksehir, 34480, Istanbul, Turkey
| | - Sabahattin Yuzkan
- Department of Radiology, University of Health Sciences, Basaksehir Cam and Sakura City Hospital, Basaksehir, 34480, Istanbul, Turkey
| | - Ali Keles
- Department of Radiology, University of Health Sciences, Basaksehir Cam and Sakura City Hospital, Basaksehir, 34480, Istanbul, Turkey
| | - Omer Altun
- Department of Radiology, University of Health Sciences, Basaksehir Cam and Sakura City Hospital, Basaksehir, 34480, Istanbul, Turkey
| | - Elif Bulut
- Department of Radiology, University of Health Sciences, Basaksehir Cam and Sakura City Hospital, Basaksehir, 34480, Istanbul, Turkey
| | - Osman Nuri Bayrak
- Department of Radiology, University of Health Sciences, Basaksehir Cam and Sakura City Hospital, Basaksehir, 34480, Istanbul, Turkey
| | - Ahmet Arda Okumus
- Department of Radiology, University of Health Sciences, Basaksehir Cam and Sakura City Hospital, Basaksehir, 34480, Istanbul, Turkey
| |
Collapse
|
32
|
Pepe A, Egger J, Codari M, Willemink MJ, Gsaxner C, Li J, Roth PM, Schmalstieg D, Mistelbauer G, Fleischmann D. Automated cross-sectional view selection in CT angiography of aortic dissections with uncertainty awareness and retrospective clinical annotations. Comput Biol Med 2023; 165:107365. [PMID: 37647783 DOI: 10.1016/j.compbiomed.2023.107365] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2023] [Revised: 07/20/2023] [Accepted: 08/12/2023] [Indexed: 09/01/2023]
Abstract
Surveillance imaging of patients with chronic aortic diseases, such as aneurysms and dissections, relies on obtaining and comparing cross-sectional diameter measurements along the aorta at predefined aortic landmarks, over time. The orientation of the cross-sectional measuring planes at each landmark is currently defined manually by highly trained operators. Centerline-based approaches are unreliable in patients with chronic aortic dissection, because of the asymmetric flow channels, differences in contrast opacification, and presence of mural thrombus, making centerline computations or measurements difficult to generate and reproduce. In this work, we present three alternative approaches - INS, MCDS, MCDbS - based on convolutional neural networks and uncertainty quantification methods to predict the orientation (ϕ,θ) of such cross-sectional planes. For the monitoring of chronic aortic dissections, we show how a dataset of 162 CTA volumes with overall 3273 imperfect manual annotations routinely collected in a clinic can be efficiently used to accomplish this task, despite the presence of non-negligible interoperator variabilities in terms of mean absolute error (MAE) and 95% limits of agreement (LOA). We show how, despite the large limits of agreement in the training data, the trained model provides faster and more reproducible results than either an expert user or a centerline method. The remaining disagreement lies within the variability produced by three independent expert annotators and matches the current state of the art, providing a similar error, but in a fraction of the time.
Collapse
Affiliation(s)
- Antonio Pepe
- Graz University of Technology, Institute of Computer Graphics and Vision, Inffeldgasse 16/II, 8010 Graz, Austria; Stanford University, School of Medicine, 3D and Quantitative Imaging Lab, 300 Pasteur Drive Stanford, CA 94305, USA; Computer Algorithms for Médicine (Café) Laboratory, Graz, Austria.
| | - Jan Egger
- Computer Algorithms for Médicine (Café) Laboratory, Graz, Austria; University Medicine Essen, Institute for AI in Medicine (IKIM), Girardetstraße 2, 45131 Essen, Germany.
| | - Marina Codari
- Stanford University, School of Medicine, 3D and Quantitative Imaging Lab, 300 Pasteur Drive Stanford, CA 94305, USA.
| | - Martin J Willemink
- Stanford University, School of Medicine, 3D and Quantitative Imaging Lab, 300 Pasteur Drive Stanford, CA 94305, USA.
| | - Christina Gsaxner
- Graz University of Technology, Institute of Computer Graphics and Vision, Inffeldgasse 16/II, 8010 Graz, Austria; Computer Algorithms for Médicine (Café) Laboratory, Graz, Austria.
| | - Jianning Li
- Computer Algorithms for Médicine (Café) Laboratory, Graz, Austria; University Medicine Essen, Institute for AI in Medicine (IKIM), Girardetstraße 2, 45131 Essen, Germany.
| | - Peter M Roth
- Graz University of Technology, Institute of Computer Graphics and Vision, Inffeldgasse 16/II, 8010 Graz, Austria.
| | - Dieter Schmalstieg
- Graz University of Technology, Institute of Computer Graphics and Vision, Inffeldgasse 16/II, 8010 Graz, Austria.
| | - Gabriel Mistelbauer
- Stanford University, School of Medicine, 3D and Quantitative Imaging Lab, 300 Pasteur Drive Stanford, CA 94305, USA.
| | - Dominik Fleischmann
- Stanford University, School of Medicine, 3D and Quantitative Imaging Lab, 300 Pasteur Drive Stanford, CA 94305, USA.
| |
Collapse
|
33
|
Affiliation(s)
- David J Hunter
- From the Nuffield Department of Population Health (D.J.H.) and the Department of Statistics and Nuffield Department of Medicine (C.H.), University of Oxford, Oxford, and the Alan Turing Institute, London (C.H.) - both in the United Kingdom
| | - Christopher Holmes
- From the Nuffield Department of Population Health (D.J.H.) and the Department of Statistics and Nuffield Department of Medicine (C.H.), University of Oxford, Oxford, and the Alan Turing Institute, London (C.H.) - both in the United Kingdom
| |
Collapse
|
34
|
Vinayahalingam S, Kempers S, Schoep J, Hsu TMH, Moin DA, van Ginneken B, Flügge T, Hanisch M, Xi T. Intra-oral scan segmentation using deep learning. BMC Oral Health 2023; 23:643. [PMID: 37670290 PMCID: PMC10481506 DOI: 10.1186/s12903-023-03362-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2023] [Accepted: 08/26/2023] [Indexed: 09/07/2023] Open
Abstract
OBJECTIVE Intra-oral scans and gypsum cast scans (OS) are widely used in orthodontics, prosthetics, implantology, and orthognathic surgery to plan patient-specific treatments, which require teeth segmentations with high accuracy and resolution. Manual teeth segmentation, the gold standard up until now, is time-consuming, tedious, and observer-dependent. This study aims to develop an automated teeth segmentation and labeling system using deep learning. MATERIAL AND METHODS As a reference, 1750 OS were manually segmented and labeled. A deep-learning approach based on PointCNN and 3D U-net in combination with a rule-based heuristic algorithm and a combinatorial search algorithm was trained and validated on 1400 OS. Subsequently, the trained algorithm was applied to a test set consisting of 350 OS. The intersection over union (IoU), as a measure of accuracy, was calculated to quantify the degree of similarity between the annotated ground truth and the model predictions. RESULTS The model achieved accurate teeth segmentations with a mean IoU score of 0.915. The FDI labels of the teeth were predicted with a mean accuracy of 0.894. The optical inspection showed excellent position agreements between the automatically and manually segmented teeth components. Minor flaws were mostly seen at the edges. CONCLUSION The proposed method forms a promising foundation for time-effective and observer-independent teeth segmentation and labeling on intra-oral scans. CLINICAL SIGNIFICANCE Deep learning may assist clinicians in virtual treatment planning in orthodontics, prosthetics, implantology, and orthognathic surgery. The impact of using such models in clinical practice should be explored.
Collapse
Affiliation(s)
- Shankeeth Vinayahalingam
- Department of Oral and Maxillofacial Surgery, Radboud University Nijmegen Medical Centre, Nijmegen, the Netherlands
- Department of Artificial Intelligence, Radboud University, Nijmegen, the Netherlands
- Department of Oral and Maxillofacial Surgery, Universitätsklinikum Münster, Münster, Germany
| | - Steven Kempers
- Department of Oral and Maxillofacial Surgery, Radboud University Nijmegen Medical Centre, Nijmegen, the Netherlands
- Department of Artificial Intelligence, Radboud University, Nijmegen, the Netherlands
| | - Julian Schoep
- Promaton Co. Ltd, 1076 GR, Amsterdam, The Netherlands
| | - Tzu-Ming Harry Hsu
- MIT Computer Science & Artificial Intelligence Laboratory, 32 Vassar St, Cambridge, MA, 02139, USA
| | | | - Bram van Ginneken
- Department of Radiology, Radboud University Nijmegen Medical Centre, Nijmegen, the Netherlands
| | - Tabea Flügge
- Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität Zu Berlin, Department of Oral and Maxillofacial Surgery, Hindenburgdamm 30, 12203, Berlin, Germany.
| | - Marcel Hanisch
- Department of Oral and Maxillofacial Surgery, Universitätsklinikum Münster, Münster, Germany
- Promaton Co. Ltd, 1076 GR, Amsterdam, The Netherlands
| | - Tong Xi
- Department of Oral and Maxillofacial Surgery, Radboud University Nijmegen Medical Centre, Nijmegen, the Netherlands
| |
Collapse
|
35
|
Basereh M, Caputo A, Brennan R. Automatic transparency evaluation for open knowledge extraction systems. J Biomed Semantics 2023; 14:12. [PMID: 37653549 PMCID: PMC10468861 DOI: 10.1186/s13326-023-00293-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2022] [Accepted: 07/30/2023] [Indexed: 09/02/2023] Open
Abstract
BACKGROUND This paper proposes Cyrus, a new transparency evaluation framework, for Open Knowledge Extraction (OKE) systems. Cyrus is based on the state-of-the-art transparency models and linked data quality assessment dimensions. It brings together a comprehensive view of transparency dimensions for OKE systems. The Cyrus framework is used to evaluate the transparency of three linked datasets, which are built from the same corpus by three state-of-the-art OKE systems. The evaluation is automatically performed using a combination of three state-of-the-art FAIRness (Findability, Accessibility, Interoperability, Reusability) assessment tools and a linked data quality evaluation framework, called Luzzu. This evaluation includes six Cyrus data transparency dimensions for which existing assessment tools could be identified. OKE systems extract structured knowledge from unstructured or semi-structured text in the form of linked data. These systems are fundamental components of advanced knowledge services. However, due to the lack of a transparency framework for OKE, most OKE systems are not transparent. This means that their processes and outcomes are not understandable and interpretable. A comprehensive framework sheds light on different aspects of transparency, allows comparison between the transparency of different systems by supporting the development of transparency scores, gives insight into the transparency weaknesses of the system, and ways to improve them. Automatic transparency evaluation helps with scalability and facilitates transparency assessment. The transparency problem has been identified as critical by the European Union Trustworthy Artificial Intelligence (AI) guidelines. In this paper, Cyrus provides the first comprehensive view of transparency dimensions for OKE systems by merging the perspectives of the FAccT (Fairness, Accountability, and Transparency), FAIR, and linked data quality research communities. RESULTS In Cyrus, data transparency includes ten dimensions which are grouped in two categories. In this paper, six of these dimensions, i.e., provenance, interpretability, understandability, licensing, availability, interlinking have been evaluated automatically for three state-of-the-art OKE systems, using the state-of-the-art metrics and tools. Covid-on-the-Web is identified to have the highest mean transparency. CONCLUSIONS This is the first research to study the transparency of OKE systems that provides a comprehensive set of transparency dimensions spanning ethics, trustworthy AI, and data quality approaches to transparency. It also demonstrates how to perform automated transparency evaluation that combines existing FAIRness and linked data quality assessment tools for the first time. We show that state-of-the-art OKE systems vary in the transparency of the linked data generated and that these differences can be automatically quantified leading to potential applications in trustworthy AI, compliance, data protection, data governance, and future OKE system design and testing.
Collapse
Affiliation(s)
- Maryam Basereh
- School of Computing, Dublin City University, Dublin, Ireland.
| | - Annalina Caputo
- School of Computing, Dublin City University, Dublin, Ireland
| | - Rob Brennan
- ADAPT Centre, School of Computer Science, University College Dublin, Dublin, Ireland
| |
Collapse
|
36
|
Hayashi S, Caron BA, Heinsfeld AS, Vinci-Booher S, McPherson B, Bullock DN, Bertò G, Niso G, Hanekamp S, Levitas D, Ray K, MacKenzie A, Kitchell L, Leong JK, Nascimento-Silva F, Koudoro S, Willis H, Jolly JK, Pisner D, Zuidema TR, Kurzawski JW, Mikellidou K, Bussalb A, Rorden C, Victory C, Bhatia D, Baran Aydogan D, Yeh FCF, Delogu F, Guaje J, Veraart J, Bollman S, Stewart A, Fischer J, Faskowitz J, Chaumon M, Fabrega R, Hunt D, McKee S, Brown ST, Heyman S, Iacovella V, Mejia AF, Marinazzo D, Craddock RC, Olivetti E, Hanson JL, Avesani P, Garyfallidis E, Stanzione D, Carson J, Henschel R, Hancock DY, Stewart CA, Schnyer D, Eke DO, Poldrack RA, George N, Bridge H, Sani I, Freiwald WA, Puce A, Port NL, Pestilli F. brainlife.io: A decentralized and open source cloud platform to support neuroscience research. ARXIV 2023:arXiv:2306.02183v3. [PMID: 37332566 PMCID: PMC10274934] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Subscribe] [Scholar Register] [Indexed: 06/20/2023]
Abstract
Neuroscience research has expanded dramatically over the past 30 years by advancing standardization and tool development to support rigor and transparency. Consequently, the complexity of the data pipeline has also increased, hindering access to FAIR data analysis to portions of the worldwide research community. brainlife.io was developed to reduce these burdens and democratize modern neuroscience research across institutions and career levels. Using community software and hardware infrastructure, the platform provides open-source data standardization, management, visualization, and processing and simplifies the data pipeline. brainlife.io automatically tracks the provenance history of thousands of data objects, supporting simplicity, efficiency, and transparency in neuroscience research. Here brainlife.io's technology and data services are described and evaluated for validity, reliability, reproducibility, replicability, and scientific utility. Using data from 4 modalities and 3,200 participants, we demonstrate that brainlife.io's services produce outputs that adhere to best practices in modern neuroscience research.
Collapse
|
37
|
Maasch JRMA, Torres MDT, Melo MCR, de la Fuente-Nunez C. Molecular de-extinction of ancient antimicrobial peptides enabled by machine learning. Cell Host Microbe 2023; 31:1260-1274.e6. [PMID: 37516110 DOI: 10.1016/j.chom.2023.07.001] [Citation(s) in RCA: 17] [Impact Index Per Article: 17.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2022] [Revised: 05/12/2023] [Accepted: 07/06/2023] [Indexed: 07/31/2023]
Abstract
Molecular de-extinction could offer avenues for drug discovery by reintroducing bioactive molecules that are no longer encoded by extant organisms. To prospect for antimicrobial peptides encrypted within extinct and extant human proteins, we introduce the panCleave random forest model for proteome-wide cleavage site prediction. Our model outperformed multiple protease-specific cleavage site classifiers for three modern human caspases, despite its pan-protease design. Antimicrobial activity was observed in vitro for modern and archaic protein fragments identified with panCleave. Lead peptides showed resistance to proteolysis and exhibited variable membrane permeabilization. Additionally, representative modern and archaic protein fragments showed anti-infective efficacy against A. baumannii in both a skin abscess infection model and a preclinical murine thigh infection model. These results suggest that machine-learning-based encrypted peptide prospection can identify stable, nontoxic peptide antibiotics. Moreover, we establish molecular de-extinction through paleoproteome mining as a framework for antibacterial drug discovery.
Collapse
Affiliation(s)
- Jacqueline R M A Maasch
- Department of Computer and Information Science, School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, PA 19104, USA; Machine Biology Group, Departments of Psychiatry and Microbiology, Institute for Biomedical Informatics, Institute for Translational Medicine and Therapeutics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA; Department of Bioengineering, Department of Chemical and Biomolecular Engineering, School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, PA 19104, USA; Penn Institute for Computational Science, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Marcelo D T Torres
- Machine Biology Group, Departments of Psychiatry and Microbiology, Institute for Biomedical Informatics, Institute for Translational Medicine and Therapeutics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA; Department of Bioengineering, Department of Chemical and Biomolecular Engineering, School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, PA 19104, USA; Penn Institute for Computational Science, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Marcelo C R Melo
- Machine Biology Group, Departments of Psychiatry and Microbiology, Institute for Biomedical Informatics, Institute for Translational Medicine and Therapeutics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA; Department of Bioengineering, Department of Chemical and Biomolecular Engineering, School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, PA 19104, USA; Penn Institute for Computational Science, University of Pennsylvania, Philadelphia, PA 19104, USA
| | - Cesar de la Fuente-Nunez
- Machine Biology Group, Departments of Psychiatry and Microbiology, Institute for Biomedical Informatics, Institute for Translational Medicine and Therapeutics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA 19104, USA; Department of Bioengineering, Department of Chemical and Biomolecular Engineering, School of Engineering and Applied Science, University of Pennsylvania, Philadelphia, PA 19104, USA; Penn Institute for Computational Science, University of Pennsylvania, Philadelphia, PA 19104, USA.
| |
Collapse
|
38
|
Bi X, Shen X. Distribution-Invariant Differential Privacy. JOURNAL OF ECONOMETRICS 2023; 235:444-453. [PMID: 37701878 PMCID: PMC10495082 DOI: 10.1016/j.jeconom.2022.05.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/14/2023]
Abstract
Differential privacy is becoming one gold standard for protecting the privacy of publicly shared data. It has been widely used in social science, data science, public health, information technology, and the U.S. decennial census. Nevertheless, to guarantee differential privacy, existing methods may unavoidably alter the conclusion of original data analysis, as privatization often changes the sample distribution. This phenomenon is known as the trade-off between privacy protection and statistical accuracy. In this work, we mitigate this trade-off by developing a distribution-invariant privatization (DIP) method to reconcile both high statistical accuracy and strict differential privacy. As a result, any downstream statistical or machine learning task yields essentially the same conclusion as if one used the original data. Numerically, under the same strictness of privacy protection, DIP achieves superior statistical accuracy in a wide range of simulation studies and real-world benchmarks.
Collapse
Affiliation(s)
- Xuan Bi
- Information and Decision Sciences, Carlson School of Management, University of Minnesota, Minneapolis, MN
| | - Xiaotong Shen
- School of Statistics, University of Minnesota, Minneapolis, MN
| |
Collapse
|
39
|
Polevikov S. Advancing AI in healthcare: A comprehensive review of best practices. Clin Chim Acta 2023; 548:117519. [PMID: 37595864 DOI: 10.1016/j.cca.2023.117519] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2023] [Revised: 08/14/2023] [Accepted: 08/15/2023] [Indexed: 08/20/2023]
Abstract
Artificial Intelligence (AI) and Machine Learning (ML) are powerful tools shaping the healthcare sector. This review considers twelve key aspects of AI in clinical practice: 1) Ethical AI; 2) Explainable AI; 3) Health Equity and Bias in AI; 4) Sponsorship Bias; 5) Data Privacy; 6) Genomics and Privacy; 7) Insufficient Sample Size and Self-Serving Bias; 8) Bridging the Gap Between Training Datasets and Real-World Scenarios; 9) Open Source and Collaborative Development; 10) Dataset Bias and Synthetic Data; 11) Measurement Bias; 12) Reproducibility in AI Research. These categories represent both the challenges and opportunities of AI implementation in healthcare. While AI holds significant potential for improving patient care, it also presents risks and challenges, such as ensuring privacy, combating bias, and maintaining transparency and ethics. The review underscores the necessity of developing comprehensive best practices for healthcare organizations and fostering a diverse dialogue involving data scientists, clinicians, patient advocates, ethicists, economists, and policymakers. We are at the precipice of significant transformation in healthcare powered by AI. By continuing to reassess and refine our approach, we can ensure that AI is implemented responsibly and ethically, maximizing its benefit to patient care and public health.
Collapse
|
40
|
Rokhshad R, Ducret M, Chaurasia A, Karteva T, Radenkovic M, Roganovic J, Hamdan M, Mohammad-Rahimi H, Krois J, Lahoud P, Schwendicke F. Ethical considerations on artificial intelligence in dentistry: A framework and checklist. J Dent 2023; 135:104593. [PMID: 37355089 DOI: 10.1016/j.jdent.2023.104593] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2023] [Revised: 06/13/2023] [Accepted: 06/14/2023] [Indexed: 06/26/2023] Open
Abstract
OBJECTIVE Artificial Intelligence (AI) refers to the ability of machines to perform cognitive and intellectual human tasks. In dentistry, AI offers the potential to enhance diagnostic accuracy, improve patient outcomes and streamline workflows. The present study provides a framework and a checklist to evaluate AI applications in dentistry from this perspective. METHODS Lending from existing guidance documents, an initial draft of the checklist and an explanatory paper were derived and discussed among the groups members. RESULTS The checklist was consented to in an anonymous voting process by 29 Topic Group Dental Diagnostics and Digital Dentistry, ITU/WHO Focus Group AI on Health's members. Overall, 11 principles were identified (diversity, transparency, wellness, privacy protection, solidarity, equity, prudence, law and governance, sustainable development, accountability, and responsibility, respect of autonomy, decision-making). CONCLUSIONS Providers, patients, researchers, industry, and other stakeholders should consider these principles when developing, implementing, or receiving AI applications in dentistry. CLINICAL SIGNIFICANCE While AI has become increasingly commonplace in dentistry, there are ethical concerns around its usage, and users (providers, patients, and other stakeholders), as well as the industry should consider these when developing, implementing, or receiving AI applications based on comprehensive framework to address the associated ethical challenges.
Collapse
Affiliation(s)
- Rata Rokhshad
- Topic Group Dental Diagnostics and Digital Dentistry, ITU/WHO Focus Group AI on Health, Berlin, Germany.
| | - Maxime Ducret
- Topic Group Dental Diagnostics and Digital Dentistry, ITU/WHO Focus Group AI on Health, Berlin, Germany; Faculty of Odontology, University Claude Bernard Lyon Il, University of Lyon, Lyon, France
| | - Akhilanand Chaurasia
- Topic Group Dental Diagnostics and Digital Dentistry, ITU/WHO Focus Group AI on Health, Berlin, Germany; Department of Oral Medicine and Radiology, King George's Medical University, Lucknow, India; Faculty of Dentistry, University of Puthisashtra, Combodia
| | - Teodora Karteva
- Topic Group Dental Diagnostics and Digital Dentistry, ITU/WHO Focus Group AI on Health, Berlin, Germany; Department of Operative Dentistry and Endodontics, Faculty of Dental Medicine, Medical University Plovdiv, Bulgaria
| | - Miroslav Radenkovic
- Topic Group Dental Diagnostics and Digital Dentistry, ITU/WHO Focus Group AI on Health, Berlin, Germany; Department of Pharmacology, Clinical Pharmacology and Toxicology, Faculty of Medicine, University of Belgrade, Serbia
| | - Jelena Roganovic
- Topic Group Dental Diagnostics and Digital Dentistry, ITU/WHO Focus Group AI on Health, Berlin, Germany; Department of Pharmacology in Dentistry, School of Dental medicine, University of Belgrade, Serbia
| | - Manal Hamdan
- Topic Group Dental Diagnostics and Digital Dentistry, ITU/WHO Focus Group AI on Health, Berlin, Germany; General Dental Sciences Department, Marquette University School of Dentistry, USA
| | - Hossein Mohammad-Rahimi
- Topic Group Dental Diagnostics and Digital Dentistry, ITU/WHO Focus Group AI on Health, Berlin, Germany
| | - Joachim Krois
- Topic Group Dental Diagnostics and Digital Dentistry, ITU/WHO Focus Group AI on Health, Berlin, Germany; Department of Oral Diagnostics, Digital Health and Health Services Research, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Pierre Lahoud
- Topic Group Dental Diagnostics and Digital Dentistry, ITU/WHO Focus Group AI on Health, Berlin, Germany; Department of Oral Diagnostics, Digital Health and Health Services Research, Charité - Universitätsmedizin Berlin, Berlin, Germany; Department of Oral and MaxilloFacial Surgery & Imaging and Pathology- OMFS-IMPATH Research Group, KU Leuven, Belgium; Division of Periodontology and Oral Microbiology, Department of Oral Health Sciences, KU Leuven, Belgium
| | - Falk Schwendicke
- Topic Group Dental Diagnostics and Digital Dentistry, ITU/WHO Focus Group AI on Health, Berlin, Germany; Department of Oral Diagnostics, Digital Health and Health Services Research, Charité - Universitätsmedizin Berlin, Berlin, Germany
| |
Collapse
|
41
|
Kazmierski M, Welch M, Kim S, McIntosh C, Rey-McIntyre K, Huang SH, Patel T, Tadic T, Milosevic M, Liu FF, Ryczkowski A, Kazmierska J, Ye Z, Plana D, Aerts HJ, Kann BH, Bratman SV, Hope AJ, Haibe-Kains B. Multi-institutional Prognostic Modeling in Head and Neck Cancer: Evaluating Impact and Generalizability of Deep Learning and Radiomics. CANCER RESEARCH COMMUNICATIONS 2023; 3:1140-1151. [PMID: 37397861 PMCID: PMC10309070 DOI: 10.1158/2767-9764.crc-22-0152] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/07/2022] [Revised: 11/14/2022] [Accepted: 05/19/2023] [Indexed: 07/04/2023]
Abstract
Artificial intelligence (AI) and machine learning (ML) are becoming critical in developing and deploying personalized medicine and targeted clinical trials. Recent advances in ML have enabled the integration of wider ranges of data including both medical records and imaging (radiomics). However, the development of prognostic models is complex as no modeling strategy is universally superior to others and validation of developed models requires large and diverse datasets to demonstrate that prognostic models developed (regardless of method) from one dataset are applicable to other datasets both internally and externally. Using a retrospective dataset of 2,552 patients from a single institution and a strict evaluation framework that included external validation on three external patient cohorts (873 patients), we crowdsourced the development of ML models to predict overall survival in head and neck cancer (HNC) using electronic medical records (EMR) and pretreatment radiological images. To assess the relative contributions of radiomics in predicting HNC prognosis, we compared 12 different models using imaging and/or EMR data. The model with the highest accuracy used multitask learning on clinical data and tumor volume, achieving high prognostic accuracy for 2-year and lifetime survival prediction, outperforming models relying on clinical data only, engineered radiomics, or complex deep neural network architecture. However, when we attempted to extend the best performing models from this large training dataset to other institutions, we observed significant reductions in the performance of the model in those datasets, highlighting the importance of detailed population-based reporting for AI/ML model utility and stronger validation frameworks. We have developed highly prognostic models for overall survival in HNC using EMRs and pretreatment radiological images based on a large, retrospective dataset of 2,552 patients from our institution.Diverse ML approaches were used by independent investigators. The model with the highest accuracy used multitask learning on clinical data and tumor volume.External validation of the top three performing models on three datasets (873 patients) with significant differences in the distributions of clinical and demographic variables demonstrated significant decreases in model performance. Significance ML combined with simple prognostic factors outperformed multiple advanced CT radiomics and deep learning methods. ML models provided diverse solutions for prognosis of patients with HNC but their prognostic value is affected by differences in patient populations and require extensive validation.
Collapse
Affiliation(s)
- Michal Kazmierski
- Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada
- Princess Margaret Cancer Centre, Toronto, Ontario, Canada
| | - Mattea Welch
- Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada
- Princess Margaret Cancer Centre, Toronto, Ontario, Canada
- TECHNA Institute, Toronto, Ontario, Canada
| | - Sejin Kim
- Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada
- Princess Margaret Cancer Centre, Toronto, Ontario, Canada
| | - Chris McIntosh
- Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada
- TECHNA Institute, Toronto, Ontario, Canada
- Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, Ontario, Canada
| | - Katrina Rey-McIntyre
- Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, Ontario, Canada
| | - Shao Hui Huang
- Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, Ontario, Canada
- Department of Radiation Oncology, University of Toronto, Ontario, Canada
| | - Tirth Patel
- TECHNA Institute, Toronto, Ontario, Canada
- Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, Ontario, Canada
| | - Tony Tadic
- Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, Ontario, Canada
- Department of Radiation Oncology, University of Toronto, Ontario, Canada
| | - Michael Milosevic
- TECHNA Institute, Toronto, Ontario, Canada
- Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, Ontario, Canada
- Department of Radiation Oncology, University of Toronto, Ontario, Canada
| | - Fei-Fei Liu
- Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, Ontario, Canada
- Department of Radiation Oncology, University of Toronto, Ontario, Canada
| | - Adam Ryczkowski
- Department of Medical Physics, Greater Poland Cancer Centre, Poznan, Poland
- Department of Electroradiology, University of Medical Sciences, Poznan, Poland
| | - Joanna Kazmierska
- Department of Electroradiology, University of Medical Sciences, Poznan, Poland
- Department of Radiotherapy II, Greater Poland Cancer Centre, Poznan, Poland
| | - Zezhong Ye
- Artificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, Massachusetts
- Department of Radiation Oncology, Dana-Farber Cancer Institute / Brigham and Women's Hosptial, Boston, Massachusetts
| | - Deborah Plana
- Artificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, Massachusetts
- Department of Radiation Oncology, Dana-Farber Cancer Institute / Brigham and Women's Hosptial, Boston, Massachusetts
| | - Hugo J.W.L. Aerts
- Artificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, Massachusetts
- Department of Radiation Oncology, Dana-Farber Cancer Institute / Brigham and Women's Hosptial, Boston, Massachusetts
- Radiology and Nuclear Medicine, CARIM and GROW, Maastricht University, Maastricht, the Netherlands
| | - Benjamin H. Kann
- Artificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, Massachusetts
- Department of Radiation Oncology, Dana-Farber Cancer Institute / Brigham and Women's Hosptial, Boston, Massachusetts
| | - Scott V. Bratman
- Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada
- Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, Ontario, Canada
- Department of Radiation Oncology, University of Toronto, Ontario, Canada
| | - Andrew J. Hope
- Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, Ontario, Canada
- Department of Radiation Oncology, University of Toronto, Ontario, Canada
| | - Benjamin Haibe-Kains
- Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada
- Princess Margaret Cancer Centre, Toronto, Ontario, Canada
| |
Collapse
|
42
|
Imrie F, Cebere B, McKinney EF, van der Schaar M. AutoPrognosis 2.0: Democratizing diagnostic and prognostic modeling in healthcare with automated machine learning. PLOS DIGITAL HEALTH 2023; 2:e0000276. [PMID: 37347752 DOI: 10.1371/journal.pdig.0000276] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/21/2022] [Accepted: 05/17/2023] [Indexed: 06/24/2023]
Abstract
Diagnostic and prognostic models are increasingly important in medicine and inform many clinical decisions. Recently, machine learning approaches have shown improvement over conventional modeling techniques by better capturing complex interactions between patient covariates in a data-driven manner. However, the use of machine learning introduces technical and practical challenges that have thus far restricted widespread adoption of such techniques in clinical settings. To address these challenges and empower healthcare professionals, we present an open-source machine learning framework, AutoPrognosis 2.0, to facilitate the development of diagnostic and prognostic models. AutoPrognosis leverages state-of-the-art advances in automated machine learning to develop optimized machine learning pipelines, incorporates model explainability tools, and enables deployment of clinical demonstrators, without requiring significant technical expertise. To demonstrate AutoPrognosis 2.0, we provide an illustrative application where we construct a prognostic risk score for diabetes using the UK Biobank, a prospective study of 502,467 individuals. The models produced by our automated framework achieve greater discrimination for diabetes than expert clinical risk scores. We have implemented our risk score as a web-based decision support tool, which can be publicly accessed by patients and clinicians. By open-sourcing our framework as a tool for the community, we aim to provide clinicians and other medical practitioners with an accessible resource to develop new risk scores, personalized diagnostics, and prognostics using machine learning techniques. Software: https://github.com/vanderschaarlab/AutoPrognosis.
Collapse
Affiliation(s)
- Fergus Imrie
- Department of Electrical and Computer Engineering, University of California, Los Angeles, California, United States of America
| | - Bogdan Cebere
- Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, United Kingdom
| | - Eoin F McKinney
- Department of Medicine, University of Cambridge, Cambridge, United Kingdom
| | - Mihaela van der Schaar
- Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge, United Kingdom
- The Alan Turing Institute, London, United Kingdom
| |
Collapse
|
43
|
Kocak B, Baessler B, Bakas S, Cuocolo R, Fedorov A, Maier-Hein L, Mercaldo N, Müller H, Orlhac F, Pinto Dos Santos D, Stanzione A, Ugga L, Zwanenburg A. CheckList for EvaluAtion of Radiomics research (CLEAR): a step-by-step reporting guideline for authors and reviewers endorsed by ESR and EuSoMII. Insights Imaging 2023; 14:75. [PMID: 37142815 PMCID: PMC10160267 DOI: 10.1186/s13244-023-01415-8] [Citation(s) in RCA: 61] [Impact Index Per Article: 61.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Accepted: 03/24/2023] [Indexed: 05/06/2023] Open
Abstract
Even though radiomics can hold great potential for supporting clinical decision-making, its current use is mostly limited to academic research, without applications in routine clinical practice. The workflow of radiomics is complex due to several methodological steps and nuances, which often leads to inadequate reporting and evaluation, and poor reproducibility. Available reporting guidelines and checklists for artificial intelligence and predictive modeling include relevant good practices, but they are not tailored to radiomic research. There is a clear need for a complete radiomics checklist for study planning, manuscript writing, and evaluation during the review process to facilitate the repeatability and reproducibility of studies. We here present a documentation standard for radiomic research that can guide authors and reviewers. Our motivation is to improve the quality and reliability and, in turn, the reproducibility of radiomic research. We name the checklist CLEAR (CheckList for EvaluAtion of Radiomics research), to convey the idea of being more transparent. With its 58 items, the CLEAR checklist should be considered a standardization tool providing the minimum requirements for presenting clinical radiomics research. In addition to a dynamic online version of the checklist, a public repository has also been set up to allow the radiomics community to comment on the checklist items and adapt the checklist for future versions. Prepared and revised by an international group of experts using a modified Delphi method, we hope the CLEAR checklist will serve well as a single and complete scientific documentation tool for authors and reviewers to improve the radiomics literature.
Collapse
Affiliation(s)
- Burak Kocak
- Department of Radiology, University of Health Sciences, Basaksehir Cam and Sakura City Hospital, Basaksehir, Istanbul, 34480, Turkey.
| | - Bettina Baessler
- Institute of Diagnostic and Interventional Radiology, University Hospital Würzburg, Würzburg, Germany
| | - Spyridon Bakas
- Center for Artificial Intelligence for Integrated Diagnostics (AI2D) & Center for Biomedical Image Computing & Analytics (CBICA), University of Pennsylvania, Philadelphia, PA, USA
- Department of Radiology, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
- Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Renato Cuocolo
- Department of Medicine, Surgery, and Dentistry, University of Salerno, Baronissi, Italy
| | - Andrey Fedorov
- Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Lena Maier-Hein
- Division of Intelligent Medical Systems, German Cancer Research Center, Heidelberg, Germany
- National Center for Tumor Diseases (NCT), Heidelberg, Germany
| | - Nathaniel Mercaldo
- Institute for Technology Assessment, Massachusetts General Hospital, Boston, MA, USA
- Department of Radiology, Massachusetts General Hospital, Boston, MA, USA
| | - Henning Müller
- University of Applied Sciences of Western Switzerland (HES-SO Valais), Valais, Switzerland
- Department of Radiology and Medical Informatics, University of Geneva (UniGe), Geneva, Switzerland
| | - Fanny Orlhac
- Laboratoire d'Imagerie Translationnelle en Oncologie (LITO)-U1288, Institut Curie, Inserm, Université PSL, Orsay, France
| | - Daniel Pinto Dos Santos
- Department of Radiology, University Hospital of Cologne, Cologne, Germany
- Institute for Diagnostic and Interventional Radiology, Goethe-University Frankfurt Am Main, Frankfurt, Germany
| | - Arnaldo Stanzione
- Department of Advanced Biomedical Sciences, University of Naples "Federico II", Naples, Italy
| | - Lorenzo Ugga
- Department of Advanced Biomedical Sciences, University of Naples "Federico II", Naples, Italy
| | - Alex Zwanenburg
- OncoRay-National Center for Radiation Research in Oncology, Faculty of Medicine and University Hospital Carl Gustav Carus, Technische Universität Dresden, Helmholtz-Zentrum Dresden-Rossendorf, Dresden, Germany
- National Center for Tumor Diseases (NCT), Partner Site Dresden, Dresden, Germany
- German Cancer Research Center (DKFZ), Heidelberg, Germany
| |
Collapse
|
44
|
Alberto IRI, Alberto NRI, Ghosh AK, Jain B, Jayakumar S, Martinez-Martin N, McCague N, Moukheiber D, Moukheiber L, Moukheiber M, Moukheiber S, Yaghy A, Zhang A, Celi LA. The impact of commercial health datasets on medical research and health-care algorithms. Lancet Digit Health 2023; 5:e288-e294. [PMID: 37100543 PMCID: PMC10155113 DOI: 10.1016/s2589-7500(23)00025-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2022] [Revised: 12/26/2022] [Accepted: 02/03/2023] [Indexed: 04/28/2023]
Abstract
As the health-care industry emerges into a new era of digital health driven by cloud data storage, distributed computing, and machine learning, health-care data have become a premium commodity with value for private and public entities. Current frameworks of health data collection and distribution, whether from industry, academia, or government institutions, are imperfect and do not allow researchers to leverage the full potential of downstream analytical efforts. In this Health Policy paper, we review the current landscape of commercial health data vendors, with special emphasis on the sources of their data, challenges associated with data reproducibility and generalisability, and ethical considerations for data vending. We argue for sustainable approaches to curating open-source health data to enable global populations to be included in the biomedical research community. However, to fully implement these approaches, key stakeholders should come together to make health-care datasets increasingly accessible, inclusive, and representative, while balancing the privacy and rights of individuals whose data are being collected.
Collapse
Affiliation(s)
| | | | - Arnab K Ghosh
- Department of Medicine, Weill Cornell Medical College, Cornell University, New York, NY, USA
| | - Bhav Jain
- Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, USA
| | | | | | - Ned McCague
- Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, USA; Markforged, Watertown, MA, USA
| | - Dana Moukheiber
- Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Lama Moukheiber
- Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Mira Moukheiber
- The Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Sulaiman Moukheiber
- Department of Computer Science, Worcester Polytechnic Institute, Worcester, MA, USA
| | - Antonio Yaghy
- Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, USA; New England Eye Center, Tufts University Medical Center, Boston, MA, USA
| | - Andrew Zhang
- Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, USA; Department of Stem Cell and Regenerative Biology, Harvard University, Cambridge, MA, USA
| | - Leo Anthony Celi
- Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, USA; Department of Medicine, Beth Israel Deaconess Medical Center, Boston, MA, USA; Department of Biostatistics, Harvard T H Chan School of Public Health, Boston, MA, USA.
| |
Collapse
|
45
|
Paudyal R, Shah AD, Akin O, Do RKG, Konar AS, Hatzoglou V, Mahmood U, Lee N, Wong RJ, Banerjee S, Shin J, Veeraraghavan H, Shukla-Dave A. Artificial Intelligence in CT and MR Imaging for Oncological Applications. Cancers (Basel) 2023; 15:cancers15092573. [PMID: 37174039 PMCID: PMC10177423 DOI: 10.3390/cancers15092573] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Revised: 04/13/2023] [Accepted: 04/17/2023] [Indexed: 05/15/2023] Open
Abstract
Cancer care increasingly relies on imaging for patient management. The two most common cross-sectional imaging modalities in oncology are computed tomography (CT) and magnetic resonance imaging (MRI), which provide high-resolution anatomic and physiological imaging. Herewith is a summary of recent applications of rapidly advancing artificial intelligence (AI) in CT and MRI oncological imaging that addresses the benefits and challenges of the resultant opportunities with examples. Major challenges remain, such as how best to integrate AI developments into clinical radiology practice, the vigorous assessment of quantitative CT and MR imaging data accuracy, and reliability for clinical utility and research integrity in oncology. Such challenges necessitate an evaluation of the robustness of imaging biomarkers to be included in AI developments, a culture of data sharing, and the cooperation of knowledgeable academics with vendor scientists and companies operating in radiology and oncology fields. Herein, we will illustrate a few challenges and solutions of these efforts using novel methods for synthesizing different contrast modality images, auto-segmentation, and image reconstruction with examples from lung CT as well as abdome, pelvis, and head and neck MRI. The imaging community must embrace the need for quantitative CT and MRI metrics beyond lesion size measurement. AI methods for the extraction and longitudinal tracking of imaging metrics from registered lesions and understanding the tumor environment will be invaluable for interpreting disease status and treatment efficacy. This is an exciting time to work together to move the imaging field forward with narrow AI-specific tasks. New AI developments using CT and MRI datasets will be used to improve the personalized management of cancer patients.
Collapse
Affiliation(s)
- Ramesh Paudyal
- Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York City, NY 10065, USA
| | - Akash D Shah
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York City, NY 10065, USA
| | - Oguz Akin
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York City, NY 10065, USA
| | - Richard K G Do
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York City, NY 10065, USA
| | - Amaresha Shridhar Konar
- Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York City, NY 10065, USA
| | - Vaios Hatzoglou
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York City, NY 10065, USA
| | - Usman Mahmood
- Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York City, NY 10065, USA
| | - Nancy Lee
- Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York City, NY 10065, USA
| | - Richard J Wong
- Head and Neck Service, Department of Surgery, Memorial Sloan Kettering Cancer Center, New York City, NY 10065, USA
| | | | | | - Harini Veeraraghavan
- Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York City, NY 10065, USA
| | - Amita Shukla-Dave
- Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York City, NY 10065, USA
- Department of Radiology, Memorial Sloan Kettering Cancer Center, New York City, NY 10065, USA
| |
Collapse
|
46
|
Ostropolets A, Albogami Y, Conover M, Banda JM, Baumgartner WA, Blacketer C, Desai P, DuVall SL, Fortin S, Gilbert JP, Golozar A, Ide J, Kanter AS, Kern DM, Kim C, Lai LYH, Li C, Liu F, Lynch KE, Minty E, Neves MI, Ng DQ, Obene T, Pera V, Pratt N, Rao G, Rappoport N, Reinecke I, Saroufim P, Shoaibi A, Simon K, Suchard MA, Swerdel JN, Voss EA, Weaver J, Zhang L, Hripcsak G, Ryan PB. Reproducible variability: assessing investigator discordance across 9 research teams attempting to reproduce the same observational study. J Am Med Inform Assoc 2023; 30:859-868. [PMID: 36826399 PMCID: PMC10114120 DOI: 10.1093/jamia/ocad009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2022] [Revised: 01/04/2023] [Accepted: 01/23/2023] [Indexed: 02/25/2023] Open
Abstract
OBJECTIVE Observational studies can impact patient care but must be robust and reproducible. Nonreproducibility is primarily caused by unclear reporting of design choices and analytic procedures. This study aimed to: (1) assess how the study logic described in an observational study could be interpreted by independent researchers and (2) quantify the impact of interpretations' variability on patient characteristics. MATERIALS AND METHODS Nine teams of highly qualified researchers reproduced a cohort from a study by Albogami et al. The teams were provided the clinical codes and access to the tools to create cohort definitions such that the only variable part was their logic choices. We executed teams' cohort definitions against the database and compared the number of subjects, patient overlap, and patient characteristics. RESULTS On average, the teams' interpretations fully aligned with the master implementation in 4 out of 10 inclusion criteria with at least 4 deviations per team. Cohorts' size varied from one-third of the master cohort size to 10 times the cohort size (2159-63 619 subjects compared to 6196 subjects). Median agreement was 9.4% (interquartile range 15.3-16.2%). The teams' cohorts significantly differed from the master implementation by at least 2 baseline characteristics, and most of the teams differed by at least 5. CONCLUSIONS Independent research teams attempting to reproduce the study based on its free-text description alone produce different implementations that vary in the population size and composition. Sharing analytical code supported by a common data model and open-source tools allows reproducing a study unambiguously thereby preserving initial design choices.
Collapse
Affiliation(s)
- Anna Ostropolets
- Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, New York, USA
| | - Yasser Albogami
- Department of Clinical Pharmacy, College of Pharmacy, King Saud University, Riyadh, Saudi Arabia
| | - Mitchell Conover
- Observational Health Data Analytics, Janssen Research & Development, Titusville, New Jersey, USA
| | - Juan M Banda
- Department of Computer Science, Georgia State University, Atlanta, Georgia, USA
| | - William A Baumgartner
- Division of General Internal Medicine, Department of Medicine, University of Colorado Anschutz Medical Campus, Aurora, Colorado, USA
| | - Clair Blacketer
- Observational Health Data Analytics, Janssen Research & Development, Titusville, New Jersey, USA
| | - Priyamvada Desai
- Research IT, Technology and Digital Solutions, Stanford Medicine, Stanford, California, USA
| | - Scott L DuVall
- VA Salt Lake City Health Care System, Salt Lake City, Utah, USA
- Department of Internal Medicine, University of Utah School of Medicine, Salt Lake City, Utah, USA
| | - Stephen Fortin
- Observational Health Data Analytics, Janssen Research & Development, Titusville, New Jersey, USA
| | - James P Gilbert
- Observational Health Data Analytics, Janssen Research & Development, Titusville, New Jersey, USA
| | | | - Joshua Ide
- Johnson & Johnson, Titusville, New Jersey, USA
| | - Andrew S Kanter
- Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, New York, USA
| | - David M Kern
- Observational Health Data Analytics, Janssen Research & Development, Titusville, New Jersey, USA
| | - Chungsoo Kim
- Department of Biomedical Sciences, Ajou University Graduate School of Medicine, Suwon, South Korea
| | - Lana Y H Lai
- Department of Informatics, Imaging & Data Sciences, University of Manchester, Manchester, UK
| | - Chenyu Li
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Feifan Liu
- Department of Population and Quantitative Health Sciences, University of Massachusetts Chan Medical School, Worcester, Massachusetts, USA
| | - Kristine E Lynch
- VA Salt Lake City Health Care System, Salt Lake City, Utah, USA
- Department of Internal Medicine, University of Utah School of Medicine, Salt Lake City, Utah, USA
| | - Evan Minty
- O’Brien Institute for Public Health, Faculty of Medicine, University of Calgary, Calgary, Canada
| | | | - Ding Quan Ng
- Department of Pharmaceutical Sciences, School of Pharmacy & Pharmaceutical Sciences, University of California, Irvine, California, USA
| | - Tontel Obene
- Mississippi Urban Research Center, Jackson State University, Jackson, Mississippi, USA
| | - Victor Pera
- Department of Medical Informatics, Erasmus University Medical Center, Rotterdam, The Netherlands
| | - Nicole Pratt
- Quality Use of Medicines and Pharmacy Research Centre, University of South Australia, Adelaide, Australia
| | - Gowtham Rao
- Observational Health Data Analytics, Janssen Research & Development, Titusville, New Jersey, USA
| | - Nadav Rappoport
- Department of Software and Information Systems Engineering, Ben-Gurion University of the Negev, Israel
| | - Ines Reinecke
- Institute for Medical Informatics and Biometry, Carl Gustav Carus Faculty of Medicine, Technische Universität Dresden, Dresden, Germany
| | - Paola Saroufim
- Cleveland Institute for Computational Biology, Case Western Reserve University, Cleveland, Ohio, USA
| | - Azza Shoaibi
- Observational Health Data Analytics, Janssen Research & Development, Titusville, New Jersey, USA
| | - Katherine Simon
- VA Tennessee Valley Health Care System, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Marc A Suchard
- Department of Biostatistics, University of California, Los Angeles, California, USA
- Department of Human Genetics, University of California, Los Angeles, California, USA
| | - Joel N Swerdel
- Observational Health Data Analytics, Janssen Research & Development, Titusville, New Jersey, USA
| | - Erica A Voss
- Observational Health Data Analytics, Janssen Research & Development, Titusville, New Jersey, USA
| | - James Weaver
- Observational Health Data Analytics, Janssen Research & Development, Titusville, New Jersey, USA
| | - Linying Zhang
- Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, New York, USA
| | - George Hripcsak
- Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, New York, USA
- Medical Informatics Services, New York-Presbyterian Hospital, New York, New York, USA
| | - Patrick B Ryan
- Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, New York, USA
- Observational Health Data Analytics, Janssen Research & Development, Titusville, New Jersey, USA
| |
Collapse
|
47
|
Burnell R, Schellaert W, Burden J, Ullman TD, Martinez-Plumed F, Tenenbaum JB, Rutar D, Cheke LG, Sohl-Dickstein J, Mitchell M, Kiela D, Shanahan M, Voorhees EM, Cohn AG, Leibo JZ, Hernandez-Orallo J. Rethink reporting of evaluation results in AI. Science 2023; 380:136-138. [PMID: 37053341 DOI: 10.1126/science.adf6369] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/15/2023]
Abstract
Aggregate metrics and lack of access to results limit understanding.
Collapse
Affiliation(s)
- Ryan Burnell
- Leverhulme Centre for the Future of Intelligence, University of Cambridge, Cambridge, UK
| | - Wout Schellaert
- Valencian Research Institute for Artificial Intelligence, Universitat Politècnica de Valencia, València, Spain
| | - John Burden
- Leverhulme Centre for the Future of Intelligence, University of Cambridge, Cambridge, UK
- Centre for the Study of Existential Risk, University of Cambridge, Cambridge, UK
| | - Tomer D Ullman
- Department of Psychology, Harvard University, Cambridge, MA, USA
| | - Fernando Martinez-Plumed
- Valencian Research Institute for Artificial Intelligence, Universitat Politècnica de Valencia, València, Spain
| | - Joshua B Tenenbaum
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Danaja Rutar
- Leverhulme Centre for the Future of Intelligence, University of Cambridge, Cambridge, UK
| | - Lucy G Cheke
- Leverhulme Centre for the Future of Intelligence, University of Cambridge, Cambridge, UK
- Department of Psychology, University of Cambridge, Cambridge, UK
| | | | | | | | - Murray Shanahan
- DeepMind, London, UK
- Department of Computing, Imperial College London, London, UK
| | - Ellen M Voorhees
- National Institute of Standards and Technology (Retired), Gaithersburg, MD, USA
| | - Anthony G Cohn
- School of Computing, University of Leeds, Leeds, UK
- Alan Turing Institute, London, UK
- Tongji University, Shanghai, China
- Shandong University, Jinan, China
| | | | - Jose Hernandez-Orallo
- Leverhulme Centre for the Future of Intelligence, University of Cambridge, Cambridge, UK
- Valencian Research Institute for Artificial Intelligence, Universitat Politècnica de Valencia, València, Spain
- Centre for the Study of Existential Risk, University of Cambridge, Cambridge, UK
| |
Collapse
|
48
|
Kempers S, van Lierop P, Hsu TMH, Moin DA, Bergé S, Ghaeminia H, Xi T, Vinayahalingam S. Positional assessment of lower third molar and mandibular canal using explainable artificial intelligence. J Dent 2023; 133:104519. [PMID: 37061117 DOI: 10.1016/j.jdent.2023.104519] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2022] [Revised: 03/28/2023] [Accepted: 04/12/2023] [Indexed: 04/17/2023] Open
Abstract
OBJECTIVE The aim of this study is to automatically assess the positional relationship between lower third molars (M3i) and the mandibular canal (MC) based on the panoramic radiograph(s) (PR(s)). MATERIAL AND METHODS A total of 1444 M3s were manually annotated and labeled on 863 PRs as a reference. A deep-learning approach, based on MobileNet-V2 combination with a skeletonization algorithm and a signed distance method, was trained and validated on 733 PRs with 1227 M3s to classify the positional relationship between M3i and MC into three categories. Subsequently, the trained algorithm was applied to a test set consisting of 130 PRs (217 M3s). Accuracy, precision, sensitivity, specificity, negative predictive value, and F1-score were calculated. RESULTS The proposed method achieved a weighted accuracy of 0.951, precision of 0.943, sensitivity of 0.941, specificity of 0.800, negative predictive value of 0.865 and an F1-score of 0.938. CONCLUSION AI-enhanced assessment of PRs can objectively, accurately, and reproducibly determine the positional relationship between M3i and MC. CLINICAL SIGNIFICANCE The use of such an explainable AI system can assist clinicians in the intuitive positional assessment of lower third molars and mandibular canals. Further research is required to automatically assess the risk of alveolar nerve injury on panoramic radiographs.
Collapse
Affiliation(s)
- Steven Kempers
- Department of Oral and Maxillofacial Surgery, Radboud University Nijmegen Medical Centre, Nijmegen, the Netherlands; Department of Artificial Intelligence, Radboud University, Nijmegen, the Netherlands
| | - Pieter van Lierop
- Department of Oral and Maxillofacial Surgery, Radboud University Nijmegen Medical Centre, Nijmegen, the Netherlands
| | - Tzu-Ming Harry Hsu
- MIT Computer Science & Artificial Intelligence Laboratory, 32 Vassar St, Cambridge, MA 02139, United States
| | | | - Stefaan Bergé
- Department of Oral and Maxillofacial Surgery, Radboud University Nijmegen Medical Centre, Nijmegen, the Netherlands
| | - Hossein Ghaeminia
- Department of Oral and Maxillofacial Surgery, Rijnstate Hospital, Arnhem, the Netherlands
| | - Tong Xi
- Department of Oral and Maxillofacial Surgery, Radboud University Nijmegen Medical Centre, Nijmegen, the Netherlands
| | - Shankeeth Vinayahalingam
- Department of Oral and Maxillofacial Surgery, Radboud University Nijmegen Medical Centre, Nijmegen, the Netherlands; Department of Artificial Intelligence, Radboud University, Nijmegen, the Netherlands; Department of Oral and Maxillofacial Surgery, Universitätsklinikum Münster, Münster, Germany.
| |
Collapse
|
49
|
Chow R, Midroni J, Kaur J, Boldt G, Liu G, Eng L, Liu FF, Haibe-Kains B, Lock M, Raman S. Use of artificial intelligence for cancer clinical trial enrollment: a systematic review and meta-analysis. J Natl Cancer Inst 2023; 115:365-374. [PMID: 36688707 PMCID: PMC10086633 DOI: 10.1093/jnci/djad013] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2022] [Revised: 12/13/2022] [Accepted: 01/11/2023] [Indexed: 01/24/2023] Open
Abstract
BACKGROUND The aim of this study is to provide a comprehensive understanding of the current landscape of artificial intelligence (AI) for cancer clinical trial enrollment and its predictive accuracy in identifying eligible patients for inclusion in such trials. METHODS Databases of PubMed, Embase, and Cochrane CENTRAL were searched until June 2022. Articles were included if they reported on AI actively being used in the clinical trial enrollment process. Narrative synthesis was conducted among all extracted data: accuracy, sensitivity, specificity, positive predictive value, and negative predictive value. For studies where the 2x2 contingency table could be calculated or supplied by authors, a meta-analysis to calculate summary statistics was conducted using the hierarchical summary receiver operating characteristics curve model. RESULTS Ten articles reporting on more than 50 000 patients in 19 datasets were included. Accuracy, sensitivity, and specificity exceeded 80% in all but 1 dataset. Positive predictive value exceeded 80% in 5 of 17 datasets. Negative predictive value exceeded 80% in all datasets. Summary sensitivity was 90.5% (95% confidence interval [CI] = 70.9% to 97.4%); summary specificity was 99.3% (95% CI = 81.8% to 99.9%). CONCLUSIONS AI demonstrated comparable, if not superior, performance to manual screening for patient enrollment into cancer clinical trials. As well, AI is highly efficient, requiring less time and human resources to screen patients. AI should be further investigated and implemented for patient recruitment into cancer clinical trials. Future research should validate the use of AI for clinical trials enrollment in less resource-rich regions and ensure broad inclusion for generalizability to all sexes, ages, and ethnicities.
Collapse
Affiliation(s)
- Ronald Chow
- Princess Margaret Cancer Centre, University Health Network, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
- London Regional Cancer Program, London Health Sciences Centre, Schulich School of Medicine and Dentistry, University of Western Ontario, London, ON, Canada
- Institute of Biomedical Engineering, Faculty of Applied Science and Engineering, University of Toronto, Toronto, ON, Canada
| | - Julie Midroni
- Princess Margaret Cancer Centre, University Health Network, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
| | - Jagdeep Kaur
- London Regional Cancer Program, London Health Sciences Centre, Schulich School of Medicine and Dentistry, University of Western Ontario, London, ON, Canada
| | - Gabriel Boldt
- London Regional Cancer Program, London Health Sciences Centre, Schulich School of Medicine and Dentistry, University of Western Ontario, London, ON, Canada
| | - Geoffrey Liu
- Princess Margaret Cancer Centre, University Health Network, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
| | - Lawson Eng
- Princess Margaret Cancer Centre, University Health Network, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
| | - Fei-Fei Liu
- Princess Margaret Cancer Centre, University Health Network, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
| | - Benjamin Haibe-Kains
- Princess Margaret Cancer Centre, University Health Network, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
| | - Michael Lock
- London Regional Cancer Program, London Health Sciences Centre, Schulich School of Medicine and Dentistry, University of Western Ontario, London, ON, Canada
| | - Srinivas Raman
- Princess Margaret Cancer Centre, University Health Network, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
50
|
Johns M, Meurers T, Wirth FN, Haber AC, Müller A, Halilovic M, Balzer F, Prasser F. Data Provenance in Biomedical Research: Scoping Review. J Med Internet Res 2023; 25:e42289. [PMID: 36972116 PMCID: PMC10132013 DOI: 10.2196/42289] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2022] [Revised: 12/14/2022] [Accepted: 12/23/2022] [Indexed: 03/29/2023] Open
Abstract
BACKGROUND Data provenance refers to the origin, processing, and movement of data. Reliable and precise knowledge about data provenance has great potential to improve reproducibility as well as quality in biomedical research and, therefore, to foster good scientific practice. However, despite the increasing interest on data provenance technologies in the literature and their implementation in other disciplines, these technologies have not yet been widely adopted in biomedical research. OBJECTIVE The aim of this scoping review was to provide a structured overview of the body of knowledge on provenance methods in biomedical research by systematizing articles covering data provenance technologies developed for or used in this application area; describing and comparing the functionalities as well as the design of the provenance technologies used; and identifying gaps in the literature, which could provide opportunities for future research on technologies that could receive more widespread adoption. METHODS Following a methodological framework for scoping studies and the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines, articles were identified by searching the PubMed, IEEE Xplore, and Web of Science databases and subsequently screened for eligibility. We included original articles covering software-based provenance management for scientific research published between 2010 and 2021. A set of data items was defined along the following five axes: publication metadata, application scope, provenance aspects covered, data representation, and functionalities. The data items were extracted from the articles, stored in a charting spreadsheet, and summarized in tables and figures. RESULTS We identified 44 original articles published between 2010 and 2021. We found that the solutions described were heterogeneous along all axes. We also identified relationships among motivations for the use of provenance information, feature sets (capture, storage, retrieval, visualization, and analysis), and implementation details such as the data models and technologies used. The important gap that we identified is that only a few publications address the analysis of provenance data or use established provenance standards, such as PROV. CONCLUSIONS The heterogeneity of provenance methods, models, and implementations found in the literature points to the lack of a unified understanding of provenance concepts for biomedical data. Providing a common framework, a biomedical reference, and benchmarking data sets could foster the development of more comprehensive provenance solutions.
Collapse
Affiliation(s)
- Marco Johns
- Medical Informatics Group, Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Thierry Meurers
- Medical Informatics Group, Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Felix N Wirth
- Medical Informatics Group, Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Anna C Haber
- Medical Informatics Group, Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Armin Müller
- Medical Informatics Group, Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Mehmed Halilovic
- Medical Informatics Group, Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Felix Balzer
- Institute of Medical Informatics, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Fabian Prasser
- Medical Informatics Group, Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Berlin, Germany
| |
Collapse
|