1
|
Hosseini MS, Bejnordi BE, Trinh VQH, Chan L, Hasan D, Li X, Yang S, Kim T, Zhang H, Wu T, Chinniah K, Maghsoudlou S, Zhang R, Zhu J, Khaki S, Buin A, Chaji F, Salehi A, Nguyen BN, Samaras D, Plataniotis KN. Computational pathology: A survey review and the way forward. J Pathol Inform 2024; 15:100357. [PMID: 38420608 PMCID: PMC10900832 DOI: 10.1016/j.jpi.2023.100357] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2023] [Revised: 12/21/2023] [Accepted: 12/23/2023] [Indexed: 03/02/2024] Open
Abstract
Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 800 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field's future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub. Updated version of this draft can also be found from arXiv.
Collapse
Affiliation(s)
- Mahdi S Hosseini
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | | | - Vincent Quoc-Huy Trinh
- Institute for Research in Immunology and Cancer of the University of Montreal, Montreal, QC H3T 1J4, Canada
| | - Lyndon Chan
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Danial Hasan
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Xingwen Li
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Stephen Yang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Taehyo Kim
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Haochen Zhang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Theodore Wu
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Kajanan Chinniah
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Sina Maghsoudlou
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | - Ryan Zhang
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Jiadai Zhu
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Samir Khaki
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| | - Andrei Buin
- Huron Digitial Pathology, St. Jacobs, ON N0B 2N0, Canada
| | - Fatemeh Chaji
- Department of Computer Science and Software Engineering (CSSE), Concordia Univeristy, Montreal, QC H3H 2R9, Canada
| | - Ala Salehi
- Department of Electrical and Computer Engineering, University of New Brunswick, Fredericton, NB E3B 5A3, Canada
| | - Bich Ngoc Nguyen
- University of Montreal Hospital Center, Montreal, QC H2X 0C2, Canada
| | - Dimitris Samaras
- Department of Computer Science, Stony Brook University, Stony Brook, NY 11794, United States
| | - Konstantinos N Plataniotis
- The Edward S. Rogers Sr. Department of Electrical & Computer Engineering (ECE), University of Toronto, Toronto, ON M5S 3G4, Canada
| |
Collapse
|
2
|
Wang J, Jin Y, Jiang A, Chen W, Shan G, Gu Y, Ming Y, Li J, Yue C, Huang Z, Librach C, Lin G, Wang X, Zhao H, Sun Y, Zhang Z. Testing the generalizability and effectiveness of deep learning models among clinics: sperm detection as a pilot study. Reprod Biol Endocrinol 2024; 22:59. [PMID: 38778327 PMCID: PMC11110326 DOI: 10.1186/s12958-024-01232-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/03/2024] [Accepted: 05/14/2024] [Indexed: 05/25/2024] Open
Abstract
BACKGROUND Deep learning has been increasingly investigated for assisting clinical in vitro fertilization (IVF). The first technical step in many tasks is to visually detect and locate sperm, oocytes, and embryos in images. For clinical deployment of such deep learning models, different clinics use different image acquisition hardware and different sample preprocessing protocols, raising the concern over whether the reported accuracy of a deep learning model by one clinic could be reproduced in another clinic. Here we aim to investigate the effect of each imaging factor on the generalizability of object detection models, using sperm analysis as a pilot example. METHODS Ablation studies were performed using state-of-the-art models for detecting human sperm to quantitatively assess how model precision (false-positive detection) and recall (missed detection) were affected by imaging magnification, imaging mode, and sample preprocessing protocols. The results led to the hypothesis that the richness of image acquisition conditions in a training dataset deterministically affects model generalizability. The hypothesis was tested by first enriching the training dataset with a wide range of imaging conditions, then validated through internal blind tests on new samples and external multi-center clinical validations. RESULTS Ablation experiments revealed that removing subsets of data from the training dataset significantly reduced model precision. Removing raw sample images from the training dataset caused the largest drop in model precision, whereas removing 20x images caused the largest drop in model recall. by incorporating different imaging and sample preprocessing conditions into a rich training dataset, the model achieved an intraclass correlation coefficient (ICC) of 0.97 (95% CI: 0.94-0.99) for precision, and an ICC of 0.97 (95% CI: 0.93-0.99) for recall. Multi-center clinical validation showed no significant differences in model precision or recall across different clinics and applications. CONCLUSIONS The results validated the hypothesis that the richness of data in the training dataset is a key factor impacting model generalizability. These findings highlight the importance of diversity in a training dataset for model evaluation and suggest that future deep learning models in andrology and reproductive medicine should incorporate comprehensive feature sets for enhanced generalizability across clinics.
Collapse
Affiliation(s)
- Jiaqi Wang
- School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, China
| | - Yufei Jin
- School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, China
| | - Aojun Jiang
- Department of Mechanical Engineering, University of Toronto, Toronto, Canada
| | - Wenyuan Chen
- Department of Mechanical Engineering, University of Toronto, Toronto, Canada
| | - Guanqiao Shan
- Department of Mechanical Engineering, University of Toronto, Toronto, Canada
| | - Yifan Gu
- Institute of Reproductive and Stem Cell Engineering, School of Basic Medical Science, Central South University, Changsha, China
- Reproductive & Genetic Hospital of Citic-Xiangya, Changsha, China
| | - Yue Ming
- School of Medicine, The Chinese University of Hong Kong, Shenzhen, China
| | - Jichang Li
- School of Medicine, The Chinese University of Hong Kong, Shenzhen, China
| | - Chunfeng Yue
- Suzhou Boundless Medical Technology Ltd., Co., Suzhou, China
| | - Zongjie Huang
- Suzhou Boundless Medical Technology Ltd., Co., Suzhou, China
| | | | - Ge Lin
- Institute of Reproductive and Stem Cell Engineering, School of Basic Medical Science, Central South University, Changsha, China
- Reproductive & Genetic Hospital of Citic-Xiangya, Changsha, China
| | - Xibu Wang
- The 3rd Affiliated Hospital of Shenzhen University, Shenzhen, China
| | - Huan Zhao
- The 3rd Affiliated Hospital of Shenzhen University, Shenzhen, China.
| | - Yu Sun
- Department of Mechanical Engineering, University of Toronto, Toronto, Canada.
- Department of Computer Science, University of Toronto, Toronto, Canada.
- Institute of Biomedical Engineering, University of Toronto, Toronto, Canada.
- Department of Electrical and Computer Engineering, University of Toronto, Toronto, Canada.
| | - Zhuoran Zhang
- School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, China.
| |
Collapse
|
3
|
Salloch S, Eriksen A. What Are Humans Doing in the Loop? Co-Reasoning and Practical Judgment When Using Machine Learning-Driven Decision Aids. THE AMERICAN JOURNAL OF BIOETHICS : AJOB 2024:1-12. [PMID: 38767971 DOI: 10.1080/15265161.2024.2353800] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/22/2024]
Abstract
Within the ethical debate on Machine Learning-driven decision support systems (ML_CDSS), notions such as "human in the loop" or "meaningful human control" are often cited as being necessary for ethical legitimacy. In addition, ethical principles usually serve as the major point of reference in ethical guidance documents, stating that conflicts between principles need to be weighed and balanced against each other. Starting from a neo-Kantian viewpoint inspired by Onora O'Neill, this article makes a concrete suggestion of how to interpret the role of the "human in the loop" and to overcome the perspective of rivaling ethical principles in the evaluation of AI in health care. We argue that patients should be perceived as "fellow workers" and epistemic partners in the interpretation of ML_CDSS outputs. We further highlight that a meaningful process of integrating (rather than weighing and balancing) ethical principles is most appropriate in the evaluation of medical AI.
Collapse
|
4
|
McGenity C, Clarke EL, Jennings C, Matthews G, Cartlidge C, Freduah-Agyemang H, Stocken DD, Treanor D. Artificial intelligence in digital pathology: a systematic review and meta-analysis of diagnostic test accuracy. NPJ Digit Med 2024; 7:114. [PMID: 38704465 PMCID: PMC11069583 DOI: 10.1038/s41746-024-01106-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2023] [Accepted: 04/12/2024] [Indexed: 05/06/2024] Open
Abstract
Ensuring diagnostic performance of artificial intelligence (AI) before introduction into clinical practice is essential. Growing numbers of studies using AI for digital pathology have been reported over recent years. The aim of this work is to examine the diagnostic accuracy of AI in digital pathology images for any disease. This systematic review and meta-analysis included diagnostic accuracy studies using any type of AI applied to whole slide images (WSIs) for any disease. The reference standard was diagnosis by histopathological assessment and/or immunohistochemistry. Searches were conducted in PubMed, EMBASE and CENTRAL in June 2022. Risk of bias and concerns of applicability were assessed using the QUADAS-2 tool. Data extraction was conducted by two investigators and meta-analysis was performed using a bivariate random effects model, with additional subgroup analyses also performed. Of 2976 identified studies, 100 were included in the review and 48 in the meta-analysis. Studies were from a range of countries, including over 152,000 whole slide images (WSIs), representing many diseases. These studies reported a mean sensitivity of 96.3% (CI 94.1-97.7) and mean specificity of 93.3% (CI 90.5-95.4). There was heterogeneity in study design and 99% of studies identified for inclusion had at least one area at high or unclear risk of bias or applicability concerns. Details on selection of cases, division of model development and validation data and raw performance data were frequently ambiguous or missing. AI is reported as having high diagnostic accuracy in the reported areas but requires more rigorous evaluation of its performance.
Collapse
Affiliation(s)
- Clare McGenity
- University of Leeds, Leeds, UK.
- Leeds Teaching Hospitals NHS Trust, Leeds, UK.
| | - Emily L Clarke
- University of Leeds, Leeds, UK
- Leeds Teaching Hospitals NHS Trust, Leeds, UK
| | - Charlotte Jennings
- University of Leeds, Leeds, UK
- Leeds Teaching Hospitals NHS Trust, Leeds, UK
| | | | | | | | | | - Darren Treanor
- University of Leeds, Leeds, UK
- Leeds Teaching Hospitals NHS Trust, Leeds, UK
- Department of Clinical Pathology and Department of Clinical and Experimental Medicine, Linköping University, Linköping, Sweden
- Centre for Medical Image Science and Visualization (CMIV), Linköping University, Linköping, Sweden
| |
Collapse
|
5
|
Varghese C, Harrison EM, O'Grady G, Topol EJ. Artificial intelligence in surgery. Nat Med 2024; 30:1257-1268. [PMID: 38740998 DOI: 10.1038/s41591-024-02970-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2024] [Accepted: 04/03/2024] [Indexed: 05/16/2024]
Abstract
Artificial intelligence (AI) is rapidly emerging in healthcare, yet applications in surgery remain relatively nascent. Here we review the integration of AI in the field of surgery, centering our discussion on multifaceted improvements in surgical care in the preoperative, intraoperative and postoperative space. The emergence of foundation model architectures, wearable technologies and improving surgical data infrastructures is enabling rapid advances in AI interventions and utility. We discuss how maturing AI methods hold the potential to improve patient outcomes, facilitate surgical education and optimize surgical care. We review the current applications of deep learning approaches and outline a vision for future advances through multimodal foundation models.
Collapse
Affiliation(s)
- Chris Varghese
- Department of Surgery, University of Auckland, Auckland, New Zealand
| | - Ewen M Harrison
- Centre for Medical Informatics, Usher Institute, University of Edinburgh, Edinburgh, UK
| | - Greg O'Grady
- Department of Surgery, University of Auckland, Auckland, New Zealand
- Auckland Bioengineering Institute, University of Auckland, Auckland, New Zealand
| | - Eric J Topol
- Scripps Research Translational Institute, La Jolla, CA, USA.
| |
Collapse
|
6
|
Li Y, Liu H, Lv Q, Long J. Diagnosis model of early Pneumocystis jirovecii pneumonia based on convolutional neural network: a comparison with traditional PCR diagnostic method. BMC Pulm Med 2024; 24:205. [PMID: 38664747 PMCID: PMC11046959 DOI: 10.1186/s12890-024-02987-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2023] [Accepted: 03/29/2024] [Indexed: 04/28/2024] Open
Abstract
BACKGROUND Pneumocystis jirovecii pneumonia (PJP) is an interstitial pneumonia caused by pneumocystis jirovecii (PJ). The diagnosis of PJP primarily relies on the detection of the pathogen from lower respiratory tract specimens. However, it faces challenges such as difficulty in obtaining specimens and low detection rates. In the clinical diagnosis process, it is necessary to combine clinical symptoms, serological test results, chest Computed tomography (CT) images, molecular biology techniques, and metagenomics next-generation sequencing (mNGS) for comprehensive analysis. PURPOSE This study aims to overcome the limitations of traditional PJP diagnosis methods and develop a non-invasive, efficient, and accurate diagnostic approach for PJP. By using this method, patients can receive early diagnosis and treatment, effectively improving their prognosis. METHODS We constructed an intelligent diagnostic model for PJP based on the different Convolutional Neural Networks. Firstly, we used the Convolutional Neural Network to extract CT image features from patients. Then, we fused the CT image features with clinical information features using a feature fusion function. Finally, the fused features were input into the classification network to obtain the patient's diagnosis result. RESULTS In this study, for the diagnosis of PJP, the accuracy of the traditional PCR diagnostic method is 77.58%, while the mean accuracy of the optimal diagnostic model based on convolutional neural networks is 88.90%. CONCLUSION The accuracy of the diagnostic method proposed in this paper is 11.32% higher than that of the traditional PCR diagnostic method. The method proposed in this paper is an efficient, accurate, and non-invasive early diagnosis approach for PJP.
Collapse
Affiliation(s)
- Yingying Li
- Department of Clinical Laboratory, Zhujiang Hospital, Southern Medical University, Guangzhou, China
| | - Hailin Liu
- School of Biomedical Engineering, Southern Medical University, Guangzhou, China
- Department of Information, Zhujiang Hospital, Southern Medical University, Guangzhou, China
| | - Qingwen Lv
- Department of Information, Zhujiang Hospital, Southern Medical University, Guangzhou, China.
| | - Jun Long
- Department of Clinical Laboratory, Zhujiang Hospital, Southern Medical University, Guangzhou, China.
| |
Collapse
|
7
|
Omar M, Xu Z, Rand SB, Alexanderani MK, Salles DC, Valencia I, Schaeffer EM, Robinson BD, Lotan TL, Loda M, Marchionni L. Semi-Supervised, Attention-Based Deep Learning for Predicting TMPRSS2:ERG Fusion Status in Prostate Cancer Using Whole Slide Images. Mol Cancer Res 2024; 22:347-359. [PMID: 38284821 PMCID: PMC10985477 DOI: 10.1158/1541-7786.mcr-23-0639] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2023] [Revised: 12/26/2023] [Accepted: 01/22/2024] [Indexed: 01/30/2024]
Abstract
IMPLICATIONS Our study illuminates the potential of deep learning in effectively inferring key prostate cancer genetic alterations from the tissue morphology depicted in routinely available histology slides, offering a cost-effective method that could revolutionize diagnostic strategies in oncology.
Collapse
Affiliation(s)
- Mohamed Omar
- Department of Pathology and Laboratory Medicine, Weill Cornell Medicine, New York, New York
- Dana-Farber Cancer Institute, Boston, Massachusetts
| | - Zhuoran Xu
- Department of Pathology and Laboratory Medicine, Weill Cornell Medicine, New York, New York
- Dana-Farber Cancer Institute, Boston, Massachusetts
| | - Sophie B. Rand
- Department of Pathology and Laboratory Medicine, Weill Cornell Medicine, New York, New York
- Dana-Farber Cancer Institute, Boston, Massachusetts
| | | | - Daniela C. Salles
- Department of Pathology, Johns Hopkins University, Baltimore, Maryland
| | - Itzel Valencia
- Department of Pathology and Laboratory Medicine, Weill Cornell Medicine, New York, New York
| | | | - Brian D. Robinson
- Department of Pathology and Laboratory Medicine, Weill Cornell Medicine, New York, New York
| | - Tamara L. Lotan
- Department of Pathology, Johns Hopkins University, Baltimore, Maryland
| | - Massimo Loda
- Department of Pathology and Laboratory Medicine, Weill Cornell Medicine, New York, New York
| | - Luigi Marchionni
- Department of Pathology and Laboratory Medicine, Weill Cornell Medicine, New York, New York
| |
Collapse
|
8
|
Jongsma KR, Sand M, Milota M. Why we should not mistake accuracy of medical AI for efficiency. NPJ Digit Med 2024; 7:57. [PMID: 38438477 PMCID: PMC10912629 DOI: 10.1038/s41746-024-01047-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2023] [Accepted: 02/16/2024] [Indexed: 03/06/2024] Open
Affiliation(s)
- Karin Rolanda Jongsma
- Bioethics & Health Humanities, Julius Center, University Medical Center Utrecht, Utrecht University, PO Box 85500, 3508 CA, Utrecht, The Netherlands.
| | - Martin Sand
- TU Delft, Department of Values, Technology and Innovation, Faculty of Technology, Policy and Management, Jaffalaan 5, 2628 BX, Delft, The Netherlands
| | - Megan Milota
- Bioethics & Health Humanities, Julius Center, University Medical Center Utrecht, Utrecht University, PO Box 85500, 3508 CA, Utrecht, The Netherlands
| |
Collapse
|
9
|
Gupta P, Basu S, Arora C. Applications of artificial intelligence in biliary tract cancers. Indian J Gastroenterol 2024:10.1007/s12664-024-01518-0. [PMID: 38427281 DOI: 10.1007/s12664-024-01518-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/03/2023] [Accepted: 12/29/2023] [Indexed: 03/02/2024]
Abstract
Biliary tract cancers are malignant neoplasms arising from bile duct epithelial cells. They include cholangiocarcinomas and gallbladder cancer. Gallbladder cancer has a marked geographical preference and is one of the most common cancers in women in northern India. Biliary tract cancers are usually diagnosed at an advanced, unresectable stage. Hence, the prognosis is extremely dismal. The five-year survival rate in advanced gallbladder cancer is < 5%. Hence, early detection and radical surgery are critical to improving biliary tract cancer prognoses. Radiological imaging plays an essential role in diagnosing and managing biliary tract cancers. However, the diagnosis is challenging because the biliary tract is affected by many diseases that may have radiological appearances similar to cancer. Artificial intelligence (AI) can improve radiologists' performance in various tasks. Deep learning (DL)-based approaches are increasingly incorporated into medical imaging to improve diagnostic performance. This paper reviews the AI-based strategies in biliary tract cancers to improve the diagnosis and prognosis.
Collapse
Affiliation(s)
- Pankaj Gupta
- Department of Radiodiagnosis and Imaging, Postgraduate Institute of Medical Education and Research, Chandigarh, 160 012, India.
| | - Soumen Basu
- Department of Computer Science and Engineering, Indian Institute of Technology - Delhi, New Delhi, 110 016, India
| | - Chetan Arora
- Department of Computer Science and Engineering, Indian Institute of Technology - Delhi, New Delhi, 110 016, India
| |
Collapse
|
10
|
Huang J, Bai X, Qiu Y, He X. Application of AI on cholangiocarcinoma. Front Oncol 2024; 14:1324222. [PMID: 38347839 PMCID: PMC10859478 DOI: 10.3389/fonc.2024.1324222] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2023] [Accepted: 01/08/2024] [Indexed: 02/15/2024] Open
Abstract
Cholangiocarcinoma, classified as intrahepatic, perihilar, and extrahepatic, is considered a deadly malignancy of the hepatobiliary system. Most cases of cholangiocarcinoma are asymptomatic. Therefore, early detection of cholangiocarcinoma is significant but still challenging. The routine screening of a tumor lacks specificity and accuracy. With the application of AI, high-risk patients can be easily found by analyzing their clinical characteristics, serum biomarkers, and medical images. Moreover, AI can be used to predict the prognosis including recurrence risk and metastasis. Although they have some limitations, AI algorithms will still significantly improve many aspects of cholangiocarcinoma in the medical field with the development of computing power and technology.
Collapse
Affiliation(s)
| | | | | | - Xiaodong He
- Department of General Surgery, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Peking Union Medical College and Chinese Academy of Medical Sciences, Beijing, China
| |
Collapse
|
11
|
Klauschen F, Dippel J, Keyl P, Jurmeister P, Bockmayr M, Mock A, Buchstab O, Alber M, Ruff L, Montavon G, Müller KR. Toward Explainable Artificial Intelligence for Precision Pathology. ANNUAL REVIEW OF PATHOLOGY 2024; 19:541-570. [PMID: 37871132 DOI: 10.1146/annurev-pathmechdis-051222-113147] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/25/2023]
Abstract
The rapid development of precision medicine in recent years has started to challenge diagnostic pathology with respect to its ability to analyze histological images and increasingly large molecular profiling data in a quantitative, integrative, and standardized way. Artificial intelligence (AI) and, more precisely, deep learning technologies have recently demonstrated the potential to facilitate complex data analysis tasks, including clinical, histological, and molecular data for disease classification; tissue biomarker quantification; and clinical outcome prediction. This review provides a general introduction to AI and describes recent developments with a focus on applications in diagnostic pathology and beyond. We explain limitations including the black-box character of conventional AI and describe solutions to make machine learning decisions more transparent with so-called explainable AI. The purpose of the review is to foster a mutual understanding of both the biomedical and the AI side. To that end, in addition to providing an overview of the relevant foundations in pathology and machine learning, we present worked-through examples for a better practical understanding of what AI can achieve and how it should be done.
Collapse
Affiliation(s)
- Frederick Klauschen
- Institute of Pathology, Ludwig-Maximilians-Universität München, Munich, Germany;
- Institute of Pathology, Charité Universitätsmedizin Berlin, Berlin, Germany
- Berlin Institute for the Foundations of Learning and Data (BIFOLD), Berlin, Germany
- German Cancer Consortium, German Cancer Research Center (DKTK/DKFZ), Munich Partner Site, Munich, Germany
| | - Jonas Dippel
- Berlin Institute for the Foundations of Learning and Data (BIFOLD), Berlin, Germany
- Machine Learning Group, Department of Electrical Engineering and Computer Science, Technische Universität Berlin, Berlin, Germany;
| | - Philipp Keyl
- Institute of Pathology, Ludwig-Maximilians-Universität München, Munich, Germany;
| | - Philipp Jurmeister
- Institute of Pathology, Ludwig-Maximilians-Universität München, Munich, Germany;
- German Cancer Consortium, German Cancer Research Center (DKTK/DKFZ), Munich Partner Site, Munich, Germany
| | - Michael Bockmayr
- Institute of Pathology, Charité Universitätsmedizin Berlin, Berlin, Germany
- Department of Pediatric Hematology and Oncology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
- Research Institute Children's Cancer Center Hamburg, Hamburg, Germany
| | - Andreas Mock
- Institute of Pathology, Ludwig-Maximilians-Universität München, Munich, Germany;
- German Cancer Consortium, German Cancer Research Center (DKTK/DKFZ), Munich Partner Site, Munich, Germany
| | - Oliver Buchstab
- Institute of Pathology, Ludwig-Maximilians-Universität München, Munich, Germany;
| | - Maximilian Alber
- Institute of Pathology, Charité Universitätsmedizin Berlin, Berlin, Germany
- Aignostics, Berlin, Germany
| | | | - Grégoire Montavon
- Berlin Institute for the Foundations of Learning and Data (BIFOLD), Berlin, Germany
- Machine Learning Group, Department of Electrical Engineering and Computer Science, Technische Universität Berlin, Berlin, Germany;
- Department of Mathematics and Computer Science, Freie Universität Berlin, Berlin, Germany
| | - Klaus-Robert Müller
- Berlin Institute for the Foundations of Learning and Data (BIFOLD), Berlin, Germany
- Machine Learning Group, Department of Electrical Engineering and Computer Science, Technische Universität Berlin, Berlin, Germany;
- Department of Artificial Intelligence, Korea University, Seoul, Korea
- Max Planck Institute for Informatics, Saarbrücken, Germany
| |
Collapse
|
12
|
Moterani VC, Abbade JF, Borges VTM, Fonseca CGF, Desiderio N, Moterani Junior NJW, Gonçalves Moterani LBB. [Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extensionDiretrizes para protocolos de ensaios clínicos com intervenções que utilizam inteligência artificial: a extensão SPIRIT-AI]. Rev Panam Salud Publica 2023; 47:e149. [PMID: 38361499 PMCID: PMC10868409 DOI: 10.26633/rpsp.2023.149] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2020] [Accepted: 07/23/2020] [Indexed: 01/10/2024] Open
Abstract
The SPIRIT 2013 statement aims to improve the completeness of clinical trial protocol reporting by providing evidence-based recommendations for the minimum set of items to be addressed. This guidance has been instrumental in promoting transparent evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate their impact on health outcomes. The SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence) extension is a new reporting guideline for clinical trial protocols evaluating interventions with an AI component. It was developed in parallel with its companion statement for trial reports: CONSORT-AI (Consolidated Standards of Reporting Trials-Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 26 candidate items, which were consulted upon by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The SPIRIT-AI extension includes 15 new items that were considered sufficiently important for clinical trial protocols of AI interventions. These new items should be routinely reported in addition to the core SPIRIT 2013 items. SPIRIT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention will be integrated, considerations for the handling of input and output data, the human-AI interaction and analysis of error cases. SPIRIT-AI will help promote transparency and completeness for clinical trial protocols for AI interventions. Its use will assist editors and peer reviewers, as well as the general readership, to understand, interpret and critically appraise the design and risk of bias for a planned clinical trial.
Collapse
Affiliation(s)
- Vinicius Cesar Moterani
- Universidade Estadual Paulista “Júlio de Mesquita Filho,”BotucatuBrazilUniversidade Estadual Paulista “Júlio de Mesquita Filho,” Botucatu, Brazil
| | - Joelcio Francisco Abbade
- Universidade Estadual Paulista “Júlio de Mesquita Filho,”BotucatuBrazilUniversidade Estadual Paulista “Júlio de Mesquita Filho,” Botucatu, Brazil
| | - Vera Therezinha Medeiros Borges
- Universidade Estadual Paulista “Júlio de Mesquita Filho,”BotucatuBrazilUniversidade Estadual Paulista “Júlio de Mesquita Filho,” Botucatu, Brazil
| | - Cecilia Guimarães Ferreira Fonseca
- Universidade Estadual Paulista “Júlio de Mesquita Filho,”BotucatuBrazilUniversidade Estadual Paulista “Júlio de Mesquita Filho,” Botucatu, Brazil
| | - Nathalia Desiderio
- Marilia Medical SchoolMariliaBrazilMarilia Medical School, Marilia, Brazil
| | | | | |
Collapse
|
13
|
Ke J, Liu K, Sun Y, Xue Y, Huang J, Lu Y, Dai J, Chen Y, Han X, Shen Y, Shen D. Artifact Detection and Restoration in Histology Images With Stain-Style and Structural Preservation. IEEE TRANSACTIONS ON MEDICAL IMAGING 2023; 42:3487-3500. [PMID: 37352087 DOI: 10.1109/tmi.2023.3288940] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/25/2023]
Abstract
The artifacts in histology images may encumber the accurate interpretation of medical information and cause misdiagnosis. Accordingly, prepending manual quality control of artifacts considerably decreases the degree of automation. To close this gap, we propose a methodical pre-processing framework to detect and restore artifacts, which minimizes their impact on downstream AI diagnostic tasks. First, the artifact recognition network AR-Classifier first differentiates common artifacts from normal tissues, e.g., tissue folds, marking dye, tattoo pigment, spot, and out-of-focus, and also catalogs artifact patches by their restorability. Then, the succeeding artifact restoration network AR-CycleGAN performs de-artifact processing where stain styles and tissue structures can be maximally retained. We construct a benchmark for performance evaluation, curated from both clinically collected WSIs and public datasets of colorectal and breast cancer. The functional structures are compared with state-of-the-art methods, and also comprehensively evaluated by multiple metrics across multiple tasks, including artifact classification, artifact restoration, downstream diagnostic tasks of tumor classification and nuclei segmentation. The proposed system allows full automation of deep learning based histology image analysis without human intervention. Moreover, the structure-independent characteristic enables its processing with various artifact subtypes. The source code and data in this research are available at https://github.com/yunboer/AR-classifier-and-AR-CycleGAN.
Collapse
|
14
|
Masoumian Hosseini M, Masoumian Hosseini ST, Qayumi K, Hosseinzadeh S, Sajadi Tabar SS. Smartwatches in healthcare medicine: assistance and monitoring; a scoping review. BMC Med Inform Decis Mak 2023; 23:248. [PMID: 37924029 PMCID: PMC10625201 DOI: 10.1186/s12911-023-02350-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2023] [Accepted: 10/22/2023] [Indexed: 11/06/2023] Open
Abstract
Smartwatches have become increasingly popular in recent times because of their capacity to track different health indicators, including heart rate, patterns of sleep, and physical movements. This scoping review aims to explore the utilisation of smartwatches within the healthcare sector. According to Arksey and O'Malley's methodology, an organised search was performed in PubMed/Medline, Scopus, Embase, Web of Science, ERIC and Google Scholar. In our search strategy, 761 articles were returned. The exclusion/inclusion criteria were applied. Finally, 35 articles were selected for extracting data. These included six studies on stress monitoring, six on movement disorders, three on sleep tracking, three on blood pressure, two on heart disease, six on covid pandemic, three on safety and six on validation. The use of smartwatches has been found to be effective in diagnosing the symptoms of various diseases. In particular, smartwatches have shown promise in detecting heart diseases, movement disorders, and even early signs of COVID-19. Nevertheless, it should be emphasised that there is an ongoing discussion concerning the reliability of smartwatch diagnoses within healthcare systems. Despite the potential advantages offered by utilising smartwatches for disease detection, it is imperative to approach their data interpretation with prudence. The discrepancies in detection between smartwatches and their algorithms have important implications for healthcare use. The accuracy and reliability of the algorithms used are crucial, as well as high accuracy in detecting changes in health status by the smartwatches themselves. This calls for the development of medical watches and the creation of AI-hospital assistants. These assistants will be designed to help with patient monitoring, appointment scheduling, and medication management tasks. They can educate patients and answer common questions, freeing healthcare providers to focus on more complex tasks.
Collapse
Affiliation(s)
- Mohsen Masoumian Hosseini
- Department of E-Learning in Medical Sciences, Tehran University of Medical Sciences, Tehran, Iran
- CyberPatient Research Affiliate, Interactive Health International, Department of the surgery, University of British Columbia, Vancouver, Canada
| | - Seyedeh Toktam Masoumian Hosseini
- CyberPatient Research Affiliate, Interactive Health International, Department of the surgery, University of British Columbia, Vancouver, Canada.
- Department of Nursing, School of Nursing and Midwifery, Torbat Heydariyeh University of Medical Sciences, Torbat Heydariyeh, Iran.
| | - Karim Qayumi
- Professor at Department of Surgery, University of British Columbia, Vancouver, Canada
| | - Shahriar Hosseinzadeh
- CyberPatient Research Coordinator, Interactive Health International, Department of Surgery, University of British Columbia, Vancouver, Canada
| | | |
Collapse
|
15
|
Albrecht T, Rossberg A, Albrecht JD, Nicolay JP, Straub BK, Gerber TS, Albrecht M, Brinkmann F, Charbel A, Schwab C, Schreck J, Brobeil A, Flechtenmacher C, von Winterfeld M, Köhler BC, Springfeld C, Mehrabi A, Singer S, Vogel MN, Neumann O, Stenzinger A, Schirmacher P, Weis CA, Roessler S, Kather JN, Goeppert B. Deep Learning-Enabled Diagnosis of Liver Adenocarcinoma. Gastroenterology 2023; 165:1262-1275. [PMID: 37562657 DOI: 10.1053/j.gastro.2023.07.026] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/26/2023] [Revised: 07/18/2023] [Accepted: 07/20/2023] [Indexed: 08/12/2023]
Abstract
BACKGROUND & AIMS Diagnosis of adenocarcinoma in the liver is a frequent scenario in routine pathology and has a critical impact on clinical decision making. However, rendering a correct diagnosis can be challenging, and often requires the integration of clinical, radiologic, and immunohistochemical information. We present a deep learning model (HEPNET) to distinguish intrahepatic cholangiocarcinoma from colorectal liver metastasis, as the most frequent primary and secondary forms of liver adenocarcinoma, with clinical grade accuracy using H&E-stained whole-slide images. METHODS HEPNET was trained on 714,589 image tiles from 456 patients who were randomly selected in a stratified manner from a pool of 571 patients who underwent surgical resection or biopsy at Heidelberg University Hospital. Model performance was evaluated on a hold-out internal test set comprising 115 patients and externally validated on 159 patients recruited at Mainz University Hospital. RESULTS On the hold-out internal test set, HEPNET achieved an area under the receiver operating characteristic curve of 0.994 (95% CI, 0.989-1.000) and an accuracy of 96.522% (95% CI, 94.521%-98.694%) at the patient level. Validation on the external test set yielded an area under the receiver operating characteristic curve of 0.997 (95% CI, 0.995-1.000), corresponding to an accuracy of 98.113% (95% CI, 96.907%-100.000%). HEPNET surpassed the performance of 6 pathology experts with different levels of experience in a reader study of 50 patients (P = .0005), boosted the performance of resident pathologists to the level of senior pathologists, and reduced potential downstream analyses. CONCLUSIONS We provided a ready-to-use tool with clinical grade performance that may facilitate routine pathology by rendering a definitive diagnosis and guiding ancillary testing. The incorporation of HEPNET into pathology laboratories may optimize the diagnostic workflow, complemented by test-related labor and cost savings.
Collapse
Affiliation(s)
- Thomas Albrecht
- Institute of Pathology, Heidelberg University Hospital, Heidelberg, Germany; Liver Cancer Center Heidelberg, Heidelberg, Germany.
| | - Annik Rossberg
- Institute of Pathology, Heidelberg University Hospital, Heidelberg, Germany
| | | | - Jan Peter Nicolay
- Department of Dermatology, University Medical Centre Mannheim, Mannheim, Germany
| | - Beate Katharina Straub
- Institute of Pathology, University Medicine, Johannes Gutenberg University, Mainz, Germany
| | - Tiemo Sven Gerber
- Institute of Pathology, University Medicine, Johannes Gutenberg University, Mainz, Germany
| | - Michael Albrecht
- European Center for Angioscience, Medical Faculty of Mannheim, Mannheim, Germany
| | - Fritz Brinkmann
- Institute of Pathology, Heidelberg University Hospital, Heidelberg, Germany
| | - Alphonse Charbel
- Institute of Pathology, Heidelberg University Hospital, Heidelberg, Germany
| | - Constantin Schwab
- Institute of Pathology, Heidelberg University Hospital, Heidelberg, Germany
| | - Johannes Schreck
- Institute of Pathology, Heidelberg University Hospital, Heidelberg, Germany
| | - Alexander Brobeil
- Institute of Pathology, Heidelberg University Hospital, Heidelberg, Germany
| | | | | | - Bruno Christian Köhler
- Liver Cancer Center Heidelberg, Heidelberg, Germany; Department of Medical Oncology, National Center for Tumor Diseases, Heidelberg University Hospital, Heidelberg, Germany
| | - Christoph Springfeld
- Liver Cancer Center Heidelberg, Heidelberg, Germany; Department of Medical Oncology, National Center for Tumor Diseases, Heidelberg University Hospital, Heidelberg, Germany
| | - Arianeb Mehrabi
- Liver Cancer Center Heidelberg, Heidelberg, Germany; Department of General, Visceral and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - Stephan Singer
- Institute of Pathology and Neuropathology, Eberhard-Karls University, Tübingen, Germany
| | - Monika Nadja Vogel
- Diagnostic and Interventional Radiology, Thoraxklinik at Heidelberg University Hospital, Heidelberg, Germany
| | - Olaf Neumann
- Institute of Pathology, Heidelberg University Hospital, Heidelberg, Germany
| | | | - Peter Schirmacher
- Institute of Pathology, Heidelberg University Hospital, Heidelberg, Germany; Liver Cancer Center Heidelberg, Heidelberg, Germany
| | - Cleo-Aron Weis
- Institute of Pathology, Heidelberg University Hospital, Heidelberg, Germany
| | - Stephanie Roessler
- Institute of Pathology, Heidelberg University Hospital, Heidelberg, Germany; Liver Cancer Center Heidelberg, Heidelberg, Germany
| | - Jakob Nikolas Kather
- Department of Medical Oncology, National Center for Tumor Diseases, Heidelberg University Hospital, Heidelberg, Germany; Else Kroener Fresenius Center for Digital Health, Medical Faculty Carl Gustav Carus, Technical University Dresden, Dresden, Germany
| | - Benjamin Goeppert
- Institute of Pathology, Heidelberg University Hospital, Heidelberg, Germany; Institute of Pathology and Neuropathology, RKH Hospital Ludwigsburg, Ludwigsburg, Germany; Institute of Tissue Medicine and Pathology, University of Bern, Bern, Switzerland
| |
Collapse
|
16
|
Feng S, Wang J, Wang L, Qiu Q, Chen D, Su H, Li X, Xiao Y, Lin C. Current Status and Analysis of Machine Learning in Hepatocellular Carcinoma. J Clin Transl Hepatol 2023; 11:1184-1191. [PMID: 37577233 PMCID: PMC10412715 DOI: 10.14218/jcth.2022.00077s] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/26/2022] [Revised: 12/11/2022] [Accepted: 02/21/2023] [Indexed: 07/03/2023] Open
Abstract
Hepatocellular carcinoma (HCC) is a common tumor. Although the diagnosis and treatment of HCC have made great progress, the overall prognosis remains poor. As the core component of artificial intelligence, machine learning (ML) has developed rapidly in the past decade. In particular, ML has become widely used in the medical field, and it has helped in the diagnosis and treatment of cancer. Different algorithms of ML have different roles in diagnosis, treatment, and prognosis. This article reviews recent research, explains the application of different ML models in HCC, and provides suggestions for follow-up research.
Collapse
Affiliation(s)
- Sijia Feng
- General Surgery, Central South University Xiangya Hospital, Changsha, Hunan, China
| | - Jianhua Wang
- General Surgery, Central South University Xiangya Hospital, Changsha, Hunan, China
| | - Liheng Wang
- General Surgery, Central South University Xiangya Hospital, Changsha, Hunan, China
| | - Qixuan Qiu
- General Surgery, Central South University Xiangya Hospital, Changsha, Hunan, China
| | - Dongdong Chen
- General Surgery, Central South University Xiangya Hospital, Changsha, Hunan, China
| | - Huo Su
- General Surgery, Central South University Xiangya Hospital, Changsha, Hunan, China
| | - Xiaoli Li
- General Surgery, Central South University Xiangya Hospital, Changsha, Hunan, China
| | - Yao Xiao
- General Surgery, Central South University Xiangya Hospital, Changsha, Hunan, China
| | - Chiayen Lin
- General Surgery, Central South University Xiangya Hospital, Changsha, Hunan, China
| |
Collapse
|
17
|
Oudin A, Maatoug R, Bourla A, Ferreri F, Bonnot O, Millet B, Schoeller F, Mouchabac S, Adrien V. Digital Phenotyping: Data-Driven Psychiatry to Redefine Mental Health. J Med Internet Res 2023; 25:e44502. [PMID: 37792430 PMCID: PMC10585447 DOI: 10.2196/44502] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2022] [Revised: 07/10/2023] [Accepted: 08/21/2023] [Indexed: 10/05/2023] Open
Abstract
The term "digital phenotype" refers to the digital footprint left by patient-environment interactions. It has potential for both research and clinical applications but challenges our conception of health care by opposing 2 distinct approaches to medicine: one centered on illness with the aim of classifying and curing disease, and the other centered on patients, their personal distress, and their lived experiences. In the context of mental health and psychiatry, the potential benefits of digital phenotyping include creating new avenues for treatment and enabling patients to take control of their own well-being. However, this comes at the cost of sacrificing the fundamental human element of psychotherapy, which is crucial to addressing patients' distress. In this viewpoint paper, we discuss the advances rendered possible by digital phenotyping and highlight the risk that this technology may pose by partially excluding health care professionals from the diagnosis and therapeutic process, thereby foregoing an essential dimension of care. We conclude by setting out concrete recommendations on how to improve current digital phenotyping technology so that it can be harnessed to redefine mental health by empowering patients without alienating them.
Collapse
Affiliation(s)
- Antoine Oudin
- Infrastructure for Clinical Research in Neurosciences, Paris Brain Institute, Sorbonne University- Institut national de la santé et de la recherche médicale - Centre national de la recherche scientifique, Paris, France
- Department of Psychiatry, Pitié-Salpêtrière Hospital, Public Hospitals of Sorbonne University, Paris, France
| | - Redwan Maatoug
- Infrastructure for Clinical Research in Neurosciences, Paris Brain Institute, Sorbonne University- Institut national de la santé et de la recherche médicale - Centre national de la recherche scientifique, Paris, France
- Department of Psychiatry, Pitié-Salpêtrière Hospital, Public Hospitals of Sorbonne University, Paris, France
| | - Alexis Bourla
- Infrastructure for Clinical Research in Neurosciences, Paris Brain Institute, Sorbonne University- Institut national de la santé et de la recherche médicale - Centre national de la recherche scientifique, Paris, France
- Department of Psychiatry, Saint-Antoine Hospital, Public Hospitals of Sorbonne University, Paris, France
- Medical Strategy and Innovation Department, Clariane, Paris, France
- NeuroStim Psychiatry Practice, Paris, France
| | - Florian Ferreri
- Infrastructure for Clinical Research in Neurosciences, Paris Brain Institute, Sorbonne University- Institut national de la santé et de la recherche médicale - Centre national de la recherche scientifique, Paris, France
- Department of Psychiatry, Saint-Antoine Hospital, Public Hospitals of Sorbonne University, Paris, France
| | - Olivier Bonnot
- Department of Child and Adolescent Psychiatry, Nantes University Hospital, Nantes, France
- Pays de la Loire Psychology Laboratory, Nantes University, Nantes, France
| | - Bruno Millet
- Infrastructure for Clinical Research in Neurosciences, Paris Brain Institute, Sorbonne University- Institut national de la santé et de la recherche médicale - Centre national de la recherche scientifique, Paris, France
- Department of Psychiatry, Pitié-Salpêtrière Hospital, Public Hospitals of Sorbonne University, Paris, France
| | - Félix Schoeller
- Institute for Advanced Consciousness Studies, Santa Monica, CA, United States
- Media Lab, Massachusetts Institute of Technology, Cambridge, MA, United States
| | - Stéphane Mouchabac
- Infrastructure for Clinical Research in Neurosciences, Paris Brain Institute, Sorbonne University- Institut national de la santé et de la recherche médicale - Centre national de la recherche scientifique, Paris, France
- Department of Psychiatry, Saint-Antoine Hospital, Public Hospitals of Sorbonne University, Paris, France
| | - Vladimir Adrien
- Infrastructure for Clinical Research in Neurosciences, Paris Brain Institute, Sorbonne University- Institut national de la santé et de la recherche médicale - Centre national de la recherche scientifique, Paris, France
- Department of Psychiatry, Saint-Antoine Hospital, Public Hospitals of Sorbonne University, Paris, France
| |
Collapse
|
18
|
Gichoya JW, Thomas K, Celi LA, Safdar N, Banerjee I, Banja JD, Seyyed-Kalantari L, Trivedi H, Purkayastha S. AI pitfalls and what not to do: mitigating bias in AI. Br J Radiol 2023; 96:20230023. [PMID: 37698583 PMCID: PMC10546443 DOI: 10.1259/bjr.20230023] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/08/2023] [Revised: 08/10/2023] [Accepted: 08/14/2023] [Indexed: 09/13/2023] Open
Abstract
Various forms of artificial intelligence (AI) applications are being deployed and used in many healthcare systems. As the use of these applications increases, we are learning the failures of these models and how they can perpetuate bias. With these new lessons, we need to prioritize bias evaluation and mitigation for radiology applications; all the while not ignoring the impact of changes in the larger enterprise AI deployment which may have downstream impact on performance of AI models. In this paper, we provide an updated review of known pitfalls causing AI bias and discuss strategies for mitigating these biases within the context of AI deployment in the larger healthcare enterprise. We describe these pitfalls by framing them in the larger AI lifecycle from problem definition, data set selection and curation, model training and deployment emphasizing that bias exists across a spectrum and is a sequela of a combination of both human and machine factors.
Collapse
Affiliation(s)
| | - Kaesha Thomas
- Department of Radiology, Emory University, Atlanta, United States
| | | | - Nabile Safdar
- Department of Radiology, Emory University, Atlanta, United States
| | - Imon Banerjee
- School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, Tempe, United States
| | - John D Banja
- Emory University Center for Ethics, Emory University, Atlanta, United States
| | - Laleh Seyyed-Kalantari
- Department of Electrical Engineering and Computer Science, Lassonde School of Engineering, York University, North York, United States
| | - Hari Trivedi
- Department of Radiology, Emory University, Atlanta, United States
| | - Saptarshi Purkayastha
- School of Informatics and Computing, Indiana University Purdue University, Indianapolis, United States
| |
Collapse
|
19
|
Wang DY, Ding J, Sun AL, Liu SG, Jiang D, Li N, Yu JK. Artificial intelligence suppression as a strategy to mitigate artificial intelligence automation bias. J Am Med Inform Assoc 2023; 30:1684-1692. [PMID: 37561535 PMCID: PMC10531198 DOI: 10.1093/jamia/ocad118] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2022] [Revised: 05/30/2023] [Accepted: 06/19/2023] [Indexed: 08/11/2023] Open
Abstract
BACKGROUND Incorporating artificial intelligence (AI) into clinics brings the risk of automation bias, which potentially misleads the clinician's decision-making. The purpose of this study was to propose a potential strategy to mitigate automation bias. METHODS This was a laboratory study with a randomized cross-over design. The diagnosis of anterior cruciate ligament (ACL) rupture, a common injury, on magnetic resonance imaging (MRI) was used as an example. Forty clinicians were invited to diagnose 200 ACLs with and without AI assistance. The AI's correcting and misleading (automation bias) effects on the clinicians' decision-making processes were analyzed. An ordinal logistic regression model was employed to predict the correcting and misleading probabilities of the AI. We further proposed an AI suppression strategy that retracted AI diagnoses with a higher misleading probability and provided AI diagnoses with a higher correcting probability. RESULTS The AI significantly increased clinicians' accuracy from 87.2%±13.1% to 96.4%±1.9% (P < .001). However, the clinicians' errors in the AI-assisted round were associated with automation bias, accounting for 45.5% of the total mistakes. The automation bias was found to affect clinicians of all levels of expertise. Using a logistic regression model, we identified an AI output zone with higher probability to generate misleading diagnoses. The proposed AI suppression strategy was estimated to decrease clinicians' automation bias by 41.7%. CONCLUSION Although AI improved clinicians' diagnostic performance, automation bias was a serious problem that should be addressed in clinical practice. The proposed AI suppression strategy is a practical method for decreasing automation bias.
Collapse
Affiliation(s)
- Ding-Yu Wang
- Department of Sports Medicine, Peking University Third Hospital, Institute of Sports Medicine of Peking University, Beijing, China
- Beijing Key Laboratory of Sports Injuries, Beijing, China
- Engineering Research Center of Sports Trauma Treatment Technology and Devices, Ministry of Education, Beijing, China
| | - Jia Ding
- Beijing Yizhun Medical AI Co., Ltd, Beijing, China
| | - An-Lan Sun
- Beijing Yizhun Medical AI Co., Ltd, Beijing, China
| | - Shang-Gui Liu
- Department of Sports Medicine, Peking University Third Hospital, Institute of Sports Medicine of Peking University, Beijing, China
- Beijing Key Laboratory of Sports Injuries, Beijing, China
- Engineering Research Center of Sports Trauma Treatment Technology and Devices, Ministry of Education, Beijing, China
| | - Dong Jiang
- Department of Sports Medicine, Peking University Third Hospital, Institute of Sports Medicine of Peking University, Beijing, China
- Beijing Key Laboratory of Sports Injuries, Beijing, China
- Engineering Research Center of Sports Trauma Treatment Technology and Devices, Ministry of Education, Beijing, China
| | - Nan Li
- Research Center of Clinical Epidemiology, Peking University Third Hospital, Beijing, China
| | - Jia-Kuo Yu
- Department of Sports Medicine, Peking University Third Hospital, Institute of Sports Medicine of Peking University, Beijing, China
- Beijing Key Laboratory of Sports Injuries, Beijing, China
- Engineering Research Center of Sports Trauma Treatment Technology and Devices, Ministry of Education, Beijing, China
| |
Collapse
|
20
|
Solomon BD, Chung WK. Artificial intelligence and the impact on medical genetics. AMERICAN JOURNAL OF MEDICAL GENETICS. PART C, SEMINARS IN MEDICAL GENETICS 2023; 193:e32060. [PMID: 37565625 DOI: 10.1002/ajmg.c.32060] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/02/2023] [Revised: 07/24/2023] [Accepted: 07/29/2023] [Indexed: 08/12/2023]
Abstract
Virtually all areas of biomedicine will be increasingly affected by applications of artificial intelligence (AI). We discuss how AI may affect fields of medical genetics, including both clinicians and laboratorians. In addition to reviewing the anticipated impact, we provide recommendations for ways in which these groups may want to evolve in light of the influence of AI. We also briefly discuss how educational and training programs can play a key role in preparing the future workforce given these anticipated changes.
Collapse
Affiliation(s)
- Benjamin D Solomon
- Medical Genetics Branch, National Human Genome Research Institute, Bethesda, Maryland, USA
| | - Wendy K Chung
- Department of Pediatrics, Boston Children's Hospital, Harvard Medical School, Boston, Massachusetts, USA
| |
Collapse
|
21
|
Köteles MM, Vigdorovits A, Kumar D, Mihai IM, Jurescu A, Gheju A, Bucur A, Harich OO, Olteanu GE. Comparative Evaluation of Breast Ductal Carcinoma Grading: A Deep-Learning Model and General Pathologists' Assessment Approach. Diagnostics (Basel) 2023; 13:2326. [PMID: 37510069 PMCID: PMC10377791 DOI: 10.3390/diagnostics13142326] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2023] [Revised: 07/05/2023] [Accepted: 07/07/2023] [Indexed: 07/30/2023] Open
Abstract
Breast cancer is the most prevalent neoplasia among women, with early and accurate diagnosis critical for effective treatment. In clinical practice, however, the subjective nature of histological grading of infiltrating ductal adenocarcinoma of the breast (DAC-NOS) often leads to inconsistencies among pathologists, posing a significant challenge to achieving optimal patient outcomes. Our study aimed to address this reproducibility problem by leveraging artificial intelligence (AI). We trained a deep-learning model using a convolutional neural network-based algorithm (CNN-bA) on 100 whole slide images (WSIs) of DAC-NOS from the Cancer Genome Atlas Breast Invasive Carcinoma (TCGA-BRCA) dataset. Our model demonstrated high precision, sensitivity, and F1 score across different grading components in about 17.5 h with 19,000 iterations. However, the agreement between the model's grading and that of general pathologists varied, showing the highest agreement for the mitotic count score. These findings suggest that AI has the potential to enhance the accuracy and reproducibility of breast cancer grading, warranting further refinement and validation of this approach.
Collapse
Affiliation(s)
| | - Alon Vigdorovits
- Bihor County Clinical Emergency Hospital, Gh. Doja Street No. 65, 410169 Oradea, Romania
- Center for Research and Innovation in Personalized Medicine of Respiratory Diseases, "Victor Babes" University of Medicine and Pharmacy, Timisoara Eftimie Murgu Sq. No. 2, 300041 Timisoara, Romania
- Victor Babes Institute of Pathology-Next Generation Pathology Research Group, Splaiul Independenţei 99-101, 050096 Bucharest, Romania
| | | | - Ioana-Maria Mihai
- Department of Microscopic Morphology-Morphopatology, ANAPATMOL Research Center, "Victor Babes" University of Medicine and Pharmacy, 300041 Timisoara, Romania
| | - Aura Jurescu
- Department of Microscopic Morphology-Morphopatology, ANAPATMOL Research Center, "Victor Babes" University of Medicine and Pharmacy, 300041 Timisoara, Romania
| | - Adelina Gheju
- Emergency County Hospital Deva, Bulevardul 22 Decembrie 58, 330032 Deva, Romania
| | - Adeline Bucur
- Department of Microscopic Morphology, Discipline of Histology, "Victor Babes" University of Medicine and Pharmacy, Timisoara Eftimie Murgu Sq. No. 2, 300041 Timisoara, Romania
| | - Octavia Oana Harich
- Department of Functional Sciences, "Victor Babes" University of Medicine and Pharmacy, Timisoara Eftimie Murgu Sq. No. 2, 300041 Timisoara, Romania
| | - Gheorghe-Emilian Olteanu
- Center for Research and Innovation in Personalized Medicine of Respiratory Diseases, "Victor Babes" University of Medicine and Pharmacy, Timisoara Eftimie Murgu Sq. No. 2, 300041 Timisoara, Romania
- Faculty of Pharmacy, "Victor Babes" University of Medicine and Pharmacy, Eftimie Murgu Square No. 2, 300041 Timisoara, Romania
- Research Center for Pharmaco-Toxicological Evaluations, Faculty of Pharmacy, "Victor Babes" University of Medicine and Pharmacy, Eftimie Murgu Square No. 2, 300041 Timisoara, Romania
- Center of Expertise for Rare Lung Diseases, Clinical Hospital of Infectious Diseases and Pneumophthisiology "Dr. Victor Babes" Timisoara, Gh. Adam Street No. 13, 300310 Timisoara, Romania
| |
Collapse
|
22
|
Kriegsmann M, Kriegsmann K, Steinbuss G, Zgorzelski C, Albrecht T, Heinrich S, Farkas S, Roth W, Dang H, Hausen A, Gaida MM. Implementation of deep learning in liver pathology optimizes diagnosis of benign lesions and adenocarcinoma metastasis. Clin Transl Med 2023; 13:e1299. [PMID: 37415390 PMCID: PMC10326372 DOI: 10.1002/ctm2.1299] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/22/2023] [Accepted: 05/28/2023] [Indexed: 07/08/2023] Open
Abstract
INTRODUCTION Differentiation of histologically similar structures in the liver, including anatomical structures, benign bile duct lesions, or common types of liver metastases, can be challenging with conventional histological tissue sections alone. Accurate histopathological classification is paramount for the diagnosis and adequate treatment of the disease. Deep learning algorithms have been proposed for objective and consistent assessment of digital histopathological images. MATERIALS AND METHODS In the present study, we trained and evaluated deep learning algorithms based on the EfficientNetV2 and ResNetRS architectures to discriminate between different histopathological classes. For the required dataset, specialized surgical pathologists annotated seven different histological classes, including different non-neoplastic anatomical structures, benign bile duct lesions, and liver metastases from colorectal and pancreatic adenocarcinoma in a large patient cohort. Annotation resulted in a total of 204.159 image patches, followed by discrimination analysis using our deep learning models. Model performance was evaluated on validation and test data using confusion matrices. RESULTS Evaluation of the test set based on tiles and cases revealed overall highly satisfactory prediction capability of our algorithm for the different histological classes, resulting in a tile accuracy of 89% (38 413/43 059) and case accuracy of 94% (198/211). Importantly, the separation of metastasis versus benign lesions was certainly confident on case level, confirming the classification model performed with high diagnostic accuracy. Moreover, the whole curated raw data set is made publically available. CONCLUSIONS Deep learning is a promising approach in surgical liver pathology supporting decision making in personalized medicine.
Collapse
Affiliation(s)
- Mark Kriegsmann
- Institute of PathologyHeidelberg UniversityHeidelbergGermany
- Pathology WiesbadenWiesbadenGermany
| | - Katharina Kriegsmann
- Department of HematologyOncology and RheumatologyHeidelberg UniversityHeidelbergGermany
- Laborarztpraxis Rhein‐Main MVZ GbRFrankfurt am MainFrankfurtGermany
| | - Georg Steinbuss
- Department of HematologyOncology and RheumatologyHeidelberg UniversityHeidelbergGermany
| | | | - Thomas Albrecht
- Institute of PathologyHeidelberg UniversityHeidelbergGermany
| | - Stefan Heinrich
- Department of SurgeryJGU‐MainzUniversity Medical Center MainzMainzGermany
| | - Stefan Farkas
- Department of SurgerySt. Josefs‐ HospitalWiesbadenGermany
| | - Wilfried Roth
- Institute of PathologyJGU‐MainzUniversity Medical Center MainzMainzGermany
| | - Hien Dang
- Department of SurgeryDepartment of Surgical ResearchThomas Jefferson UniversityPhiladelphiaPennsylvaniaUSA
| | - Anne Hausen
- Institute of PathologyJGU‐MainzUniversity Medical Center MainzMainzGermany
| | - Matthias M. Gaida
- Institute of PathologyJGU‐MainzUniversity Medical Center MainzMainzGermany
- TRONJGU‐MainzTranslational Oncology at the University Medical CenterMainzGermany
- Research Center for ImmunotherapyJGU‐MainzUniversity Medical Center MainzMainzGermany
| |
Collapse
|
23
|
Nuutinen M, Leskelä RL. Systematic review of the performance evaluation of clinicians with or without the aid of machine learning clinical decision support system. HEALTH AND TECHNOLOGY 2023; 13:1-14. [PMID: 37363342 PMCID: PMC10262137 DOI: 10.1007/s12553-023-00763-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2023] [Accepted: 06/01/2023] [Indexed: 06/28/2023]
Abstract
Background For the adoption of machine learning clinical decision support systems (ML-CDSS) it is critical to understand the performance aid of the ML-CDSS. However, it is not trivial, how the performance aid should be evaluated. To design reliable performance evaluation study, both the knowledge from the practical framework of experimental study design and the understanding of domain specific design factors are required. Objective The aim of this review study was to form a practical framework and identify key design factors for experimental design in evaluating the performance of clinicians with or without the aid of ML-CDSS. Methods The study was based on published ML-CDSS performance evaluation studies. We systematically searched articles published between January 2016 and December 2022. From the articles we collected a set of design factors. Only the articles comparing the performance of clinicians with or without the aid of ML-CDSS using experimental study methods were considered. Results The identified key design factors for the practical framework of ML-CDSS experimental study design were performance measures, user interface, ground truth data and the selection of samples and participants. In addition, we identified the importance of randomization, crossover design and training and practice rounds. Previous studies had shortcomings in the rationale and documentation of choices regarding the number of participants and the duration of the experiment. Conclusion The design factors of ML-CDSS experimental study are interdependent and all factors must be considered in individual choices. Supplementary Information The online version contains supplementary material available at 10.1007/s12553-023-00763-1.
Collapse
Affiliation(s)
- Mikko Nuutinen
- Nordic Healthcare Group, Helsinki, Finland
- Haartman Institute, University of Helsinki, Helsinki, Finland
| | | |
Collapse
|
24
|
Lee JH, Hong H, Nam G, Hwang EJ, Park CM. Effect of Human-AI Interaction on Detection of Malignant Lung Nodules on Chest Radiographs. Radiology 2023; 307:e222976. [PMID: 37367443 DOI: 10.1148/radiol.222976] [Citation(s) in RCA: 14] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/28/2023]
Abstract
Background The factors affecting radiologists' diagnostic determinations in artificial intelligence (AI)-assisted image reading remain underexplored. Purpose To assess how AI diagnostic performance and reader characteristics influence detection of malignant lung nodules during AI-assisted reading of chest radiographs. Materials and Methods This retrospective study consisted of two reading sessions from April 2021 to June 2021. Based on the first session without AI assistance, 30 readers were assigned into two groups with equivalent areas under the free-response receiver operating characteristic curve (AUFROCs). In the second session, each group reinterpreted radiographs assisted by either a high or low accuracy AI model (blinded to the fact that two different AI models were used). Reader performance for detecting lung cancer and reader susceptibility (changing the original reading following the AI suggestion) were compared. A generalized linear mixed model was used to identify the factors influencing AI-assisted detection performance, including readers' attitudes and experiences of AI and Grit score. Results Of the 120 chest radiographs assessed, 60 were obtained in patients with lung cancer (mean age, 67 years ± 12 [SD]; 32 male; 63 cancers) and 60 in controls (mean age, 67 years ± 12; 36 male). Readers included 20 thoracic radiologists (5-18 years of experience) and 10 radiology residents (2-3 years of experience). Use of the high accuracy AI model improved readers' detection performance to a greater extent than use of the low accuracy AI model (area under the receiver operating characteristic curve, 0.77 to 0.82 vs 0.75 to 0.75; AUFROC, 0.71 to 0.79 vs 0.7 to 0.72). Readers who used the high accuracy AI showed a higher susceptibility (67%, 224 of 334 cases) to changing their diagnosis based on the AI suggestions than those using the low accuracy AI (59%, 229 of 386 cases). Accurate readings at the first session, correct AI suggestions, high accuracy Al, and diagnostic difficulty were associated with accurate AI-assisted readings, but readers' characteristics were not. Conclusion An AI model with high diagnostic accuracy led to improved performance of radiologists in detecting lung cancer on chest radiographs and increased radiologists' susceptibility to AI suggestions. © RSNA, 2023 Supplemental material is available for this article.
Collapse
Affiliation(s)
- Jong Hyuk Lee
- From the Department of Radiology (J.H.L., E.J.H., C.M.P.) and Medical Research Collaborating Center (H.H.), Seoul National University Hospital, Seoul, Korea; Lunit, Seoul, Korea (G.N.); Institute of Medical and Biological Engineering and Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea (C.M.P.); and Department of Radiology, Seoul National University College of Medicine, 101 Daehak-ro, Jongno-gu, Seoul 03080, Korea (C.M.P.)
| | - Hyunsook Hong
- From the Department of Radiology (J.H.L., E.J.H., C.M.P.) and Medical Research Collaborating Center (H.H.), Seoul National University Hospital, Seoul, Korea; Lunit, Seoul, Korea (G.N.); Institute of Medical and Biological Engineering and Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea (C.M.P.); and Department of Radiology, Seoul National University College of Medicine, 101 Daehak-ro, Jongno-gu, Seoul 03080, Korea (C.M.P.)
| | - Gunhee Nam
- From the Department of Radiology (J.H.L., E.J.H., C.M.P.) and Medical Research Collaborating Center (H.H.), Seoul National University Hospital, Seoul, Korea; Lunit, Seoul, Korea (G.N.); Institute of Medical and Biological Engineering and Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea (C.M.P.); and Department of Radiology, Seoul National University College of Medicine, 101 Daehak-ro, Jongno-gu, Seoul 03080, Korea (C.M.P.)
| | - Eui Jin Hwang
- From the Department of Radiology (J.H.L., E.J.H., C.M.P.) and Medical Research Collaborating Center (H.H.), Seoul National University Hospital, Seoul, Korea; Lunit, Seoul, Korea (G.N.); Institute of Medical and Biological Engineering and Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea (C.M.P.); and Department of Radiology, Seoul National University College of Medicine, 101 Daehak-ro, Jongno-gu, Seoul 03080, Korea (C.M.P.)
| | - Chang Min Park
- From the Department of Radiology (J.H.L., E.J.H., C.M.P.) and Medical Research Collaborating Center (H.H.), Seoul National University Hospital, Seoul, Korea; Lunit, Seoul, Korea (G.N.); Institute of Medical and Biological Engineering and Institute of Radiation Medicine, Seoul National University Medical Research Center, Seoul, Korea (C.M.P.); and Department of Radiology, Seoul National University College of Medicine, 101 Daehak-ro, Jongno-gu, Seoul 03080, Korea (C.M.P.)
| |
Collapse
|
25
|
Kim M, Sekiya H, Yao G, Martin NB, Castanedes-Casey M, Dickson DW, Hwang TH, Koga S. Diagnosis of Alzheimer Disease and Tauopathies on Whole-Slide Histopathology Images Using a Weakly Supervised Deep Learning Algorithm. J Transl Med 2023; 103:100127. [PMID: 36889541 DOI: 10.1016/j.labinv.2023.100127] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2022] [Revised: 02/06/2023] [Accepted: 02/17/2023] [Indexed: 03/08/2023] Open
Abstract
Neuropathologic assessment during autopsy is the gold standard for diagnosing neurodegenerative disorders. Neurodegenerative conditions, such as Alzheimer disease (AD) neuropathological change, are a continuous process from normal aging rather than categorical; therefore, diagnosing neurodegenerative disorders is a complicated task. We aimed to develop a pipeline for diagnosing AD and other tauopathies, including corticobasal degeneration (CBD), globular glial tauopathy, Pick disease, and progressive supranuclear palsy. We used a weakly supervised deep learning-based approach called clustering-constrained-attention multiple-instance learning (CLAM) on the whole-slide images (WSIs) of patients with AD (n = 30), CBD (n = 20), globular glial tauopathy (n = 10), Pick disease (n = 20), and progressive supranuclear palsy (n = 20), as well as nontauopathy controls (n = 21). Three sections (A: motor cortex; B: cingulate gyrus and superior frontal gyrus; and C: corpus striatum) that had been immunostained for phosphorylated tau were scanned and converted to WSIs. We evaluated 3 models (classic multiple-instance learning, single-attention-branch CLAM, and multiattention-branch CLAM) using 5-fold cross-validation. Attention-based interpretation analysis was performed to identify the morphologic features contributing to the classification. Within highly attended regions, we also augmented gradient-weighted class activation mapping to the model to visualize cellular-level evidence of the model's decisions. The multiattention-branch CLAM model using section B achieved the highest area under the curve (0.970 ± 0.037) and diagnostic accuracy (0.873 ± 0.087). A heatmap showed the highest attention in the gray matter of the superior frontal gyrus in patients with AD and the white matter of the cingulate gyrus in patients with CBD. Gradient-weighted class activation mapping showed the highest attention in characteristic tau lesions for each disease (eg, numerous tau-positive threads in the white matter inclusions for CBD). Our findings support the feasibility of deep learning-based approaches for the classification of neurodegenerative disorders on WSIs. Further investigation of this method, focusing on clinicopathologic correlations, is warranted.
Collapse
Affiliation(s)
- Minji Kim
- Department of Artificial Intelligence and Informatics Research, Mayo Clinic, Jacksonville, Florida
| | - Hiroaki Sekiya
- Department of Neuroscience, Mayo Clinic, Jacksonville, Florida
| | - Gary Yao
- Department of Artificial Intelligence and Informatics Research, Mayo Clinic, Jacksonville, Florida
| | | | | | | | - Tae Hyun Hwang
- Department of Artificial Intelligence and Informatics Research, Mayo Clinic, Jacksonville, Florida
| | - Shunsuke Koga
- Department of Neuroscience, Mayo Clinic, Jacksonville, Florida.
| |
Collapse
|
26
|
Allaume P, Rabilloud N, Turlin B, Bardou-Jacquet E, Loréal O, Calderaro J, Khene ZE, Acosta O, De Crevoisier R, Rioux-Leclercq N, Pecot T, Kammerer-Jacquet SF. Artificial Intelligence-Based Opportunities in Liver Pathology-A Systematic Review. Diagnostics (Basel) 2023; 13:diagnostics13101799. [PMID: 37238283 DOI: 10.3390/diagnostics13101799] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2023] [Revised: 05/04/2023] [Accepted: 05/12/2023] [Indexed: 05/28/2023] Open
Abstract
BACKGROUND Artificial Intelligence (AI)-based Deep Neural Networks (DNNs) can handle a wide range of applications in image analysis, ranging from automated segmentation to diagnostic and prediction. As such, they have revolutionized healthcare, including in the liver pathology field. OBJECTIVE The present study aims to provide a systematic review of applications and performances provided by DNN algorithms in liver pathology throughout the Pubmed and Embase databases up to December 2022, for tumoral, metabolic and inflammatory fields. RESULTS 42 articles were selected and fully reviewed. Each article was evaluated through the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool, highlighting their risks of bias. CONCLUSIONS DNN-based models are well represented in the field of liver pathology, and their applications are diverse. Most studies, however, presented at least one domain with a high risk of bias according to the QUADAS-2 tool. Hence, DNN models in liver pathology present future opportunities and persistent limitations. To our knowledge, this review is the first one solely focused on DNN-based applications in liver pathology, and to evaluate their bias through the lens of the QUADAS2 tool.
Collapse
Affiliation(s)
- Pierre Allaume
- Department of Pathology CHU de Rennes, Rennes 1 University, Pontchaillou Hospital, 2 rue Henri Le Guilloux, CEDEX 09, 35033 Rennes, France
| | - Noémie Rabilloud
- Impact TEAM, Laboratoire Traitement du Signal et de l'Image (LTSI) INSERM, Rennes 1 University, Pontchaillou Hospital, 35033 Rennes, France
| | - Bruno Turlin
- Department of Pathology CHU de Rennes, Rennes 1 University, Pontchaillou Hospital, 2 rue Henri Le Guilloux, CEDEX 09, 35033 Rennes, France
- Research Unit n°UMR1341 NuMeCan-Nutrition, Métabolismes et Cancer, Rennes 1 University, Pontchaillou Hospital, 2 rue Henri Le Guilloux, CEDEX 09, 35033 Rennes, France
| | - Edouard Bardou-Jacquet
- Research Unit n°UMR1341 NuMeCan-Nutrition, Métabolismes et Cancer, Rennes 1 University, Pontchaillou Hospital, 2 rue Henri Le Guilloux, CEDEX 09, 35033 Rennes, France
- Department of Liver Diseases CHU de Rennes, Rennes 1 University, Pontchaillou Hospital, 35033 Rennes, France
| | - Olivier Loréal
- Research Unit n°UMR1341 NuMeCan-Nutrition, Métabolismes et Cancer, Rennes 1 University, Pontchaillou Hospital, 2 rue Henri Le Guilloux, CEDEX 09, 35033 Rennes, France
| | - Julien Calderaro
- Assistance Publique-Hôpitaux de Paris, Department of Pathology Henri Mondor, 94000 Créteil, France
- INSERM U955, Team Pathophysiology and Therapy of Chronic Viral Hepatitis and Related Cancers, 94000 Créteil, France
| | - Zine-Eddine Khene
- Impact TEAM, Laboratoire Traitement du Signal et de l'Image (LTSI) INSERM, Rennes 1 University, Pontchaillou Hospital, 35033 Rennes, France
- Department of Urology, CHU de Rennes, Rennes 1 University, Pontchaillou Hospital, 2 rue Henri Le Guilloux, CEDEX 09, 35033 Rennes, France
| | - Oscar Acosta
- Impact TEAM, Laboratoire Traitement du Signal et de l'Image (LTSI) INSERM, Rennes 1 University, Pontchaillou Hospital, 35033 Rennes, France
| | - Renaud De Crevoisier
- Impact TEAM, Laboratoire Traitement du Signal et de l'Image (LTSI) INSERM, Rennes 1 University, Pontchaillou Hospital, 35033 Rennes, France
- Department of Radiotherapy, Centre Eugène Marquis, 35033 Rennes, France
| | - Nathalie Rioux-Leclercq
- Department of Pathology CHU de Rennes, Rennes 1 University, Pontchaillou Hospital, 2 rue Henri Le Guilloux, CEDEX 09, 35033 Rennes, France
| | - Thierry Pecot
- Biosit Platform UAR 3480 CNRS US18 INSERM U955, Rennes 1 University, Pontchaillou Hospital, 35033 Rennes, France
| | - Solène-Florence Kammerer-Jacquet
- Department of Pathology CHU de Rennes, Rennes 1 University, Pontchaillou Hospital, 2 rue Henri Le Guilloux, CEDEX 09, 35033 Rennes, France
- Impact TEAM, Laboratoire Traitement du Signal et de l'Image (LTSI) INSERM, Rennes 1 University, Pontchaillou Hospital, 35033 Rennes, France
| |
Collapse
|
27
|
Nazir S, Dickson DM, Akram MU. Survey of explainable artificial intelligence techniques for biomedical imaging with deep neural networks. Comput Biol Med 2023; 156:106668. [PMID: 36863192 DOI: 10.1016/j.compbiomed.2023.106668] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2022] [Revised: 01/12/2023] [Accepted: 02/10/2023] [Indexed: 02/21/2023]
Abstract
Artificial Intelligence (AI) techniques of deep learning have revolutionized the disease diagnosis with their outstanding image classification performance. In spite of the outstanding results, the widespread adoption of these techniques in clinical practice is still taking place at a moderate pace. One of the major hindrance is that a trained Deep Neural Networks (DNN) model provides a prediction, but questions about why and how that prediction was made remain unanswered. This linkage is of utmost importance for the regulated healthcare domain to increase the trust in the automated diagnosis system by the practitioners, patients and other stakeholders. The application of deep learning for medical imaging has to be interpreted with caution due to the health and safety concerns similar to blame attribution in the case of an accident involving autonomous cars. The consequences of both a false positive and false negative cases are far reaching for patients' welfare and cannot be ignored. This is exacerbated by the fact that the state-of-the-art deep learning algorithms comprise of complex interconnected structures, millions of parameters, and a 'black box' nature, offering little understanding of their inner working unlike the traditional machine learning algorithms. Explainable AI (XAI) techniques help to understand model predictions which help develop trust in the system, accelerate the disease diagnosis, and meet adherence to regulatory requirements. This survey provides a comprehensive review of the promising field of XAI for biomedical imaging diagnostics. We also provide a categorization of the XAI techniques, discuss the open challenges, and provide future directions for XAI which would be of interest to clinicians, regulators and model developers.
Collapse
Affiliation(s)
- Sajid Nazir
- Department of Computing, Glasgow Caledonian University, Glasgow, UK.
| | - Diane M Dickson
- Department of Podiatry and Radiography, Research Centre for Health, Glasgow Caledonian University, Glasgow, UK
| | - Muhammad Usman Akram
- Computer and Software Engineering Department, National University of Sciences and Technology, Islamabad, Pakistan
| |
Collapse
|
28
|
Lundström C, Lindvall M. Mapping the Landscape of Care Providers' Quality Assurance Approaches for AI in Diagnostic Imaging. J Digit Imaging 2023; 36:379-387. [PMID: 36352164 PMCID: PMC10039170 DOI: 10.1007/s10278-022-00731-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2022] [Revised: 10/26/2022] [Accepted: 10/28/2022] [Indexed: 11/10/2022] Open
Abstract
The discussion on artificial intelligence (AI) solutions in diagnostic imaging has matured in recent years. The potential value of AI adoption is well established, as are the potential risks associated. Much focus has, rightfully, been on regulatory certification of AI products, with the strong incentive of being an enabling step for the commercial actors. It is, however, becoming evident that regulatory approval is not enough to ensure safe and effective AI usage in the local setting. In other words, care providers need to develop and implement quality assurance (QA) approaches for AI solutions in diagnostic imaging. The domain of AI-specific QA is still in an early development phase. We contribute to this development by describing the current landscape of QA-for-AI approaches in medical imaging, with focus on radiology and pathology. We map the potential quality threats and review the existing QA approaches in relation to those threats. We propose a practical categorization of QA approaches, based on key characteristics corresponding to means, situation, and purpose. The review highlights the heterogeneity of methods and practices relevant for this domain and points to targets for future research efforts.
Collapse
Affiliation(s)
- Claes Lundström
- Center for Medical Image Science and Visualization, Linköping University, Linköping, Sweden.
- Sectra AB, Linköping, Sweden.
| | | |
Collapse
|
29
|
Shen X, Wu J, Su J, Yao Z, Huang W, Zhang L, Jiang Y, Yu W, Li Z. Revisiting artificial intelligence diagnosis of hepatocellular carcinoma with DIKWH framework. Front Genet 2023; 14:1004481. [PMID: 37007970 PMCID: PMC10064216 DOI: 10.3389/fgene.2023.1004481] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2022] [Accepted: 02/14/2023] [Indexed: 03/09/2023] Open
Abstract
Hepatocellular carcinoma (HCC) is the most common type of liver cancer with a high morbidity and fatality rate. Traditional diagnostic methods for HCC are primarily based on clinical presentation, imaging features, and histopathology. With the rapid development of artificial intelligence (AI), which is increasingly used in the diagnosis, treatment, and prognosis prediction of HCC, an automated approach to HCC status classification is promising. AI integrates labeled clinical data, trains on new data of the same type, and performs interpretation tasks. Several studies have shown that AI techniques can help clinicians and radiologists be more efficient and reduce the misdiagnosis rate. However, the coverage of AI technologies leads to difficulty in which the type of AI technology is preferred to choose for a given problem and situation. Solving this concern, it can significantly reduce the time required to determine the required healthcare approach and provide more precise and personalized solutions for different problems. In our review of research work, we summarize existing research works, compare and classify the main results of these according to the specified data, information, knowledge, wisdom (DIKW) framework.
Collapse
Affiliation(s)
- Xiaomin Shen
- State Key Laboratory for Diagnosis and Treatment of Infectious Diseases, National Clinical Research Center for Infectious Diseases, Collaborative Innovation Center for Diagnosis and Treatment of Infectious Diseases, Zhejiang Provincial Key Laboratory for Drug Clinical Research and Evaluation, The First Affiliated Hospital, Zhejiang University, Hangzhou, China
| | - Jinxin Wu
- School of Computer Science, The University of Sydney, Sydney, NSW, Australia
| | - Junwei Su
- State Key Laboratory for Diagnosis and Treatment of Infectious Diseases, National Clinical Research Center for Infectious Diseases, Collaborative Innovation Center for Diagnosis and Treatment of Infectious Diseases, Zhejiang Provincial Key Laboratory for Drug Clinical Research and Evaluation, The First Affiliated Hospital, Zhejiang University, Hangzhou, China
| | - Zhenyu Yao
- School of Computer Science, King's College London, London, United Kingdom
| | - Wei Huang
- Department of Gastroenterology II, The First Affiliated Hospital, Zhejiang University, Hangzhou, China
| | - Li Zhang
- State Key Laboratory for Diagnosis and Treatment of Infectious Diseases, National Clinical Research Center for Infectious Diseases, Collaborative Innovation Center for Diagnosis and Treatment of Infectious Diseases, Zhejiang Provincial Key Laboratory for Drug Clinical Research and Evaluation, The First Affiliated Hospital, Zhejiang University, Hangzhou, China
| | - Yiheng Jiang
- Clinical Medicine, Nanjing Medical University, Nanjing, China
| | - Wei Yu
- State Key Laboratory for Diagnosis and Treatment of Infectious Diseases, National Clinical Research Center for Infectious Diseases, Collaborative Innovation Center for Diagnosis and Treatment of Infectious Diseases, Zhejiang Provincial Key Laboratory for Drug Clinical Research and Evaluation, The First Affiliated Hospital, Zhejiang University, Hangzhou, China
| | - Zhao Li
- School of Computer Science, Zhejiang University, Hangzhou, China
| |
Collapse
|
30
|
Xue P, Si M, Qin D, Wei B, Seery S, Ye Z, Chen M, Wang S, Song C, Zhang B, Ding M, Zhang W, Bai A, Yan H, Dang L, Zhao Y, Rezhake R, Zhang S, Qiao Y, Qu Y, Jiang Y. Unassisted Clinicians Versus Deep Learning-Assisted Clinicians in Image-Based Cancer Diagnostics: Systematic Review With Meta-analysis. J Med Internet Res 2023; 25:e43832. [PMID: 36862499 PMCID: PMC10020907 DOI: 10.2196/43832] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2022] [Revised: 01/19/2023] [Accepted: 02/13/2023] [Indexed: 02/16/2023] Open
Abstract
BACKGROUND A number of publications have demonstrated that deep learning (DL) algorithms matched or outperformed clinicians in image-based cancer diagnostics, but these algorithms are frequently considered as opponents rather than partners. Despite the clinicians-in-the-loop DL approach having great potential, no study has systematically quantified the diagnostic accuracy of clinicians with and without the assistance of DL in image-based cancer identification. OBJECTIVE We systematically quantified the diagnostic accuracy of clinicians with and without the assistance of DL in image-based cancer identification. METHODS PubMed, Embase, IEEEXplore, and the Cochrane Library were searched for studies published between January 1, 2012, and December 7, 2021. Any type of study design was permitted that focused on comparing unassisted clinicians and DL-assisted clinicians in cancer identification using medical imaging. Studies using medical waveform-data graphics material and those investigating image segmentation rather than classification were excluded. Studies providing binary diagnostic accuracy data and contingency tables were included for further meta-analysis. Two subgroups were defined and analyzed, including cancer type and imaging modality. RESULTS In total, 9796 studies were identified, of which 48 were deemed eligible for systematic review. Twenty-five of these studies made comparisons between unassisted clinicians and DL-assisted clinicians and provided sufficient data for statistical synthesis. We found a pooled sensitivity of 83% (95% CI 80%-86%) for unassisted clinicians and 88% (95% CI 86%-90%) for DL-assisted clinicians. Pooled specificity was 86% (95% CI 83%-88%) for unassisted clinicians and 88% (95% CI 85%-90%) for DL-assisted clinicians. The pooled sensitivity and specificity values for DL-assisted clinicians were higher than for unassisted clinicians, at ratios of 1.07 (95% CI 1.05-1.09) and 1.03 (95% CI 1.02-1.05), respectively. Similar diagnostic performance by DL-assisted clinicians was also observed across the predefined subgroups. CONCLUSIONS The diagnostic performance of DL-assisted clinicians appears better than unassisted clinicians in image-based cancer identification. However, caution should be exercised, because the evidence provided in the reviewed studies does not cover all the minutiae involved in real-world clinical practice. Combining qualitative insights from clinical practice with data-science approaches may improve DL-assisted practice, although further research is required. TRIAL REGISTRATION PROSPERO CRD42021281372; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=281372.
Collapse
Affiliation(s)
- Peng Xue
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Mingyu Si
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Dongxu Qin
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Bingrui Wei
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Samuel Seery
- Faculty of Health and Medicine, Division of Health Research, Lancaster University, Lancaster, United Kingdom
| | - Zichen Ye
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Mingyang Chen
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Sumeng Wang
- Department of Cancer Epidemiology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Cheng Song
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Bo Zhang
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Ming Ding
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Wenling Zhang
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Anying Bai
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Huijiao Yan
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Le Dang
- Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yuqian Zhao
- Sichuan Cancer Hospital & Institute, Sichuan Cancer Center, School of Medicine, University of Electronic Science & Technology of China, Sichuan, China
| | - Remila Rezhake
- Affiliated Cancer Hospital, The 3rd Affiliated Teaching Hospital of Xinjiang Medical University, Xinjiang, China
| | - Shaokai Zhang
- Henan Cancer Hospital, Affiliated Cancer Hospital of Zhengzhou University, Henan, China
| | - Youlin Qiao
- Center for Global Health, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yimin Qu
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yu Jiang
- Department of Epidemiology and Biostatistics, School of Population Medicine and Public Health, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| |
Collapse
|
31
|
Intharah T, Wiratchawa K, Wanna Y, Junsawang P, Titapun A, Techasen A, Boonrod A, Laopaiboon V, Chamadol N, Khuntikeo N. BiTNet: Hybrid deep convolutional model for ultrasound image analysis of human biliary tract and its applications. Artif Intell Med 2023; 139:102539. [PMID: 37100509 DOI: 10.1016/j.artmed.2023.102539] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2022] [Revised: 03/15/2023] [Accepted: 03/27/2023] [Indexed: 04/03/2023]
Abstract
Certain life-threatening abnormalities, such as cholangiocarcinoma, in the human biliary tract are curable if detected at an early stage, and ultrasonography has been proven to be an effective tool for identifying them. However, the diagnosis often requires a second opinion from experienced radiologists, who are usually overwhelmed by many cases. Therefore, we propose a deep convolutional neural network model, named biliary tract network (BiTNet), developed to solve problems in the current screening system and to avoid overconfidence issues of traditional deep convolutional neural networks. Additionally, we present an ultrasound image dataset for the human biliary tract and demonstrate two artificial intelligence (AI) applications: auto-prescreening and assisting tools. The proposed model is the first AI model to automatically screen and diagnose upper-abdominal abnormalities from ultrasound images in real-world healthcare scenarios. Our experiments suggest that prediction probability has an impact on both applications, and our modifications to EfficientNet solve the overconfidence problem, thereby improving the performance of both applications and of healthcare professionals. The proposed BiTNet can reduce the workload of radiologists by 35% while keeping the false negatives to as low as 1 out of every 455 images. Our experiments involving 11 healthcare professionals with four different levels of experience reveal that BiTNet improves the diagnostic performance of participants of all levels. The mean accuracy and precision of the participants with BiTNet as an assisting tool (0.74 and 0.61, respectively) are statistically higher than those of participants without the assisting tool (0.50 and 0.46, respectively (p<0.001)). These experimental results demonstrate the high potential of BiTNet for use in clinical settings.
Collapse
Affiliation(s)
- Thanapong Intharah
- Visual Intelligence Laboratory, Department of Statistics, Faculty of Science, Khon Kaen University, Khon Kaen, Thailand.
| | - Kannika Wiratchawa
- Visual Intelligence Laboratory, Department of Statistics, Faculty of Science, Khon Kaen University, Khon Kaen, Thailand
| | - Yupaporn Wanna
- Visual Intelligence Laboratory, Department of Statistics, Faculty of Science, Khon Kaen University, Khon Kaen, Thailand
| | - Prem Junsawang
- Visual Intelligence Laboratory, Department of Statistics, Faculty of Science, Khon Kaen University, Khon Kaen, Thailand
| | - Attapol Titapun
- Department of Surgery, Faculty of Medicine, Khon Kaen University, Khon Kaen, Thailand; Cholangiocarcinoma Research Institute, Khon Kaen University, Khon Kaen, Thailand
| | - Anchalee Techasen
- Faculty of Associated Medical Sciences, Khon Kaen University, Khon Kaen, Thailand; Cholangiocarcinoma Research Institute, Khon Kaen University, Khon Kaen, Thailand
| | - Arunnit Boonrod
- Department of Radiology, Faculty of Medicine, Khon Kaen University, Khon Kaen, Thailand
| | - Vallop Laopaiboon
- Department of Radiology, Faculty of Medicine, Khon Kaen University, Khon Kaen, Thailand; Cholangiocarcinoma Research Institute, Khon Kaen University, Khon Kaen, Thailand
| | - Nittaya Chamadol
- Department of Radiology, Faculty of Medicine, Khon Kaen University, Khon Kaen, Thailand; Cholangiocarcinoma Research Institute, Khon Kaen University, Khon Kaen, Thailand
| | - Narong Khuntikeo
- Department of Surgery, Faculty of Medicine, Khon Kaen University, Khon Kaen, Thailand; Cholangiocarcinoma Research Institute, Khon Kaen University, Khon Kaen, Thailand.
| |
Collapse
|
32
|
Bakrania A, Joshi N, Zhao X, Zheng G, Bhat M. Artificial intelligence in liver cancers: Decoding the impact of machine learning models in clinical diagnosis of primary liver cancers and liver cancer metastases. Pharmacol Res 2023; 189:106706. [PMID: 36813095 DOI: 10.1016/j.phrs.2023.106706] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/14/2023] [Revised: 02/17/2023] [Accepted: 02/19/2023] [Indexed: 02/22/2023]
Abstract
Liver cancers are the fourth leading cause of cancer-related mortality worldwide. In the past decade, breakthroughs in the field of artificial intelligence (AI) have inspired development of algorithms in the cancer setting. A growing body of recent studies have evaluated machine learning (ML) and deep learning (DL) algorithms for pre-screening, diagnosis and management of liver cancer patients through diagnostic image analysis, biomarker discovery and predicting personalized clinical outcomes. Despite the promise of these early AI tools, there is a significant need to explain the 'black box' of AI and work towards deployment to enable ultimate clinical translatability. Certain emerging fields such as RNA nanomedicine for targeted liver cancer therapy may also benefit from application of AI, specifically in nano-formulation research and development given that they are still largely reliant on lengthy trial-and-error experiments. In this paper, we put forward the current landscape of AI in liver cancers along with the challenges of AI in liver cancer diagnosis and management. Finally, we have discussed the future perspectives of AI application in liver cancer and how a multidisciplinary approach using AI in nanomedicine could accelerate the transition of personalized liver cancer medicine from bench side to the clinic.
Collapse
Affiliation(s)
- Anita Bakrania
- Toronto General Hospital Research Institute, Toronto, ON, Canada; Ajmera Transplant Program, University Health Network, Toronto, ON, Canada; Princess Margaret Cancer Centre, University Health Network, Toronto, ON, Canada.
| | | | - Xun Zhao
- Toronto General Hospital Research Institute, Toronto, ON, Canada; Ajmera Transplant Program, University Health Network, Toronto, ON, Canada
| | - Gang Zheng
- Princess Margaret Cancer Centre, University Health Network, Toronto, ON, Canada; Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada; Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada
| | - Mamatha Bhat
- Toronto General Hospital Research Institute, Toronto, ON, Canada; Ajmera Transplant Program, University Health Network, Toronto, ON, Canada; Division of Gastroenterology, Department of Medicine, University Health Network and University of Toronto, Toronto, ON, Canada; Department of Medical Sciences, Toronto, ON, Canada.
| |
Collapse
|
33
|
Zhang X, Gleber‐Netto FO, Wang S, Martins‐Chaves RR, Gomez RS, Vigneswaran N, Sarkar A, William WN, Papadimitrakopoulou V, Williams M, Bell D, Palsgrove D, Bishop J, Heymach JV, Gillenwater AM, Myers JN, Ferrarotto R, Lippman SM, Pickering CR, Xiao G. Deep learning-based pathology image analysis predicts cancer progression risk in patients with oral leukoplakia. Cancer Med 2023; 12:7508-7518. [PMID: 36721313 PMCID: PMC10067069 DOI: 10.1002/cam4.5478] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2022] [Revised: 10/15/2022] [Accepted: 11/14/2022] [Indexed: 02/02/2023] Open
Abstract
BACKGROUND Oral leukoplakia (OL) is associated with an increased risk for oral cancer (OC) development. Prediction of OL cancer progression may contribute to decreased OC morbidity and mortality by favoring early intervention. Current OL progression risk assessment approaches face large interobserver variability and is weakly prognostic. We hypothesized that convolutional neural networks (CNN)-based histology image analyses could accelerate the discovery of better OC progression risk models. METHODS Our CNN-based oral mucosa risk stratification model (OMRS) was trained to classify a set of nondysplastic oral mucosa (OM) and a set of OC H&E slides. As a result, the OMRS model could identify abnormal morphological features of the oral epithelium. By applying this model to OL slides, we hypothesized that the extent of OC-like features identified in the OL epithelium would correlate with its progression risk. The OMRS model scored and categorized the OL cohort (n = 62) into high- and low-risk groups. RESULTS OL patients classified as high-risk (n = 31) were 3.98 (95% CI 1.36-11.7) times more likely to develop OC than low-risk ones (n = 31). Time-to-progression significantly differed between high- and low-risk groups (p = 0.003). The 5-year OC development probability was 21.3% for low-risk and 52.5% for high-risk patients. The predictive power of the OMRS model was sustained even after adjustment for age, OL site, and OL dysplasia grading (HR = 4.52, 1.5-13.7). CONCLUSION The ORMS model successfully identified OL patients with a high risk of OC development and can potentially benefit OC early diagnosis and prevention policies.
Collapse
Affiliation(s)
- Xinyi Zhang
- Quantitative Biomedical Research Center, Department of Population and Data SciencesUniversity of Texas Southwestern Medical CenterDallasTexasUSA
| | | | - Shidan Wang
- Quantitative Biomedical Research Center, Department of Population and Data SciencesUniversity of Texas Southwestern Medical CenterDallasTexasUSA
| | | | - Ricardo Santiago Gomez
- Department of Oral Surgery and Pathology, School of DentistryUniversidade Federal de Minas GeraisBelo HorizonteBrazil
| | - Nadarajah Vigneswaran
- Department of Diagnostic and Biomedical SciencesThe University of Texas Health Science Center at Houston School of DentistryHoustonTexasUSA
| | - Arunangshu Sarkar
- Department of Head & Neck SurgeryThe University of Texas MD Anderson Cancer CenterHoustonTexasUSA
| | - William N. William
- Department of Thoracic‐Head & Neck Medical OncologyThe University of Texas MD Anderson Cancer CenterHoustonTexasUSA
- Hospital BPA Beneficência Portuguesa de São PauloSao PaoloBrazil
| | - Vassiliki Papadimitrakopoulou
- Department of Thoracic‐Head & Neck Medical OncologyThe University of Texas MD Anderson Cancer CenterHoustonTexasUSA
- Global Product DevelopmentOncology, Pfizer, Inc.New YorkNew YorkUSA
| | - Michelle Williams
- Department of Anatomical PathologyThe University of Texas MD Anderson Cancer CenterHoustonTexasUSA
| | - Diana Bell
- Department of Anatomical PathologyThe University of Texas MD Anderson Cancer CenterHoustonTexasUSA
- Department of PathologyCity of HopeDuarteCaliforniaUSA
| | - Doreen Palsgrove
- Department of PathologyUniversity of Texas Southwestern Medical CenterDallasTexasUSA
| | - Justin Bishop
- Department of PathologyUniversity of Texas Southwestern Medical CenterDallasTexasUSA
| | - John V. Heymach
- Department of Thoracic‐Head & Neck Medical OncologyThe University of Texas MD Anderson Cancer CenterHoustonTexasUSA
| | - Ann M. Gillenwater
- Department of Head & Neck SurgeryThe University of Texas MD Anderson Cancer CenterHoustonTexasUSA
| | - Jeffrey N. Myers
- Department of Head & Neck SurgeryThe University of Texas MD Anderson Cancer CenterHoustonTexasUSA
| | - Renata Ferrarotto
- Department of Thoracic‐Head & Neck Medical OncologyThe University of Texas MD Anderson Cancer CenterHoustonTexasUSA
| | - Scott M. Lippman
- Department of Thoracic‐Head & Neck Medical OncologyThe University of Texas MD Anderson Cancer CenterHoustonTexasUSA
- Department of MedicineUniversity of California San DiegoSan DiegoCaliforniaUSA
| | - Curtis Rg Pickering
- Department of Head & Neck SurgeryThe University of Texas MD Anderson Cancer CenterHoustonTexasUSA
| | - Guanghua Xiao
- Quantitative Biomedical Research Center, Department of Population and Data SciencesUniversity of Texas Southwestern Medical CenterDallasTexasUSA
- Department of BioinformaticsUniversity of Texas Southwestern Medical CenterDallasTexasUSA
| |
Collapse
|
34
|
Wen Z, Wang S, Yang DM, Xie Y, Chen M, Bishop J, Xiao G. Deep learning in digital pathology for personalized treatment plans of cancer patients. Semin Diagn Pathol 2023; 40:109-119. [PMID: 36890029 DOI: 10.1053/j.semdp.2023.02.003] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/12/2023] [Accepted: 02/22/2023] [Indexed: 02/27/2023]
Abstract
Over the past decade, many new cancer treatments have been developed and made available to patients. However, in most cases, these treatments only benefit a specific subgroup of patients, making the selection of treatment for a specific patient an essential but challenging task for oncologists. Although some biomarkers were found to associate with treatment response, manual assessment is time-consuming and subjective. With the rapid developments and expanded implementation of artificial intelligence (AI) in digital pathology, many biomarkers can be quantified automatically from histopathology images. This approach allows for a more efficient and objective assessment of biomarkers, aiding oncologists in formulating personalized treatment plans for cancer patients. This review presents an overview and summary of the recent studies on biomarker quantification and treatment response prediction using hematoxylin-eosin (H&E) stained pathology images. These studies have shown that an AI-based digital pathology approach can be practical and will become increasingly important in improving the selection of cancer treatments for patients.
Collapse
Affiliation(s)
- Zhuoyu Wen
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Shidan Wang
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Donghan M Yang
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Yang Xie
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX, USA; Simmons Comprehensive Cancer Center, UT Southwestern Medical Center, Dallas, TX, USA; Department of Pathology, University of Texas Southwestern Medical Center, Dallas, TX, USA
| | - Mingyi Chen
- Department of Bioinformatics, UT Southwestern Medical Center, Dallas, TX, USA
| | - Justin Bishop
- Department of Bioinformatics, UT Southwestern Medical Center, Dallas, TX, USA
| | - Guanghua Xiao
- Quantitative Biomedical Research Center, Department of Population and Data Sciences, University of Texas Southwestern Medical Center, Dallas, TX, USA; Simmons Comprehensive Cancer Center, UT Southwestern Medical Center, Dallas, TX, USA; Department of Pathology, University of Texas Southwestern Medical Center, Dallas, TX, USA.
| |
Collapse
|
35
|
Oliveira LC, Lai Z, Harvey D, Nzenkue K, Jin LW, Decarli C, Chuah CN, Dugger BN. Preanalytic variable effects on segmentation and quantification machine learning algorithms for amyloid-β analyses on digitized human brain slides. J Neuropathol Exp Neurol 2023; 82:212-220. [PMID: 36692190 PMCID: PMC9941825 DOI: 10.1093/jnen/nlac132] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/25/2023] Open
Abstract
Computational machine learning (ML)-based frameworks could be advantageous for scalable analyses in neuropathology. A recent deep learning (DL) framework has shown promise in automating the processes of visualizing and quantifying different types of amyloid-β deposits as well as segmenting white matter (WM) from gray matter (GM) on digitized immunohistochemically stained slides. However, this framework has only been trained and evaluated on amyloid-β-stained slides with minimal changes in preanalytic variables. In this study, we evaluated select preanalytical variables including magnification, compression rate, and storage format using three digital slides scanners (Zeiss Axioscan Z1, Leica Aperio AT2, and Leica Aperio GT 450), on over 60 whole slide images, in a cohort of 14 cases having a spectrum of amyloid-β deposits. We conducted statistical comparisons of preanalytic variables with repeated measures analysis of variance evaluating the outputs of two DL frameworks for segmentation and object classification tasks. For both WM/GM segmentation and amyloid-β plaque classification tasks, there were statistical differences with respect to scanner types (p < 0.05) and magnifications (p < 0.05). Although small numbers of cases were analyzed, this pilot study highlights the significance of preanalytic variables that may alter the performance of ML algorithms.
Collapse
Affiliation(s)
- Luca Cerny Oliveira
- Department of Electrical and Computer Engineering, University of California Davis, Davis, California, USA
| | - Zhengfeng Lai
- Department of Electrical and Computer Engineering, University of California Davis, Davis, California, USA
| | - Danielle Harvey
- Department of Public Health Sciences, University of California, Davis, Davis, California, USA
| | - Kevin Nzenkue
- Department of Pathology and Laboratory Medicine, University of California Davis, Sacramento, California, USA
| | - Lee-Way Jin
- Department of Pathology and Laboratory Medicine, University of California Davis, Sacramento, California, USA
| | - Charles Decarli
- Department of Neurology, University of California, Davis, Sacramento, California, USA
| | - Chen-Nee Chuah
- Department of Electrical and Computer Engineering, University of California Davis, Davis, California, USA
| | - Brittany N Dugger
- Department of Pathology and Laboratory Medicine, University of California Davis, Sacramento, California, USA
| |
Collapse
|
36
|
Liu X, Cruz Rivera S, Moher D, Calvert MJ, Denniston AK. [Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extensionDiretrizes para relatórios de ensaios clínicos com intervenções que utilizam inteligência artificial: a extensão CONSORT-AI]. Rev Panam Salud Publica 2023; 48:e13. [PMID: 38352035 PMCID: PMC10863743 DOI: 10.26633/rpsp.2024.13] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2020] [Accepted: 07/23/2020] [Indexed: 02/16/2024] Open
Abstract
The CONSORT 2010 statement provides minimum guidelines for reporting randomized trials. Its widespread use has been instrumental in ensuring transparency in the evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate impact on health outcomes. The CONSORT-AI (Consolidated Standards of Reporting Trials-Artificial Intelligence) extension is a new reporting guideline for clinical trials evaluating interventions with an AI component. It was developed in parallel with its companion statement for clinical trial protocols: SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 29 candidate items, which were assessed by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a two-day consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The CONSORT-AI extension includes 14 new items that were considered sufficiently important for AI interventions that they should be routinely reported in addition to the core CONSORT 2010 items. CONSORT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention is integrated, the handling of inputs and outputs of the AI intervention, the human-AI interaction and provision of an analysis of error cases. CONSORT-AI will help promote transparency and completeness in reporting clinical trials for AI interventions. It will assist editors and peer reviewers, as well as the general readership, to understand, interpret and critically appraise the quality of clinical trial design and risk of bias in the reported outcomes.
Collapse
Affiliation(s)
- Xiaoxuan Liu
- Moorfields Eye Hospital NHS Foundation TrustLondresReino UnidoMoorfields Eye Hospital NHS Foundation Trust, Londres, Reino Unido.
- Academic Unit of OphthalmologyInstitute of Inflammation and AgeingUniversity of BirminghamBirminghamReino UnidoAcademic Unit of Ophthalmology, Institute of Inflammation and Ageing, University of Birmingham, Birmingham, Reino Unido.
- University Hospitals Birmingham NHS Foundation TrustBirminghamReino UnidoUniversity Hospitals Birmingham NHS Foundation Trust, Birmingham, Reino Unido.
- Health Data Research Reino UnidoLondresReino UnidoHealth Data Research Reino Unido, Londres, Reino Unido.
- Birmingham Health Partners Centre for Regulatory Science and InnovationUniversity of BirminghamBirminghamReino UnidoBirmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, Reino Unido.
| | - Samantha Cruz Rivera
- Birmingham Health Partners Centre for Regulatory Science and InnovationUniversity of BirminghamBirminghamReino UnidoBirmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, Reino Unido.
- Centre for Patient Reported Outcomes ResearchInstitute of Applied Health ResearchUniversity of BirminghamBirmingham Centre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham.
- Institute of Applied Health ResearchUniversity of BirminghamBirminghamReino UnidoInstitute of Applied Health Research, University of Birmingham, Birmingham, Reino Unido.
| | - David Moher
- Centre for JournalologyClinical Epidemiology ProgramOttawa Hospital Research InstituteOttawaCanadáCentre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canadá.
- School of Epidemiology and Public HealthFaculty of MedicineUniversity of OttawaOttawaCanadaSchool of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Canada.
| | - Melanie J. Calvert
- Health Data Research Reino UnidoLondresReino UnidoHealth Data Research Reino Unido, Londres, Reino Unido.
- Birmingham Health Partners Centre for Regulatory Science and InnovationUniversity of BirminghamBirminghamReino UnidoBirmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, Reino Unido.
- Centre for Patient Reported Outcomes ResearchInstitute of Applied Health ResearchUniversity of BirminghamBirmingham Centre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham.
- Institute of Applied Health ResearchUniversity of BirminghamBirminghamReino UnidoInstitute of Applied Health Research, University of Birmingham, Birmingham, Reino Unido.
- National Institute of Health Research Birmingham Biomedical Research CentreUniversity of Birmingham and University Hospitals Birmingham NHS Foundation TrustBirminghamReino UnidoNational Institute of Health Research Birmingham Biomedical Research Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, Reino Unido.
- National Institute of Health Research Applied Research Collaborative West MidlandsCoventryReino Unido.National Institute of Health Research Applied Research Collaborative West Midlands, Coventry, Reino Unido.
- National Institute of Health Research Surgical Reconstruction and Microbiology CentreUniversity of Birmingham and University Hospitals Birmingham NHS Foundation TrustBirminghamReino UnidoNational Institute of Health Research Surgical Reconstruction and Microbiology Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, Reino Unido.
| | - Alastair K. Denniston
- Academic Unit of OphthalmologyInstitute of Inflammation and AgeingUniversity of BirminghamBirminghamReino UnidoAcademic Unit of Ophthalmology, Institute of Inflammation and Ageing, University of Birmingham, Birmingham, Reino Unido.
- University Hospitals Birmingham NHS Foundation TrustBirminghamReino UnidoUniversity Hospitals Birmingham NHS Foundation Trust, Birmingham, Reino Unido.
- Health Data Research Reino UnidoLondresReino UnidoHealth Data Research Reino Unido, Londres, Reino Unido.
- Birmingham Health Partners Centre for Regulatory Science and InnovationUniversity of BirminghamBirminghamReino UnidoBirmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, Reino Unido.
- Centre for Patient Reported Outcomes ResearchInstitute of Applied Health ResearchUniversity of BirminghamBirmingham Centre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham.
- NIHR Biomedical Research Center at Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of OphthalmologyLondresReino UnidoNIHR Biomedical Research Center at Moorfields Eye Hospital NHS Foundation Trust and UCL Institute of Ophthalmology, Londres, Reino Unido.
| | | |
Collapse
|
37
|
Nuutinen M, Hiltunen AM, Korhonen S, Haavisto I, Poikonen-Saksela P, Mattson J, Manikis G, Kondylakis H, Simos P, Mazzocco K, Pat-Horenczyk R, Sousa B, Cardoso F, Manica I, Kudel I, Leskelä RL. Aid of a machine learning algorithm can improve clinician predictions of patient quality of life during breast cancer treatments. HEALTH AND TECHNOLOGY 2023. [DOI: 10.1007/s12553-023-00733-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/12/2023]
|
38
|
Rams, hounds and white boxes: Investigating human-AI collaboration protocols in medical diagnosis. Artif Intell Med 2023; 138:102506. [PMID: 36990586 DOI: 10.1016/j.artmed.2023.102506] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2022] [Revised: 01/23/2023] [Accepted: 02/04/2023] [Indexed: 02/10/2023]
Abstract
In this paper, we study human-AI collaboration protocols, a design-oriented construct aimed at establishing and evaluating how humans and AI can collaborate in cognitive tasks. We applied this construct in two user studies involving 12 specialist radiologists (the knee MRI study) and 44 ECG readers of varying expertise (the ECG study), who evaluated 240 and 20 cases, respectively, in different collaboration configurations. We confirm the utility of AI support but find that XAI can be associated with a "white-box paradox", producing a null or detrimental effect. We also find that the order of presentation matters: AI-first protocols are associated with higher diagnostic accuracy than human-first protocols, and with higher accuracy than both humans and AI alone. Our findings identify the best conditions for AI to augment human diagnostic skills, rather than trigger dysfunctional responses and cognitive biases that can undermine decision effectiveness.
Collapse
|
39
|
An efficient classification of cirrhosis liver disease using hybrid convolutional neural network-capsule network. Biomed Signal Process Control 2023. [DOI: 10.1016/j.bspc.2022.104152] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
40
|
Greenberg A, Samueli B, Fahoum I, Farkash S, Greenberg O, Zemser-Werner V, Sabo E, Hagege RR, Hershkovitz D. Short Training Significantly Improves Ganglion Cell Detection Using an Algorithm-Assisted Approach. Arch Pathol Lab Med 2023; 147:215-221. [PMID: 35738006 DOI: 10.5858/arpa.2021-0481-oa] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/17/2022] [Indexed: 02/05/2023]
Abstract
CONTEXT.— Medical education in pathology relies on the accumulation of experience gained through inspection of numerous samples from each entity. Acquiring sufficient teaching material for rare diseases, such as Hirschsprung disease (HSCR), may be difficult, especially in smaller institutes. The current study makes use of a previously developed decision support system using a decision support algorithm meant to aid pathologists in the diagnosis of HSCR. OBJECTIVE.— To assess the effect of a short training session on algorithm-assisted HSCR diagnosis. DESIGN.— Five pathologists reviewed a data set of 568 image sets (1704 images in total) selected from 50 cases by the decision support algorithm and were tasked with scoring the images for the presence or absence of ganglion cells. The task was repeated a total of 3 times. Each pathologist had to complete a short educational presentation between the second and third iterations. RESULTS.— The training resulted in a significantly increased rate of correct diagnoses (true positive/negative) and a decreased need for referrals for expert consultation. No statistically significant changes in the rate of false positives/negatives were detected. CONCLUSIONS.— A very short (<10 minutes) training session can greatly improve the pathologist's performance in the algorithm-assisted diagnosis of HSCR. The same approach may be feasible in training for the diagnosis of other rare diseases.
Collapse
Affiliation(s)
- Ariel Greenberg
- From Institute of Pathology, Tel Aviv Sourasky Medical Center, Tel Aviv, Israel (A Greenberg, Fahoum, O Greenberg, Zemser-Werner, Hagege, Hershkovitz)
| | - Benzion Samueli
- From the Department of Pathology, Soroka University Medical Center, Be'er Sheva, Israel (Samueli)
| | - Ibrahim Fahoum
- From Institute of Pathology, Tel Aviv Sourasky Medical Center, Tel Aviv, Israel (A Greenberg, Fahoum, O Greenberg, Zemser-Werner, Hagege, Hershkovitz)
| | - Shai Farkash
- From the Institute of Pathology, Emek Medical Center, Afula, Israel (Farkash)
| | - Orli Greenberg
- From Institute of Pathology, Tel Aviv Sourasky Medical Center, Tel Aviv, Israel (A Greenberg, Fahoum, O Greenberg, Zemser-Werner, Hagege, Hershkovitz)
| | - Valentina Zemser-Werner
- From Institute of Pathology, Tel Aviv Sourasky Medical Center, Tel Aviv, Israel (A Greenberg, Fahoum, O Greenberg, Zemser-Werner, Hagege, Hershkovitz)
| | - Edmond Sabo
- From the Institute of Pathology, Carmel Medical Center, Haifa, Israel (Sabo).,From the Rappaport Faculty of Medicine, Technion, Haifa, Israel (Sabo)
| | - Rami R Hagege
- From Institute of Pathology, Tel Aviv Sourasky Medical Center, Tel Aviv, Israel (A Greenberg, Fahoum, O Greenberg, Zemser-Werner, Hagege, Hershkovitz).,Hagege and Hershkovitz contributed equally to the research
| | - Dov Hershkovitz
- From the Sackler Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel (Hershkovitz).,Hagege and Hershkovitz contributed equally to the research
| |
Collapse
|
41
|
Cruz Rivera S, Liu X, Chan AW, Denniston AK, Calvert MJ. [Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extensionDiretrizes para protocolos de ensaios clínicos com intervenções que utilizam inteligência artificial: a extensão SPIRIT-AI]. Rev Panam Salud Publica 2023; 48:e12. [PMID: 38304411 PMCID: PMC10832304 DOI: 10.26633/rpsp.2024.12] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2020] [Accepted: 07/23/2020] [Indexed: 02/03/2024] Open
Abstract
The SPIRIT 2013 statement aims to improve the completeness of clinical trial protocol reporting by providing evidence-based recommendations for the minimum set of items to be addressed. This guidance has been instrumental in promoting transparent evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate their impact on health outcomes. The SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence) extension is a new reporting guideline for clinical trial protocols evaluating interventions with an AI component. It was developed in parallel with its companion statement for trial reports: CONSORT-AI (Consolidated Standards of Reporting Trials-Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 26 candidate items, which were consulted upon by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The SPIRIT-AI extension includes 15 new items that were considered sufficiently important for clinical trial protocols of AI interventions. These new items should be routinely reported in addition to the core SPIRIT 2013 items. SPIRIT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention will be integrated, considerations for the handling of input and output data, the human-AI interaction and analysis of error cases. SPIRIT-AI will help promote transparency and completeness for clinical trial protocols for AI interventions. Its use will assist editors and peer reviewers, as well as the general readership, to understand, interpret and critically appraise the design and risk of bias for a planned clinical trial.
Collapse
Affiliation(s)
- Samantha Cruz Rivera
- Centre for Patient Reported Outcomes ResearchInstitute of Applied Health ResearchUniversity of BirminghamBirminghamReino UnidoCentre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham, Reino Unido.
- Institute of Applied Health ResearchUniversity of BirminghamBirminghamReino UnidoInstitute of Applied Health Research, University of Birmingham, Birmingham, Reino Unido.
- Birmingham Health Partners Centre for Regulatory Science and InnovationUniversity of BirminghamBirminghamReino UnidoBirmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, Reino Unido.
| | - Xiaoxuan Liu
- Birmingham Health Partners Centre for Regulatory Science and InnovationUniversity of BirminghamBirminghamReino UnidoBirmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, Reino Unido.
- Academic Unit of OphthalmologyInstitute of Inflammation and AgeingUniversity of BirminghamBirminghamReino UnidoAcademic Unit of Ophthalmology, Institute of Inflammation and Ageing, University of Birmingham, Birmingham, Reino Unido.
- University Hospitals Birmingham NHS Foundation TrustBirminghamReino UnidoUniversity Hospitals Birmingham NHS Foundation Trust, Birmingham, Reino Unido.
- Health Data Research UKLondresReino UnidoHealth Data Research UK, Londres, Reino Unido.
- Moorfields Eye Hospital NHS Foundation TrustLondresReino UnidoMoorfields Eye Hospital NHS Foundation Trust, Londres, Reino Unido.
| | - An-Wen Chan
- Department of Medicine, Women’s College Research InstituteWomen’s College HospitalUniversity of TorontoOntarioCanadáDepartment of Medicine, Women’s College Research Institute, Women’s College Hospital, University of Toronto, Ontario, Canadá.
| | - Alastair K. Denniston
- Centre for Patient Reported Outcomes ResearchInstitute of Applied Health ResearchUniversity of BirminghamBirminghamReino UnidoCentre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham, Reino Unido.
- Birmingham Health Partners Centre for Regulatory Science and InnovationUniversity of BirminghamBirminghamReino UnidoBirmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, Reino Unido.
- Academic Unit of OphthalmologyInstitute of Inflammation and AgeingUniversity of BirminghamBirminghamReino UnidoAcademic Unit of Ophthalmology, Institute of Inflammation and Ageing, University of Birmingham, Birmingham, Reino Unido.
- University Hospitals Birmingham NHS Foundation TrustBirminghamReino UnidoUniversity Hospitals Birmingham NHS Foundation Trust, Birmingham, Reino Unido.
- Health Data Research UKLondresReino UnidoHealth Data Research UK, Londres, Reino Unido.
- National Institute of Health Research Biomedical Research Centre for OphthalmologyMoorfields Hospital London NHS Foundation Trust and University College LondonInstitute of OphthalmologyLondresReino UnidoNational Institute of Health Research Biomedical Research Centre for Ophthalmology, Moorfields Hospital London NHS Foundation Trust and University College London, Institute of Ophthalmology, Londres, Reino Unido.
| | - Melanie J. Calvert
- Centre for Patient Reported Outcomes ResearchInstitute of Applied Health ResearchUniversity of BirminghamBirminghamReino UnidoCentre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham, Reino Unido.
- Institute of Applied Health ResearchUniversity of BirminghamBirminghamReino UnidoInstitute of Applied Health Research, University of Birmingham, Birmingham, Reino Unido.
- Birmingham Health Partners Centre for Regulatory Science and InnovationUniversity of BirminghamBirminghamReino UnidoBirmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, Reino Unido.
- Health Data Research UKLondresReino UnidoHealth Data Research UK, Londres, Reino Unido.
- National Institute of Health Research Birmingham Biomedical Research CentreUniversity of Birmingham and University Hospitals Birmingham NHS Foundation TrustBirminghamReino UnidoNational Institute of Health Research Birmingham Biomedical Research Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, Reino Unido.
- National Institute of Health Research Applied Research Collaborative West MidlandsCoventryReino UnidoNational Institute of Health Research Applied Research Collaborative West Midlands, Coventry, Reino Unido.
- National Institute of Health Research Surgical Reconstruction and Microbiology CentreUniversity of Birmingham and University Hospitals Birmingham NHS Foundation TrustBirminghamReino UnidoNational Institute of Health Research Surgical Reconstruction and Microbiology Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, Reino Unido.
| | | |
Collapse
|
42
|
Segmentation-Assisted Fully Convolutional Neural Network Enhances Deep Learning Performance to Identify Proliferative Diabetic Retinopathy. J Clin Med 2023; 12:jcm12010385. [PMID: 36615186 PMCID: PMC9821182 DOI: 10.3390/jcm12010385] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2022] [Revised: 12/27/2022] [Accepted: 12/29/2022] [Indexed: 01/05/2023] Open
Abstract
With the progression of diabetic retinopathy (DR) from the non-proliferative (NPDR) to proliferative (PDR) stage, the possibility of vision impairment increases significantly. Therefore, it is clinically important to detect the progression to PDR stage for proper intervention. We propose a segmentation-assisted DR classification methodology, that builds on (and improves) current methods by using a fully convolutional network (FCN) to segment retinal neovascularizations (NV) in retinal images prior to image classification. This study utilizes the Kaggle EyePacs dataset, containing retinal photographs from patients with varying degrees of DR (mild, moderate, severe NPDR and PDR. Two graders annotated the NV (a board-certified ophthalmologist and a trained medical student). Segmentation was performed by training an FCN to locate neovascularization on 669 retinal fundus photographs labeled with PDR status according to NV presence. The trained segmentation model was used to locate probable NV in images from the classification dataset. Finally, a CNN was trained to classify the combined images and probability maps into categories of PDR. The mean accuracy of segmentation-assisted classification was 87.71% on the test set (SD = 7.71%). Segmentation-assisted classification of PDR achieved accuracy that was 7.74% better than classification alone. Our study shows that segmentation assistance improves identification of the most severe stage of diabetic retinopathy and has the potential to improve deep learning performance in other imaging problems with limited data availability.
Collapse
|
43
|
Wu B, Moeckel G. Application of digital pathology and machine learning in the liver, kidney and lung diseases. J Pathol Inform 2023; 14:100184. [PMID: 36714454 PMCID: PMC9874068 DOI: 10.1016/j.jpi.2022.100184] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2022] [Revised: 11/28/2022] [Accepted: 12/28/2022] [Indexed: 01/05/2023] Open
Abstract
The development of rapid and accurate Whole Slide Imaging (WSI) has paved the way for the application of Artificial Intelligence (AI) to digital pathology. The availability of WSI in the recent years allowed the rapid development of various AI technologies to blossom. WSI-based digital pathology combined with neural networks can automate arduous and time-consuming tasks of slide evaluation. Machine Learning (ML)-based AI has been demonstrated to outperform pathologists by eliminating inter- and intra-observer subjectivity, obtaining quantitative data from slide images, and extracting hidden image patterns that are relevant to disease subtype and progression. In this review, we outline the functionality of different AI technologies such as neural networks and deep learning and discover how aspects of different diseases make them benefit from the implementation of AI. AI has proven to be valuable in many different organs, with this review focusing on the liver, kidney, and lungs. We also discuss how AI and image analysis not only can grade diseases objectively but also discover aspects of diseases that have prognostic value. In the end, we review the current status of the integration of AI in pathology and share our vision on the future of digital pathology.
Collapse
Affiliation(s)
- Benjamin Wu
- Horace Mann School, Bronx, NY, USA,Corresponding author at: 950 Post Rd., Scarsdale, NY 10583, USA.
| | - Gilbert Moeckel
- Department of Pathology, Yale University School of Medicine, New Haven, CT, USA
| |
Collapse
|
44
|
Kalapala R, Rughwani H, Reddy DN. Artificial Intelligence in Hepatology- Ready for the Primetime. J Clin Exp Hepatol 2023; 13:149-161. [PMID: 36647407 PMCID: PMC9840075 DOI: 10.1016/j.jceh.2022.06.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/19/2022] [Accepted: 06/23/2022] [Indexed: 02/07/2023] Open
Abstract
Artificial Intelligence (AI) is a mathematical process of computer mediating designing of algorithms to support human intelligence. AI in hepatology has shown tremendous promise to plan appropriate management and hence improve treatment outcomes. The field of AI is in a very early phase with limited clinical use. AI tools such as machine learning, deep learning, and 'big data' are in a continuous phase of evolution, presently being applied for clinical and basic research. In this review, we have summarized various AI applications in hepatology, the pitfalls and AI's future implications. Different AI models and algorithms are under study using clinical, laboratory, endoscopic and imaging parameters to diagnose and manage liver diseases and mass lesions. AI has helped to reduce human errors and improve treatment protocols. Further research and validation are required for future use of AI in hepatology.
Collapse
Key Words
- ACLF, acute on chronic liver failure
- AI, artificial intelligence
- ALD, alcoholic liver disease
- ALT, alanine transaminase
- ANN, artificial neural network
- AST, aspartate aminotransferase
- AUD, alcohol use disorder
- CHB, chronic hepatitis B
- CHC, chronic hepatitis C
- CLD, chronic liver disease
- CNN, convolutional neural network
- DL, deep learning
- FIB-4, fibrosis-4 score
- GGTP, gamma glutamyl transferase
- HCC, hepatocellular carcinoma
- HDL, high density lipoprotein
- ML, machine learning
- MLR, multi-nomial logistic regressions
- NAFLD
- NAFLD, non-alcoholic fatty liver disease
- NASH, non-alcoholic steatohepatitis
- NLP, natural language processing
- RF, random forest
- RTE, real-time tissue elastography
- SOLs, space-occupying lesions
- SVM, support vector machine
- artificial intelligence
- deep learning
- hepatology
- machine learning
Collapse
Affiliation(s)
- Rakesh Kalapala
- Department of Gastroenterology, Asian Institute of Gastroenterology and AIG Hospitals, Hyderabad, India
| | - Hardik Rughwani
- Department of Gastroenterology, Asian Institute of Gastroenterology and AIG Hospitals, Hyderabad, India
| | | |
Collapse
|
45
|
Ko YS, Choi YM, Kim M, Park Y, Ashraf M, Quiñones Robles WR, Kim MJ, Jang J, Yun S, Hwang Y, Jang H, Yi MY. Improving quality control in the routine practice for histopathological interpretation of gastrointestinal endoscopic biopsies using artificial intelligence. PLoS One 2022; 17:e0278542. [PMID: 36520777 PMCID: PMC9754254 DOI: 10.1371/journal.pone.0278542] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2022] [Accepted: 11/18/2022] [Indexed: 12/23/2022] Open
Abstract
BACKGROUND Colorectal and gastric cancer are major causes of cancer-related deaths. In Korea, gastrointestinal (GI) endoscopic biopsy specimens account for a high percentage of histopathologic examinations. Lack of a sufficient pathologist workforce can cause an increase in human errors, threatening patient safety. Therefore, we developed a digital pathology total solution combining artificial intelligence (AI) classifier models and pathology laboratory information system for GI endoscopic biopsy specimens to establish a post-analytic daily fast quality control (QC) system, which was applied in clinical practice for a 3-month trial run by four pathologists. METHODS AND FINDINGS Our whole slide image (WSI) classification framework comprised patch-generator, patch-level classifier, and WSI-level classifier. The classifiers were both based on DenseNet (Dense Convolutional Network). In laboratory tests, the WSI classifier achieved accuracy rates of 95.8% and 96.0% in classifying histopathological WSIs of colorectal and gastric endoscopic biopsy specimens, respectively, into three classes (Negative for dysplasia, Dysplasia, and Malignant). Classification by pathologic diagnosis and AI prediction were compared and daily reviews were conducted, focusing on discordant cases for early detection of potential human errors by the pathologists, allowing immediate correction, before the pathology report error is conveyed to the patients. During the 3-month AI-assisted daily QC trial run period, approximately 7-10 times the number of slides compared to that in the conventional monthly QC (33 months) were reviewed by pathologists; nearly 100% of GI endoscopy biopsy slides were double-checked by the AI models. Further, approximately 17-30 times the number of potential human errors were detected within an average of 1.2 days. CONCLUSIONS The AI-assisted daily QC system that we developed and established demonstrated notable improvements in QC, in quantitative, qualitative, and time utility aspects. Ultimately, we developed an independent AI-assisted post-analytic daily fast QC system that was clinically applicable and influential, which could enhance patient safety.
Collapse
Affiliation(s)
- Young Sin Ko
- Pathology Center, Seegene Medical Foundation, Seoul, Republic of Korea
- * E-mail: (YSK); (MYY)
| | - Yoo Mi Choi
- Pathology Center, Seegene Medical Foundation, Seoul, Republic of Korea
| | - Mujin Kim
- Graduate School of Data Science, Department of Industrial & Systems Engineering, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea
| | - Youngjin Park
- Graduate School of Data Science, Department of Industrial & Systems Engineering, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea
| | - Murtaza Ashraf
- Graduate School of Data Science, Department of Industrial & Systems Engineering, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea
| | - Willmer Rafell Quiñones Robles
- Graduate School of Data Science, Department of Industrial & Systems Engineering, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea
| | - Min-Ju Kim
- Department of Pathology, Incheon Sejong Hospital, Incheon, Republic of Korea
| | - Jiwook Jang
- AI Research Team, Digital Innovation Sector, Seegene Medical Foundation, Seoul, Republic of Korea
| | - Seokju Yun
- AI Research Team, Digital Innovation Sector, Seegene Medical Foundation, Seoul, Republic of Korea
| | - Yuri Hwang
- AI Research Team, Digital Innovation Sector, Seegene Medical Foundation, Seoul, Republic of Korea
| | - Hani Jang
- AI Research Team, Digital Innovation Sector, Seegene Medical Foundation, Seoul, Republic of Korea
| | - Mun Yong Yi
- Graduate School of Data Science, Department of Industrial & Systems Engineering, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea
- * E-mail: (YSK); (MYY)
| |
Collapse
|
46
|
Marsh P, Radif D, Rajpurkar P, Wang Z, Hariton E, Ribeiro S, Simbulan R, Kaing A, Lin W, Rajah A, Rabara F, Lungren M, Demirci U, Ng A, Rosen M. A proof of concept for a deep learning system that can aid embryologists in predicting blastocyst survival after thaw. Sci Rep 2022; 12:21119. [PMID: 36477633 PMCID: PMC9729222 DOI: 10.1038/s41598-022-25062-z] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2022] [Accepted: 11/24/2022] [Indexed: 12/12/2022] Open
Abstract
The ability to understand whether embryos survive the thaw process is crucial to transferring competent embryos that can lead to pregnancy. The objective of this study was to develop a proof of concept deep learning model capable of assisting embryologist assessment of survival of thawed blastocysts prior to embryo transfer. A deep learning model was developed using 652 labeled time-lapse videos of freeze-thaw blastocysts. The model was evaluated against and along embryologists on a test set of 99 freeze-thaw blastocysts, using images obtained at 0.5 h increments from 0 to 3 h post-thaw. The model achieved AUCs of 0.869 (95% CI 0.789, 0.934) and 0.807 (95% CI 0.717, 0.886) and the embryologists achieved average AUCs of 0.829 (95% CI 0.747, 0.896) and 0.850 (95% CI 0.773, 0.908) at 2 h and 3 h, respectively. Combining embryologist predictions with model predictions resulted in a significant increase in AUC of 0.051 (95% CI 0.021, 0.083) at 2 h, and an equivalent increase in AUC of 0.010 (95% CI -0.018, 0.037) at 3 h. This study suggests that a deep learning model can predict in vitro blastocyst survival after thaw in aneuploid embryos. After correlation with clinical outcomes of transferred embryos, this model may help embryologists ascertain which embryos may have failed to survive the thaw process and increase the likelihood of pregnancy by preventing the transfer of non-viable embryos.
Collapse
Affiliation(s)
- P. Marsh
- grid.266102.10000 0001 2297 6811Center for Reproductive Health, Department of Medicine, University of California, San Francisco, USA
| | - D. Radif
- grid.168010.e0000000419368956Department of Computer Science, Stanford University, Stanford, USA
| | - P. Rajpurkar
- grid.168010.e0000000419368956Department of Computer Science, Stanford University, Stanford, USA
| | - Z. Wang
- grid.168010.e0000000419368956Department of Computer Science, Stanford University, Stanford, USA
| | - E. Hariton
- grid.266102.10000 0001 2297 6811Center for Reproductive Health, Department of Medicine, University of California, San Francisco, USA
| | - S. Ribeiro
- grid.266102.10000 0001 2297 6811Center for Reproductive Health, Department of Medicine, University of California, San Francisco, USA
| | - R. Simbulan
- grid.266102.10000 0001 2297 6811Center for Reproductive Health, Department of Medicine, University of California, San Francisco, USA
| | - A. Kaing
- grid.266102.10000 0001 2297 6811Center for Reproductive Health, Department of Medicine, University of California, San Francisco, USA
| | - W. Lin
- grid.266102.10000 0001 2297 6811Center for Reproductive Health, Department of Medicine, University of California, San Francisco, USA
| | - A. Rajah
- grid.266102.10000 0001 2297 6811Center for Reproductive Health, Department of Medicine, University of California, San Francisco, USA
| | - F. Rabara
- grid.266102.10000 0001 2297 6811Center for Reproductive Health, Department of Medicine, University of California, San Francisco, USA
| | - M. Lungren
- grid.168010.e0000000419368956Center for Artificial Intelligence in Medicine & Imaging, Stanford University, Stanford, USA
| | - U. Demirci
- grid.168010.e0000000419368956Canary Center for Cancer Early Detection, Stanford University, Stanford, USA
| | - A. Ng
- grid.168010.e0000000419368956Department of Computer Science, Stanford University, Stanford, USA
| | - M. Rosen
- grid.266102.10000 0001 2297 6811Center for Reproductive Health, Department of Medicine, University of California, San Francisco, USA
| |
Collapse
|
47
|
Zhang A, Xing L, Zou J, Wu JC. Shifting machine learning for healthcare from development to deployment and from models to data. Nat Biomed Eng 2022; 6:1330-1345. [PMID: 35788685 DOI: 10.1038/s41551-022-00898-y] [Citation(s) in RCA: 36] [Impact Index Per Article: 18.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2021] [Accepted: 05/03/2022] [Indexed: 01/14/2023]
Abstract
In the past decade, the application of machine learning (ML) to healthcare has helped drive the automation of physician tasks as well as enhancements in clinical capabilities and access to care. This progress has emphasized that, from model development to model deployment, data play central roles. In this Review, we provide a data-centric view of the innovations and challenges that are defining ML for healthcare. We discuss deep generative models and federated learning as strategies to augment datasets for improved model performance, as well as the use of the more recent transformer models for handling larger datasets and enhancing the modelling of clinical text. We also discuss data-focused problems in the deployment of ML, emphasizing the need to efficiently deliver data to ML models for timely clinical predictions and to account for natural data shifts that can deteriorate model performance.
Collapse
Affiliation(s)
- Angela Zhang
- Stanford Cardiovascular Institute, School of Medicine, Stanford University, Stanford, CA, USA. .,Department of Genetics, School of Medicine, Stanford University, Stanford, CA, USA. .,Greenstone Biosciences, Palo Alto, CA, USA. .,Department of Computer Science, Stanford University, Stanford, CA, USA.
| | - Lei Xing
- Department of Radiation Oncology, School of Medicine, Stanford University, Stanford, CA, USA
| | - James Zou
- Department of Computer Science, Stanford University, Stanford, CA, USA.,Department of Biomedical Informatics, School of Medicine, Stanford University, Stanford, CA, USA
| | - Joseph C Wu
- Stanford Cardiovascular Institute, School of Medicine, Stanford University, Stanford, CA, USA. .,Greenstone Biosciences, Palo Alto, CA, USA. .,Departments of Medicine, Division of Cardiovascular Medicine Stanford University, Stanford, CA, USA. .,Department of Radiology, School of Medicine, Stanford University, Stanford, CA, USA.
| |
Collapse
|
48
|
Muto R, Fukuta S, Watanabe T, Shindo Y, Kanemitsu Y, Kajikawa S, Yonezawa T, Inoue T, Ichihashi T, Shiratori Y, Maruyama S. Predicting oxygen requirements in patients with coronavirus disease 2019 using an artificial intelligence-clinician model based on local non-image data. Front Med (Lausanne) 2022; 9:1042067. [DOI: 10.3389/fmed.2022.1042067] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2022] [Accepted: 11/14/2022] [Indexed: 12/05/2022] Open
Abstract
BackgroundWhen facing unprecedented emergencies such as the coronavirus disease 2019 (COVID-19) pandemic, a predictive artificial intelligence (AI) model with real-time customized designs can be helpful for clinical decision-making support in constantly changing environments. We created models and compared the performance of AI in collaboration with a clinician and that of AI alone to predict the need for supplemental oxygen based on local, non-image data of patients with COVID-19.Materials and methodsWe enrolled 30 patients with COVID-19 who were aged >60 years on admission and not treated with oxygen therapy between December 1, 2020 and January 4, 2021 in this 50-bed, single-center retrospective cohort study. The outcome was requirement for oxygen after admission.ResultsThe model performance to predict the need for oxygen by AI in collaboration with a clinician was better than that by AI alone. Sodium chloride difference >33.5 emerged as a novel indicator to predict the need for oxygen in patients with COVID-19. To prevent severe COVID-19 in older patients, dehydration compensation may be considered in pre-hospitalization care.ConclusionIn clinical practice, our approach enables the building of a better predictive model with prompt clinician feedback even in new scenarios. These can be applied not only to current and future pandemic situations but also to other diseases within the healthcare system.
Collapse
|
49
|
Li J, Zhou L, Zhan Y, Xu H, Zhang C, Shan F, Liu L. How does the artificial intelligence-based image-assisted technique help physicians in diagnosis of pulmonary adenocarcinoma? A randomized controlled experiment of multicenter physicians in China. J Am Med Inform Assoc 2022; 29:2041-2049. [PMID: 36228127 PMCID: PMC9667181 DOI: 10.1093/jamia/ocac179] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2022] [Revised: 08/24/2022] [Accepted: 09/24/2022] [Indexed: 11/14/2022] Open
Abstract
OBJECTIVE Although artificial intelligence (AI) has achieved high levels of accuracy in the diagnosis of various diseases, its impact on physicians' decision-making performance in clinical practice is uncertain. This study aims to assess the impact of AI on the diagnostic performance of physicians with differing levels of self-efficacy under working conditions involving different time pressures. MATERIALS AND METHODS A 2 (independent diagnosis vs AI-assisted diagnosis) × 2 (no time pressure vs 2-minute time limit) randomized controlled experiment of multicenter physicians was conducted. Participants diagnosed 10 pulmonary adenocarcinoma cases and their diagnostic accuracy, sensitivity, and specificity were evaluated. Data analysis was performed using multilevel logistic regression. RESULTS One hundred and four radiologists from 102 hospitals completed the experiment. The results reveal (1) AI greatly increases physicians' diagnostic accuracy, either with or without time pressure; (2) when no time pressure, AI significantly improves physicians' diagnostic sensitivity but no significant change in specificity, while under time pressure, physicians' diagnostic sensitivity and specificity are both improved with the aid of AI; (3) when no time pressure, physicians with low self-efficacy benefit from AI assistance thus improving diagnostic accuracy but those with high self-efficacy do not, whereas physicians with low and high levels of self-efficacy both benefit from AI under time pressure. DISCUSSION This study is one of the first to provide real-world evidence regarding the impact of AI on physicians' decision-making performance, taking into account 2 boundary factors: clinical time pressure and physicians' self-efficacy. CONCLUSION AI-assisted diagnosis should be prioritized for physicians working under time pressure or with low self-efficacy.
Collapse
Affiliation(s)
- Jiaoyang Li
- School of Business Administration, Faculty of Business Administration, Southwestern University of Finance and Economics, Chengdu 611130, China
| | - Lingxiao Zhou
- Institute of Microscale Optoelectronics, Shenzhen University, Shenzhen 518060, China
| | - Yi Zhan
- Department of Radiology, Shanghai Public Health Clinical Center, Fudan University, Shanghai 201508, China
| | - Haifeng Xu
- Antai College of Economics and Management, Shanghai Jiao Tong University, Shanghai 200030, China
| | - Cheng Zhang
- School of Management, Fudan University, Shanghai 200433, China
| | - Fei Shan
- Department of Radiology, Shanghai Public Health Clinical Center, Fudan University, Shanghai 201508, China
| | - Lei Liu
- Intelligent Medicine Institute, Fudan University, Shanghai 200030, China
| |
Collapse
|
50
|
Grote T. Randomised controlled trials in medical AI: ethical considerations. JOURNAL OF MEDICAL ETHICS 2022; 48:899-906. [PMID: 33990429 DOI: 10.1136/medethics-2020-107166] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/14/2020] [Revised: 03/30/2021] [Accepted: 04/08/2021] [Indexed: 06/12/2023]
Abstract
In recent years, there has been a surge of high-profile publications on applications of artificial intelligence (AI) systems for medical diagnosis and prognosis. While AI provides various opportunities for medical practice, there is an emerging consensus that the existing studies show considerable deficits and are unable to establish the clinical benefit of AI systems. Hence, the view that the clinical benefit of AI systems needs to be studied in clinical trials-particularly randomised controlled trials (RCTs)-is gaining ground. However, an issue that has been overlooked so far in the debate is that, compared with drug RCTs, AI RCTs require methodological adjustments, which entail ethical challenges. This paper sets out to develop a systematic account of the ethics of AI RCTs by focusing on the moral principles of clinical equipoise, informed consent and fairness. This way, the objective is to animate further debate on the (research) ethics of medical AI.
Collapse
Affiliation(s)
- Thomas Grote
- Ethics and Philosophy Lab, Cluster of Excellence "Machine Learning: New Perspectives for Science", University of Tübingen, Tübingen D-72076, Germany
| |
Collapse
|