1
|
Liang C, Pan S, Wu W, Chen F, Zhang C, Zhou C, Gao Y, Ruan X, Quan S, Zhao Q, Pan J. Glucocorticoid therapy for sepsis in the AI era: a survey on current and future approaches. Comput Struct Biotechnol J 2024; 24:292-305. [PMID: 38681133 PMCID: PMC11047203 DOI: 10.1016/j.csbj.2024.04.020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/05/2024] [Revised: 04/06/2024] [Accepted: 04/07/2024] [Indexed: 05/01/2024] Open
Abstract
Sepsis, a life-threatening medical condition, manifests as new or worsening organ failures due to a dysregulated host response to infection. Many patients with sepsis have manifested a hyperinflammatory phenotype leading to the identification of inflammatory modulation by corticosteroids as a key treatment modality. However, the optimal use of corticosteroids in sepsis treatment remains a contentious subject, necessitating a deeper understanding of their physiological and pharmacological effects. Our study conducts a comprehensive review of randomized controlled trials (RCTs) focusing on traditional corticosteroid treatment in sepsis, alongside an analysis of evolving clinical guidelines. Additionally, we explore the emerging role of artificial intelligence (AI) in medicine, particularly in diagnosing, prognosticating, and treating sepsis. AI's advanced data processing capabilities reveal new avenues for enhancing corticosteroid therapeutic strategies in sepsis. The integration of AI in sepsis treatment has the potential to address existing gaps in knowledge, especially in the application of corticosteroids. Our findings suggest that combining corticosteroid therapy with AI-driven insights could lead to more personalized and effective sepsis treatments. This approach holds promise for improving clinical outcomes and presents a significant advancement in the management of this complex and often fatal condition.
Collapse
Affiliation(s)
- Chenglong Liang
- The First Affiliated Hospital of Wenzhou Medical University, Wenzhou 325000, China
- Wenzhou Medical University, Wenzhou 325000, China
- School of Nursing, Wenzhou Medical University, Wenzhou 325000, China
| | - Shuo Pan
- Wenzhou Medical University, Wenzhou 325000, China
| | - Wei Wu
- School of Computer Science and Software Engineering, University of Science and Technology Liaoning, Anshan 114051, China
| | - Fanxuan Chen
- Wenzhou Medical University, Wenzhou 325000, China
- School of Biomedical Engineering, School of Ophthalmology and Optometry, Eye Hospital, Wenzhou Medical University, Wenzhou 325000, China
| | - Chengxi Zhang
- School of Materials Science and Engineering, Shandong Jianzhu University, Jinan 250101, China
| | - Chen Zhou
- The First Affiliated Hospital of Wenzhou Medical University, Wenzhou 325000, China
| | - Yifan Gao
- Department of Intensive Care Unit, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou 325000, China
| | - Xiangyuan Ruan
- The First Affiliated Hospital of Wenzhou Medical University, Wenzhou 325000, China
| | - Shichao Quan
- Department of Big Data in Health Science, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou 325000, China
| | - Qi Zhao
- School of Computer Science and Software Engineering, University of Science and Technology Liaoning, Anshan 114051, China
| | - Jingye Pan
- The First Affiliated Hospital of Wenzhou Medical University, Wenzhou 325000, China
- Department of Intensive Care Unit, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou 325000, China
- Key Laboratory of Intelligent Treatment and Life Support for Critical Diseases of Zhejiang Province, Wenzhou 325000, China
- Wenzhou Key Laboratory of Critical Care and Artificial Intelligence, Wenzhou 325000, China
- Zhejiang Engineering Research Center for Hospital Emergency and Process Digitization, Wenzhou 325000, China
| |
Collapse
|
2
|
Gonzalez R, Saha A, Campbell CJ, Nejat P, Lokker C, Norgan AP. Seeing the random forest through the decision trees. Supporting learning health systems from histopathology with machine learning models: Challenges and opportunities. J Pathol Inform 2024; 15:100347. [PMID: 38162950 PMCID: PMC10755052 DOI: 10.1016/j.jpi.2023.100347] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2023] [Revised: 10/06/2023] [Accepted: 11/01/2023] [Indexed: 01/03/2024] Open
Abstract
This paper discusses some overlooked challenges faced when working with machine learning models for histopathology and presents a novel opportunity to support "Learning Health Systems" with them. Initially, the authors elaborate on these challenges after separating them according to their mitigation strategies: those that need innovative approaches, time, or future technological capabilities and those that require a conceptual reappraisal from a critical perspective. Then, a novel opportunity to support "Learning Health Systems" by integrating hidden information extracted by ML models from digitalized histopathology slides with other healthcare big data is presented.
Collapse
Affiliation(s)
- Ricardo Gonzalez
- DeGroote School of Business, McMaster University, Hamilton, Ontario, Canada
- Division of Computational Pathology and Artificial Intelligence, Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, United States
| | - Ashirbani Saha
- Department of Oncology, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
- Escarpment Cancer Research Institute, McMaster University and Hamilton Health Sciences, Hamilton, Ontario, Canada
| | - Clinton J.V. Campbell
- William Osler Health System, Brampton, Ontario, Canada
- Department of Pathology and Molecular Medicine, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
| | - Peyman Nejat
- Department of Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, United States
| | - Cynthia Lokker
- Health Information Research Unit, Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Andrew P. Norgan
- Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, United States
| |
Collapse
|
3
|
Bird A, McMaster C, Liew D. Clinical evaluation is critical for the implementation of artificial intelligence in health care: comment on the article by Mickley et al. Arthritis Care Res (Hoboken) 2024; 76:904. [PMID: 38317297 DOI: 10.1002/acr.25310] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2023] [Accepted: 11/20/2023] [Indexed: 02/07/2024]
Affiliation(s)
- Alix Bird
- University of Adelaide, Adelaide, South Australia, Australia
| | | | - David Liew
- Austin Health and University of Melbourne, Melbourne, Victoria, Australia
| |
Collapse
|
4
|
Telenti A, Auli M, Hie BL, Maher C, Saria S, Ioannidis JPA. Large language models for science and medicine. Eur J Clin Invest 2024; 54:e14183. [PMID: 38381530 DOI: 10.1111/eci.14183] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/22/2023] [Revised: 02/06/2024] [Accepted: 02/10/2024] [Indexed: 02/23/2024]
Abstract
Large language models (LLMs) are a type of machine learning model that learn statistical patterns over text, such as predicting the next words in a sequence of text. Both general purpose and task-specific LLMs have demonstrated potential across diverse applications. Science and medicine have many data types that are highly suitable for LLMs, such as scientific texts (publications, patents and textbooks), electronic medical records, large databases of DNA and protein sequences and chemical compounds. Carefully validated systems that can understand and reason across all these modalities may maximize benefits. Despite the inevitable limitations and caveats of any new technology and some uncertainties specific to LLMs, LLMs have the potential to be transformative in science and medicine.
Collapse
Affiliation(s)
- Amalio Telenti
- Department of Integrative Structural and Computational Biology, Scripps Research, La Jolla, California, USA
- Vir Biotechnology, Inc., San Francisco, California, USA
| | | | - Brian L Hie
- FAIR, Meta, Menlo Park, California, USA
- Department of Chemical Engineering, Stanford University, Stanford, California, USA
| | - Cyrus Maher
- Vir Biotechnology, Inc., San Francisco, California, USA
| | - Suchi Saria
- Malone Center for Engineering and Healthcare, Johns Hopkins University, Baltimore, Maryland, USA
| | - John P A Ioannidis
- Department of Medicine, Stanford University, Stanford, California, USA
- Department of Epidemiology and Population Health, Stanford University, Stanford, California, USA
- Department of Biomedical Data Science, Stanford University, Stanford, California, USA
- Department of Statistics, Stanford University, Stanford, California, USA
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, California, USA
| |
Collapse
|
5
|
Reich C, Frey N, Giannitsis E. [Digitalization and clinical decision tools]. Herz 2024; 49:190-197. [PMID: 38453708 DOI: 10.1007/s00059-024-05242-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/13/2024] [Indexed: 03/09/2024]
Abstract
Digitalization in cardiovascular emergencies is rapidly evolving, analogous to the development in medicine, driven by the increasingly broader availability of digital structures and improved networks, electronic health records and the interconnectivity of systems. The potential use of digital health in patients with acute chest pain starts even in the prehospital phase with the transmission of a digital electrocardiogram (ECG) as well as telemedical support and digital emergency management, which facilitate optimization of the rescue pathways and reduce critical time intervals. The increasing dissemination and acceptance of guideline apps and clinical decision support tools as well as integrated calculators and electronic scores are anticipated to improve guideline adherence, translating into a better quality of treatment and improved outcomes. Implementation of artificial intelligence to support image analysis and also the prediction of coronary artery stenosis requiring interventional treatment or impending cardiovascular events, such as heart attacks or death, have an enormous potential especially as conventional instruments frequently yield suboptimal results; however, there are barriers to the rapid dissemination of corresponding decision aids, such as the regulatory rules related to approval as a medical product, data protection issues and other legal liability aspects, which must be considered.
Collapse
Affiliation(s)
| | | | - E Giannitsis
- Medizinische Klinik III, Universitätsklinikum Heidelberg, Im Neuenheimer Feld 410, 69120, Heidelberg, Deutschland.
| |
Collapse
|
6
|
Arina P, Mazomenos EB, Whittle J, Singer M. PROBAST Assessment of Machine Learning: Reply. Anesthesiology 2024:141507. [PMID: 38810020 DOI: 10.1097/aln.0000000000004998] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/31/2024]
Affiliation(s)
- Pietro Arina
- University College London, London, United Kingdom (P.A.).
| | | | | | | |
Collapse
|
7
|
Protocol for the development of the Chatbot Assessment Reporting Tool (CHART) for clinical advice. BMJ Open 2024; 14:e081155. [PMID: 38772889 PMCID: PMC11110548 DOI: 10.1136/bmjopen-2023-081155] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/24/2023] [Accepted: 03/26/2024] [Indexed: 05/23/2024] Open
Abstract
INTRODUCTION Large language model (LLM)-linked chatbots are being increasingly applied in healthcare due to their impressive functionality and public availability. Studies have assessed the ability of LLM-linked chatbots to provide accurate clinical advice. However, the methods applied in these Chatbot Assessment Studies are inconsistent due to the lack of reporting standards available, which obscures the interpretation of their study findings. This protocol outlines the development of the Chatbot Assessment Reporting Tool (CHART) reporting guideline. METHODS AND ANALYSIS The development of the CHART reporting guideline will consist of three phases, led by the Steering Committee. During phase one, the team will identify relevant reporting guidelines with artificial intelligence extensions that are published or in development by searching preprint servers, protocol databases, and the Enhancing the Quality and Transparency of health research Network. During phase two, we will conduct a scoping review to identify studies that have addressed the performance of LLM-linked chatbots in summarising evidence and providing clinical advice. The Steering Committee will identify methodology used in previous Chatbot Assessment Studies. Finally, the study team will use checklist items from prior reporting guidelines and findings from the scoping review to develop a draft reporting checklist. We will then perform a Delphi consensus and host two synchronous consensus meetings with an international, multidisciplinary group of stakeholders to refine reporting checklist items and develop a flow diagram. ETHICS AND DISSEMINATION We will publish the final CHART reporting guideline in peer-reviewed journals and will present findings at peer-reviewed meetings. Ethical approval was submitted to the Hamilton Integrated Research Ethics Board and deemed "not required" in accordance with the Tri-Council Policy Statement (TCPS2) for the development of the CHART reporting guideline (#17025). REGISTRATION This study protocol is preregistered with Open Science Framework: https://doi.org/10.17605/OSF.IO/59E2Q.
Collapse
|
8
|
Liu GS, Jovanovic N, Sung CK, Doyle PC. A Scoping Review of Artificial Intelligence Detection of Voice Pathology: Challenges and Opportunities. Otolaryngol Head Neck Surg 2024. [PMID: 38738887 DOI: 10.1002/ohn.809] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2023] [Revised: 04/05/2024] [Accepted: 04/19/2024] [Indexed: 05/14/2024]
Abstract
OBJECTIVE Survey the current literature on artificial intelligence (AI) applications for detecting and classifying vocal pathology using voice recordings, and identify challenges and opportunities for advancing the field forward. DATA SOURCES PubMed, EMBASE, CINAHL, and Scopus databases. REVIEW METHODS A comprehensive literature search was performed following the Preferred Reporting Items for Systematic Reviews and Meta-analyses Extension for Scoping Reviews guidelines. Peer-reviewed journal articles in the English language were included if they used an AI approach to detect or classify pathological voices using voice recordings from patients diagnosed with vocal pathologies. RESULTS Eighty-two studies were included in the review between the years 2000 and 2023, with an increase in publication rate from one study per year in 2012 to 10 per year in 2022. Seventy-two studies (88%) were aimed at detecting the presence of voice pathology, 24 (29%) at classifying the type of voice pathology present, and 4 (5%) at assessing pathological voice using the Grade, Roughness, Breathiness, Asthenia, and Strain scale. Thirty-six databases were used to collect and analyze speech samples. Fourteen articles (17%) did not provide information about their AI model validation methodology. Zero studies moved beyond the preclinical and offline AI model development stages. Zero studies specified following a reporting guideline for AI research. CONCLUSION There is rising interest in the potential of AI technology to aid the detection and classification of voice pathology. Three challenges-and areas of opportunities-for advancing this research are heterogeneity of databases, lack of clinical validation studies, and inconsistent reporting.
Collapse
Affiliation(s)
- George S Liu
- Department of Otolaryngology-Head and Neck Surgery, Stanford University, Stanford, California, USA
| | - Nedeljko Jovanovic
- Rehabilitation Sciences-Voice Production and Perception Laboratory, Western University, London, Ontario, Canada
| | - C Kwang Sung
- Department of Otolaryngology-Head and Neck Surgery, Stanford University, Stanford, California, USA
| | - Philip C Doyle
- Department of Otolaryngology-Head and Neck Surgery, Stanford University, Stanford, California, USA
| |
Collapse
|
9
|
Danilatou V, Dimopoulos D, Kostoulas T, Douketis J. Machine Learning-Based Predictive Models for Patients with Venous Thromboembolism: A Systematic Review. Thromb Haemost 2024. [PMID: 38574756 DOI: 10.1055/a-2299-4758] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/06/2024]
Abstract
BACKGROUND Venous thromboembolism (VTE) is a chronic disorder with a significant health and economic burden. Several VTE-specific clinical prediction models (CPMs) have been used to assist physicians in decision-making but have several limitations. This systematic review explores if machine learning (ML) can enhance CPMs by analyzing extensive patient data derived from electronic health records. We aimed to explore ML-CPMs' applications in VTE for risk stratification, outcome prediction, diagnosis, and treatment. METHODS Three databases were searched: PubMed, Google Scholar, and IEEE electronic library. Inclusion criteria focused on studies using structured data, excluding non-English publications, studies on non-humans, and certain data types such as natural language processing and image processing. Studies involving pregnant women, cancer patients, and children were also excluded. After excluding irrelevant studies, a total of 77 studies were included. RESULTS Most studies report that ML-CPMs outperformed traditional CPMs in terms of receiver operating area under the curve in the four clinical domains that were explored. However, the majority of the studies were retrospective, monocentric, and lacked detailed model architecture description and external validation, which are essential for quality audit. This review identified research gaps and highlighted challenges related to standardized reporting, reproducibility, and model comparison. CONCLUSION ML-CPMs show promise in improving risk assessment and individualized treatment recommendations in VTE. Apparently, there is an urgent need for standardized reporting and methodology for ML models, external validation, prospective and real-world data studies, as well as interventional studies to evaluate the impact of artificial intelligence in VTE.
Collapse
Affiliation(s)
- Vasiliki Danilatou
- School of Medicine, European University of Cyprus, Nicosia, Cyprus
- Healthcare Division, Sphynx Technology Solutions, Nicosia, Cyprus
| | - Dimitrios Dimopoulos
- School of Engineering, Department of Information and Communication Systems Engineering, University of the Aegean, North Aegean, Greece
| | - Theodoros Kostoulas
- School of Engineering, Department of Information and Communication Systems Engineering, University of the Aegean, North Aegean, Greece
| | - James Douketis
- Department of Medicine, McMaster University, Hamilton, Canada
- Department of Medicine, St. Joseph's Healthcare Hamilton, Ontario, Canada
| |
Collapse
|
10
|
van Genderen ME, van de Sande D, Hooft L, Reis AA, Cornet AD, Oosterhoff JHF, van der Ster BJP, Huiskens J, Townsend R, van Bommel J, Gommers D, van den Hoven J. Charting a new course in healthcare: early-stage AI algorithm registration to enhance trust and transparency. NPJ Digit Med 2024; 7:119. [PMID: 38720011 PMCID: PMC11078921 DOI: 10.1038/s41746-024-01104-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2023] [Accepted: 04/15/2024] [Indexed: 05/12/2024] Open
Affiliation(s)
- Michel E van Genderen
- Erasmus MC University Medical Center, Department of Adult Intensive Care, Rotterdam, The Netherlands.
| | - Davy van de Sande
- Erasmus MC University Medical Center, Department of Adult Intensive Care, Rotterdam, The Netherlands
| | - Lotty Hooft
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Andreas Alois Reis
- Department of Research for Health, Division of the Chief Scientist, World Health Organization, Geneva, Switzerland
| | - Alexander D Cornet
- Section editor Intensive Care at Nederlands Tijdschrift voor Geneeskunde (Dutch Journal of Medicine), Amsterdam, The Netherlands
- Department of Intensive Care, Medisch Spectrum Twente, Enschede, The Netherlands
| | - Jacobien H F Oosterhoff
- Delft University of Technology, Faculty of Technology, Policy and Management, Delft, The Netherlands
| | - Björn J P van der Ster
- Erasmus MC University Medical Center, Department of Adult Intensive Care, Rotterdam, The Netherlands
| | | | - Reggie Townsend
- Vice President Data Ethics Practice, SAS Worldwide Headquarters, Cary, N.C., USA
- National Artificial Intelligence Advisory Committee, Executive Office of the President of the United States, Washington, D.C., USA
| | - Jasper van Bommel
- Erasmus MC University Medical Center, Department of Adult Intensive Care, Rotterdam, The Netherlands
| | - Diederik Gommers
- Erasmus MC University Medical Center, Department of Adult Intensive Care, Rotterdam, The Netherlands
| | - Jeroen van den Hoven
- Delft University of Technology, Faculty of Technology, Policy and Management, Delft, The Netherlands
| |
Collapse
|
11
|
Meinikheim M, Mendel R, Palm C, Probst A, Muzalyova A, Scheppach MW, Nagl S, Schnoy E, Römmele C, Schulz DAH, Schlottmann J, Prinz F, Rauber D, Rückert T, Matsumura T, Fernández-Esparrach G, Parsa N, Byrne MF, Messmann H, Ebigbo A. Influence of artificial intelligence on the diagnostic performance of endoscopists in the assessment of Barrett's esophagus: a tandem randomized and video trial. Endoscopy 2024. [PMID: 38547927 DOI: 10.1055/a-2296-5696] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 05/04/2024]
Abstract
BACKGROUND This study evaluated the effect of an artificial intelligence (AI)-based clinical decision support system on the performance and diagnostic confidence of endoscopists in their assessment of Barrett's esophagus (BE). METHODS 96 standardized endoscopy videos were assessed by 22 endoscopists with varying degrees of BE experience from 12 centers. Assessment was randomized into two video sets: group A (review first without AI and second with AI) and group B (review first with AI and second without AI). Endoscopists were required to evaluate each video for the presence of Barrett's esophagus-related neoplasia (BERN) and then decide on a spot for a targeted biopsy. After the second assessment, they were allowed to change their clinical decision and confidence level. RESULTS AI had a stand-alone sensitivity, specificity, and accuracy of 92.2%, 68.9%, and 81.3%, respectively. Without AI, BE experts had an overall sensitivity, specificity, and accuracy of 83.3%, 58.1%, and 71.5%, respectively. With AI, BE nonexperts showed a significant improvement in sensitivity and specificity when videos were assessed a second time with AI (sensitivity 69.8% [95%CI 65.2%-74.2%] to 78.0% [95%CI 74.0%-82.0%]; specificity 67.3% [95%CI 62.5%-72.2%] to 72.7% [95%CI 68.2%-77.3%]). In addition, the diagnostic confidence of BE nonexperts improved significantly with AI. CONCLUSION BE nonexperts benefitted significantly from additional AI. BE experts and nonexperts remained significantly below the stand-alone performance of AI, suggesting that there may be other factors influencing endoscopists' decisions to follow or discard AI advice.
Collapse
Affiliation(s)
- Michael Meinikheim
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - Robert Mendel
- Regensburg Medical Image Computing, Ostbayerische Technische Hochschule Regensburg, Regensburg, Germany
| | - Christoph Palm
- Regensburg Medical Image Computing, Ostbayerische Technische Hochschule Regensburg, Regensburg, Germany
| | - Andreas Probst
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - Anna Muzalyova
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - Markus W Scheppach
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - Sandra Nagl
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - Elisabeth Schnoy
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - Christoph Römmele
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - Dominik A H Schulz
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - Jakob Schlottmann
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - Friederike Prinz
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - David Rauber
- Regensburg Medical Image Computing, Ostbayerische Technische Hochschule Regensburg, Regensburg, Germany
| | - Tobias Rückert
- Regensburg Medical Image Computing, Ostbayerische Technische Hochschule Regensburg, Regensburg, Germany
| | - Tomoaki Matsumura
- Department of Gastroenterology, Chiba University Graduate School of Medicine, Chiba, Japan
| | - Glòria Fernández-Esparrach
- Endoscopy Unit, Gastroenterology Department, ICMDM, Hospital Clínic de Barcelona, Barcelona, Spain
- Faculty of Medicine, University of Barcelona, Barcelona, Spain
- Institut d'Investigacions Biomèdiques August Pi i Sunyer (IDIBAPS), Barcelona, Spain
- Centro de Investigación Biomédica en Red de Enfermedades Hepáticas y Digestivas (CIBEREHD), Barcelona, Spain
| | - Nasim Parsa
- Division of Gastroenterology and Hepatology, Mayo Clinic, Scottsdale, United States
- Satisfai Health, Vancouver, Canada
| | - Michael F Byrne
- Satisfai Health, Vancouver, Canada
- Gastroenterology, Vancouver General Hospital, The University of British Columbia, Vancouver, Canada
| | - Helmut Messmann
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| | - Alanna Ebigbo
- Department of Gastroenterology, University Hospital Augsburg, Augsburg, Germany
| |
Collapse
|
12
|
Bowness JS, Liu X, Keane PA. Leading in the development, standardised evaluation, and adoption of artificial intelligence in clinical practice: regional anaesthesia as an example. Br J Anaesth 2024; 132:1016-1021. [PMID: 38302346 DOI: 10.1016/j.bja.2023.12.024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2023] [Revised: 12/19/2023] [Accepted: 12/27/2023] [Indexed: 02/03/2024] Open
Abstract
A recent study by Suissa and colleagues explored the clinical relevance of a medical image segmentation metric (Dice metric) commonly used in the field of artificial intelligence (AI). They showed that pixel-wise agreement for physician identification of structures on ultrasound images is variable, and a relatively low Dice metric (0.34) correlated to a substantial agreement on subjective clinical assessment. We highlight the need to bring structure and clinical perspective to the evaluation of medical AI, which clinicians are best placed to direct.
Collapse
Affiliation(s)
- James S Bowness
- Nuffield Department of Clinical Anaesthesia, University of Oxford, Oxford, UK; Department of Anaesthesia, Aneurin Bevan University Health Board, Newport, UK.
| | - Xiaoxuan Liu
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK; College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
| | - Pearse A Keane
- Institute of Ophthalmology, Faculty of Brain Sciences, University College London, London, UK; NIHR Biomedical Research Centre, Moorfields Eye Hospital NHS Foundation Trust, London, UK
| |
Collapse
|
13
|
Gao W, Liu J, Shtylla B, Venkatakrishnan K, Yin D, Shah M, Nicholas T, Cao Y. Realizing the promise of Project Optimus: Challenges and emerging opportunities for dose optimization in oncology drug development. CPT Pharmacometrics Syst Pharmacol 2024; 13:691-709. [PMID: 37969061 PMCID: PMC11098159 DOI: 10.1002/psp4.13079] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2023] [Revised: 10/20/2023] [Accepted: 10/30/2023] [Indexed: 11/17/2023] Open
Abstract
Project Optimus is a US Food and Drug Administration Oncology Center of Excellence initiative aimed at reforming the dose selection and optimization paradigm in oncology drug development. This project seeks to bring together pharmaceutical companies, international regulatory agencies, academic institutions, patient advocates, and other stakeholders. Although there is much promise in this initiative, there are several challenges that need to be addressed, including multidimensionality of the dose optimization problem in oncology, the heterogeneity of cancer and patients, importance of evaluating long-term tolerability beyond dose-limiting toxicities, and the lack of reliable biomarkers for long-term efficacy. Through the lens of Totality of Evidence and with the mindset of model-informed drug development, we offer insights into dose optimization by building a quantitative knowledge base integrating diverse sources of data and leveraging quantitative modeling tools to build evidence for drug dosage considering exposure, disease biology, efficacy, toxicity, and patient factors. We believe that rational dose optimization can be achieved in oncology drug development, improving patient outcomes by maximizing therapeutic benefit while minimizing toxicity.
Collapse
Affiliation(s)
- Wei Gao
- Quantitative PharmacologyEMD Serono Research & Development Institute, Inc.BillericaMassachusettsUSA
| | - Jiang Liu
- Food and Drug AdministrationSilver SpringMarylandUSA
| | - Blerta Shtylla
- Quantitative Systems PharmacologyPfizerSan DiegoCaliforniaUSA
| | - Karthik Venkatakrishnan
- Quantitative PharmacologyEMD Serono Research & Development Institute, Inc.BillericaMassachusettsUSA
| | - Donghua Yin
- Clinical PharmacologyPfizerSan DiegoCaliforniaUSA
| | - Mirat Shah
- Food and Drug AdministrationSilver SpringMarylandUSA
| | | | - Yanguang Cao
- Division of Pharmacotherapy and Experimental Therapeutics, Eshelman School of PharmacyUniversity of North Carolina at Chapel HillChapel HillNorth CarolinaUSA
| |
Collapse
|
14
|
Catling FJR, Nagendran M, Festor P, Bien Z, Harris S, Faisal AA, Gordon AC, Komorowski M. Can Machine Learning Personalize Cardiovascular Therapy in Sepsis? Crit Care Explor 2024; 6:e1087. [PMID: 38709088 DOI: 10.1097/cce.0000000000001087] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/07/2024] Open
Abstract
Large randomized trials in sepsis have generally failed to find effective novel treatments. This is increasingly attributed to patient heterogeneity, including heterogeneous cardiovascular changes in septic shock. We discuss the potential for machine learning systems to personalize cardiovascular resuscitation in sepsis. While the literature is replete with proofs of concept, the technological readiness of current systems is low, with a paucity of clinical trials and proven patient benefit. Systems may be vulnerable to confounding and poor generalization to new patient populations or contemporary patterns of care. Typical electronic health records do not capture rich enough data, at sufficient temporal resolution, to produce systems that make actionable treatment suggestions. To resolve these issues, we recommend a simultaneous focus on technical challenges and removing barriers to translation. This will involve improving data quality, adopting causally grounded models, prioritizing safety assessment and integration into healthcare workflows, conducting randomized clinical trials and aligning with regulatory requirements.
Collapse
Affiliation(s)
- Finneas J R Catling
- Institute of Healthcare Engineering, University College London, London, United Kingdom
- Division of Anaesthetics, Pain Medicine, and Intensive Care, Imperial College London, London, United Kingdom
| | - Myura Nagendran
- Division of Anaesthetics, Pain Medicine, and Intensive Care, Imperial College London, London, United Kingdom
- UKRI Centre for Doctoral Training in AI for Healthcare, Imperial College London, London, United Kingdom
| | - Paul Festor
- UKRI Centre for Doctoral Training in AI for Healthcare, Imperial College London, London, United Kingdom
- Department of Computing, Imperial College London, London, United Kingdom
| | - Zuzanna Bien
- School of Life Course & Population Sciences, King's College London, United Kingdom
| | - Steve Harris
- Department of Critical Care, University College London Hospital, London, United Kingdom
- Institute of Health Informatics, University College London, London, United Kingdom
| | - A Aldo Faisal
- UKRI Centre for Doctoral Training in AI for Healthcare, Imperial College London, London, United Kingdom
- Department of Computing, Imperial College London, London, United Kingdom
- Institute of Artificial and Human Intelligence, Universität Bayreuth, Bayreuth, Germany
- Department of Bioengineering, Imperial College London, London, United Kingdom
| | - Anthony C Gordon
- Division of Anaesthetics, Pain Medicine, and Intensive Care, Imperial College London, London, United Kingdom
| | - Matthieu Komorowski
- Division of Anaesthetics, Pain Medicine, and Intensive Care, Imperial College London, London, United Kingdom
| |
Collapse
|
15
|
Denniston AK, Liu X. Responsible and evidence-based AI: 5 years on. Lancet Digit Health 2024; 6:e305-e307. [PMID: 38670738 DOI: 10.1016/s2589-7500(24)00071-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2024] [Accepted: 04/04/2024] [Indexed: 04/28/2024]
Affiliation(s)
- Alastair K Denniston
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham B15 2WB, UK; University of Birmingham, Birmingham, UK; University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Xiaoxuan Liu
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham B15 2WB, UK; University of Birmingham, Birmingham, UK; University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.
| |
Collapse
|
16
|
Bowness JS, Metcalfe D, El-Boghdadly K, Thurley N, Morecroft M, Hartley T, Krawczyk J, Noble JA, Higham H. Artificial intelligence for ultrasound scanning in regional anaesthesia: a scoping review of the evidence from multiple disciplines. Br J Anaesth 2024; 132:1049-1062. [PMID: 38448269 PMCID: PMC11103083 DOI: 10.1016/j.bja.2024.01.036] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Revised: 01/09/2024] [Accepted: 01/24/2024] [Indexed: 03/08/2024] Open
Abstract
BACKGROUND Artificial intelligence (AI) for ultrasound scanning in regional anaesthesia is a rapidly developing interdisciplinary field. There is a risk that work could be undertaken in parallel by different elements of the community but with a lack of knowledge transfer between disciplines, leading to repetition and diverging methodologies. This scoping review aimed to identify and map the available literature on the accuracy and utility of AI systems for ultrasound scanning in regional anaesthesia. METHODS A literature search was conducted using Medline, Embase, CINAHL, IEEE Xplore, and ACM Digital Library. Clinical trial registries, a registry of doctoral theses, regulatory authority databases, and websites of learned societies in the field were searched. Online commercial sources were also reviewed. RESULTS In total, 13,014 sources were identified; 116 were included for full-text review. A marked change in AI techniques was noted in 2016-17, from which point on the predominant technique used was deep learning. Methods of evaluating accuracy are variable, meaning it is impossible to compare the performance of one model with another. Evaluations of utility are more comparable, but predominantly gained from the simulation setting with limited clinical data on efficacy or safety. Study methodology and reporting lack standardisation. CONCLUSIONS There is a lack of structure to the evaluation of accuracy and utility of AI for ultrasound scanning in regional anaesthesia, which hinders rigorous appraisal and clinical uptake. A framework for consistent evaluation is needed to inform model evaluation, allow comparison between approaches/models, and facilitate appropriate clinical adoption.
Collapse
Affiliation(s)
- James S Bowness
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK; Department of Anaesthesia, Aneurin Bevan University Health Board, Newport, UK.
| | - David Metcalfe
- Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK; Emergency Medicine Research in Oxford (EMROx), Oxford University Hospitals NHS Foundation Trust, Oxford, UK. https://twitter.com/@TraumaDataDoc
| | - Kariem El-Boghdadly
- Department of Anaesthesia and Peri-operative Medicine, Guy's & St Thomas's NHS Foundation Trust, London, UK; Centre for Human and Applied Physiological Sciences, King's College London, London, UK. https://twitter.com/@elboghdadly
| | - Neal Thurley
- Bodleian Health Care Libraries, University of Oxford, Oxford, UK
| | - Megan Morecroft
- Faculty of Medicine, Health & Life Sciences, University of Swansea, Swansea, UK
| | - Thomas Hartley
- Intelligent Ultrasound, Cardiff, UK. https://twitter.com/@tomhartley84
| | - Joanna Krawczyk
- Department of Anaesthesia, Aneurin Bevan University Health Board, Newport, UK
| | - J Alison Noble
- Institute of Biomedical Engineering, University of Oxford, Oxford, UK. https://twitter.com/@AlisonNoble_OU
| | - Helen Higham
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK; Nuffield Department of Anaesthesia, Oxford University Hospitals NHS Foundation Trust, Oxford, UK. https://twitter.com/@HelenEHigham
| |
Collapse
|
17
|
Barea Mendoza JA, Valiente Fernandez M, Pardo Fernandez A, Gómez Álvarez J. Current perspectives on the use of artificial intelligence in critical patient safety. Med Intensiva 2024:S2173-5727(24)00080-8. [PMID: 38677902 DOI: 10.1016/j.medine.2024.04.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2023] [Accepted: 03/11/2024] [Indexed: 04/29/2024]
Abstract
Intensive Care Units (ICUs) have undergone enhancements in patient safety, and artificial intelligence (AI) emerges as a disruptive technology offering novel opportunities. While the published evidence is limited and presents methodological issues, certain areas show promise, such as decision support systems, detection of adverse events, and prescription error identification. The application of AI in safety may pursue predictive or diagnostic objectives. Implementing AI-based systems necessitates procedures to ensure secure assistance, addressing challenges including trust in such systems, biases, data quality, scalability, and ethical and confidentiality considerations. The development and application of AI demand thorough testing, encompassing retrospective data assessments, real-time validation with prospective cohorts, and efficacy demonstration in clinical trials. Algorithmic transparency and explainability are essential, with active involvement of clinical professionals being crucial in the implementation process.
Collapse
Affiliation(s)
- Jesús Abelardo Barea Mendoza
- UCI de Trauma y Emergencias. Servicio de Medicina Intensiva. Hospital Universitario 12 de Octubre. Instituto de Investigación Hospital 12 de Octubre, Spain.
| | - Marcos Valiente Fernandez
- UCI de Trauma y Emergencias. Servicio de Medicina Intensiva. Hospital Universitario 12 de Octubre. Instituto de Investigación Hospital 12 de Octubre, Spain
| | | | - Josep Gómez Álvarez
- Hospital Universitari de Tarragona Joan XXIII. Universitat Rovira i Virgili. Institut d'Investigació Sanitària Pere i Virgili, Tarragona, Spain
| |
Collapse
|
18
|
Collins GS, Moons KGM, Dhiman P, Riley RD, Beam AL, Van Calster B, Ghassemi M, Liu X, Reitsma JB, van Smeden M, Boulesteix AL, Camaradou JC, Celi LA, Denaxas S, Denniston AK, Glocker B, Golub RM, Harvey H, Heinze G, Hoffman MM, Kengne AP, Lam E, Lee N, Loder EW, Maier-Hein L, Mateen BA, McCradden MD, Oakden-Rayner L, Ordish J, Parnell R, Rose S, Singh K, Wynants L, Logullo P. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ 2024; 385:e078378. [PMID: 38626948 PMCID: PMC11019967 DOI: 10.1136/bmj-2023-078378] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 01/17/2024] [Indexed: 04/19/2024]
Affiliation(s)
- Gary S Collins
- Centre for Statistics in Medicine, UK EQUATOR Centre, Nuffield Department of Orthopaedics, Rheumatology, and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK
| | - Karel G M Moons
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Paula Dhiman
- Centre for Statistics in Medicine, UK EQUATOR Centre, Nuffield Department of Orthopaedics, Rheumatology, and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK
| | - Richard D Riley
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK
| | - Andrew L Beam
- Department of Epidemiology, Harvard T H Chan School of Public Health, Boston, MA, USA
| | - Ben Van Calster
- Department of Development and Regeneration, KU Leuven, Leuven, Belgium
- Department of Biomedical Data Science, Leiden University Medical Centre, Leiden, Netherlands
| | - Marzyeh Ghassemi
- Department of Electrical Engineering and Computer Science, Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - Xiaoxuan Liu
- Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | - Johannes B Reitsma
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Maarten van Smeden
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Anne-Laure Boulesteix
- Department of Medical Information Processing, Biometry and Epidemiology, Ludwig-Maximilians-University of Munich, Munich, Germany
| | - Jennifer Catherine Camaradou
- Patient representative, Health Data Research UK patient and public involvement and engagement group
- Patient representative, University of East Anglia, Faculty of Health Sciences, Norwich Research Park, Norwich, UK
| | - Leo Anthony Celi
- Beth Israel Deaconess Medical Center, Boston, MA, USA
- Laboratory for Computational Physiology, Massachusetts Institute of Technology, Cambridge, MA, USA
- Department of Biostatistics, Harvard T H Chan School of Public Health, Boston, MA, USA
| | - Spiros Denaxas
- Institute of Health Informatics, University College London, London, UK
- British Heart Foundation Data Science Centre, London, UK
| | - Alastair K Denniston
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, Birmingham, UK
- Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
| | - Ben Glocker
- Department of Computing, Imperial College London, London, UK
| | - Robert M Golub
- Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | | | - Georg Heinze
- Section for Clinical Biometrics, Centre for Medical Data Science, Medical University of Vienna, Vienna, Austria
| | - Michael M Hoffman
- Princess Margaret Cancer Centre, University Health Network, Toronto, ON, Canada
- Department of Medical Biophysics, University of Toronto, Toronto, ON, Canada
- Department of Computer Science, University of Toronto, Toronto, ON, Canada
- Vector Institute for Artificial Intelligence, Toronto, ON, Canada
| | | | - Emily Lam
- Patient representative, Health Data Research UK patient and public involvement and engagement group
| | - Naomi Lee
- National Institute for Health and Care Excellence, London, UK
| | - Elizabeth W Loder
- The BMJ, London, UK
- Department of Neurology, Brigham and Women's Hospital, Harvard Medical School, Boston, MA, USA
| | - Lena Maier-Hein
- Department of Intelligent Medical Systems, German Cancer Research Centre, Heidelberg, Germany
| | - Bilal A Mateen
- Institute of Health Informatics, University College London, London, UK
- Wellcome Trust, London, UK
- Alan Turing Institute, London, UK
| | - Melissa D McCradden
- Department of Bioethics, Hospital for Sick Children Toronto, ON, Canada
- Genetics and Genome Biology, SickKids Research Institute, Toronto, ON, Canada
| | - Lauren Oakden-Rayner
- Australian Institute for Machine Learning, University of Adelaide, Adelaide, SA, Australia
| | - Johan Ordish
- Medicines and Healthcare products Regulatory Agency, London, UK
| | - Richard Parnell
- Patient representative, Health Data Research UK patient and public involvement and engagement group
| | - Sherri Rose
- Department of Health Policy and Center for Health Policy, Stanford University, Stanford, CA, USA
| | - Karandeep Singh
- Department of Epidemiology, CAPHRI Care and Public Health Research Institute, Maastricht University, Maastricht, Netherlands
| | - Laure Wynants
- Department of Epidemiology, CAPHRI Care and Public Health Research Institute, Maastricht University, Maastricht, Netherlands
| | - Patricia Logullo
- Centre for Statistics in Medicine, UK EQUATOR Centre, Nuffield Department of Orthopaedics, Rheumatology, and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK
| |
Collapse
|
19
|
Kolbinger FR, Veldhuizen GP, Zhu J, Truhn D, Kather JN. Reporting guidelines in medical artificial intelligence: a systematic review and meta-analysis. COMMUNICATIONS MEDICINE 2024; 4:71. [PMID: 38605106 PMCID: PMC11009315 DOI: 10.1038/s43856-024-00492-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2023] [Accepted: 03/27/2024] [Indexed: 04/13/2024] Open
Abstract
BACKGROUND The field of Artificial Intelligence (AI) holds transformative potential in medicine. However, the lack of universal reporting guidelines poses challenges in ensuring the validity and reproducibility of published research studies in this field. METHODS Based on a systematic review of academic publications and reporting standards demanded by both international consortia and regulatory stakeholders as well as leading journals in the fields of medicine and medical informatics, 26 reporting guidelines published between 2009 and 2023 were included in this analysis. Guidelines were stratified by breadth (general or specific to medical fields), underlying consensus quality, and target research phase (preclinical, translational, clinical) and subsequently analyzed regarding the overlap and variations in guideline items. RESULTS AI reporting guidelines for medical research vary with respect to the quality of the underlying consensus process, breadth, and target research phase. Some guideline items such as reporting of study design and model performance recur across guidelines, whereas other items are specific to particular fields and research stages. CONCLUSIONS Our analysis highlights the importance of reporting guidelines in clinical AI research and underscores the need for common standards that address the identified variations and gaps in current guidelines. Overall, this comprehensive overview could help researchers and public stakeholders reinforce quality standards for increased reliability, reproducibility, clinical validity, and public trust in AI research in healthcare. This could facilitate the safe, effective, and ethical translation of AI methods into clinical applications that will ultimately improve patient outcomes.
Collapse
Grants
- UM1 TR004402 NCATS NIH HHS
- JNK is supported by the German Federal Ministry of Health (DEEP LIVER, ZMVI1-2520DAT111) and the Max-Eder-Programme of the German Cancer Aid (grant #70113864), the German Federal Ministry of Education and Research (PEARL, 01KD2104C; CAMINO, 01EO2101; SWAG, 01KD2215A; TRANSFORM LIVER, 031L0312A), the German Academic Exchange Service (SECAI, 57616814), the German Federal Joint Committee (Transplant.KI, 01VSF21048) the European Union (ODELIA, 101057091; GENIAL, 101096312) and the National Institute for Health and Care Research (NIHR, NIHR213331) Leeds Biomedical Research Centre.
Collapse
Affiliation(s)
- Fiona R Kolbinger
- Else Kroener Fresenius Center for Digital Health, TUD Dresden University of Technology, Dresden, Germany
- Department of Visceral, Thoracic and Vascular Surgery, University Hospital and Faculty of Medicine Carl Gustav Carus, TUD Dresden University of Technology, Dresden, Germany
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, IN, USA
- Regenstrief Center for Healthcare Engineering, Purdue University, West Lafayette, IN, USA
- Department of Biostatistics and Health Data Science, Richard M. Fairbanks School of Public Health, Indiana University, Indianapolis, IN, USA
- Indiana University Simon Comprehensive Cancer Center, Indiana University School of Medicine, Indianapolis, IN, USA
| | - Gregory P Veldhuizen
- Else Kroener Fresenius Center for Digital Health, TUD Dresden University of Technology, Dresden, Germany
| | - Jiefu Zhu
- Else Kroener Fresenius Center for Digital Health, TUD Dresden University of Technology, Dresden, Germany
| | - Daniel Truhn
- Department of Diagnostic and Interventional Radiology, University Hospital Aachen, Aachen, Germany
| | - Jakob Nikolas Kather
- Else Kroener Fresenius Center for Digital Health, TUD Dresden University of Technology, Dresden, Germany.
- Department of Medicine III, University Hospital RWTH Aachen, Aachen, Germany.
- Department of Medicine I, University Hospital Dresden, Dresden, Germany.
- Medical Oncology, National Center for Tumor Diseases (NCT), University Hospital Heidelberg, Heidelberg, Germany.
| |
Collapse
|
20
|
Pinsky MR, Bedoya A, Bihorac A, Celi L, Churpek M, Economou-Zavlanos NJ, Elbers P, Saria S, Liu V, Lyons PG, Shickel B, Toral P, Tscholl D, Clermont G. Use of artificial intelligence in critical care: opportunities and obstacles. Crit Care 2024; 28:113. [PMID: 38589940 PMCID: PMC11000355 DOI: 10.1186/s13054-024-04860-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2024] [Accepted: 03/05/2024] [Indexed: 04/10/2024] Open
Abstract
BACKGROUND Perhaps nowhere else in the healthcare system than in the intensive care unit environment are the challenges to create useful models with direct time-critical clinical applications more relevant and the obstacles to achieving those goals more massive. Machine learning-based artificial intelligence (AI) techniques to define states and predict future events are commonplace activities of modern life. However, their penetration into acute care medicine has been slow, stuttering and uneven. Major obstacles to widespread effective application of AI approaches to the real-time care of the critically ill patient exist and need to be addressed. MAIN BODY Clinical decision support systems (CDSSs) in acute and critical care environments support clinicians, not replace them at the bedside. As will be discussed in this review, the reasons are many and include the immaturity of AI-based systems to have situational awareness, the fundamental bias in many large databases that do not reflect the target population of patient being treated making fairness an important issue to address and technical barriers to the timely access to valid data and its display in a fashion useful for clinical workflow. The inherent "black-box" nature of many predictive algorithms and CDSS makes trustworthiness and acceptance by the medical community difficult. Logistically, collating and curating in real-time multidimensional data streams of various sources needed to inform the algorithms and ultimately display relevant clinical decisions support format that adapt to individual patient responses and signatures represent the efferent limb of these systems and is often ignored during initial validation efforts. Similarly, legal and commercial barriers to the access to many existing clinical databases limit studies to address fairness and generalizability of predictive models and management tools. CONCLUSIONS AI-based CDSS are evolving and are here to stay. It is our obligation to be good shepherds of their use and further development.
Collapse
Affiliation(s)
- Michael R Pinsky
- Department of Critical Care Medicine, School of Medicine, University of Pittsburgh, 638 Scaife Hall, 3550 Terrace Street, Pittsburgh, PA, 15261, USA.
| | - Armando Bedoya
- Algorithm-Based Clinical Decision Support (ABCDS) Oversight, Office of Vice Dean of Data Science, School of Medicine, Duke University, Durham, NC, 27705, USA
- Division of Pulmonary Critical Care Medicine, Duke University School of Medicine, Durham, NC, 27713, USA
| | - Azra Bihorac
- Department of Medicine, University of Florida College of Medicine Gainesville, Malachowsky Hall, 1889 Museum Road, Suite 2410, Gainesville, FL, 32611, USA
| | - Leo Celi
- Laboratory for Computational Physiology, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA
| | - Matthew Churpek
- Department of Medicine, University of Wisconsin, 600 Highland Ave, Madison, WI, 53792, USA
| | - Nicoleta J Economou-Zavlanos
- Algorithm-Based Clinical Decision Support (ABCDS) Oversight, Office of Vice Dean of Data Science, School of Medicine, Duke University, Durham, NC, 27705, USA
| | - Paul Elbers
- Department of Intensive Care, Amsterdam UMC, Amsterdam, NL, USA
- Amsterdam UMC, ZH.7D.167, De Boelelaan 1117, 1081 HV, Amsterdam, The Netherlands
| | - Suchi Saria
- Department of Computer Science, Whiting School of Engineering, Johns Hopkins Medical Institutions, Johns Hopkins University, 333 Malone Hall, 300 Wolfe Street, Baltimore, MD, USA
- Department of Medicine, Johns Hopkins School of Medicine, AI and Health Lab, Johns Hopkins University, Baltimore, MD, USA
- Bayesian Health, New york, NY, 10282, USA
| | - Vincent Liu
- Department of Medicine, Oregon Health & Science University, 3181 S.W. Sam Jackson Park Road, Mail Code UHN67, Portland, OR, 97239-3098, USA
- , 2000 Broadway, Oakland, CA, 94612, USA
| | - Patrick G Lyons
- Department of Medicine, Oregon Health & Science University, 3181 S.W. Sam Jackson Park Road, Mail Code UHN67, Portland, OR, 97239-3098, USA
| | - Benjamin Shickel
- Department of Medicine, University of Florida College of Medicine Gainesville, Malachowsky Hall, 1889 Museum Road, Suite 2410, Gainesville, FL, 32611, USA
- Amsterdam UMC, ZH.7D.167, De Boelelaan 1117, 1081 HV, Amsterdam, The Netherlands
| | - Patrick Toral
- Department of Intensive Care, Amsterdam UMC, Amsterdam, NL, USA
- Amsterdam UMC, ZH.7D.165, De Boelelaan 1117, 1081 HV, Amsterdam, The Netherlands
| | - David Tscholl
- Institute of Anesthesiology, University Hospital Zurich, University of Zurich, Frauenklinikstrasse 10, 8091, Zurich, Switzerland
| | - Gilles Clermont
- Department of Critical Care Medicine, School of Medicine, University of Pittsburgh, 638 Scaife Hall, 3550 Terrace Street, Pittsburgh, PA, 15261, USA
- VA Pittsburgh Health System, 131A Building 30, 4100 Allequippa St, Pittsburgh, PA, 15240, USA
| |
Collapse
|
21
|
Caratsch L, Lechtenboehmer C, Caorsi M, Oung K, Zanchi F, Aleman Y, Walker UA, Omoumi P, Hügle T. Detection and Grading of Radiographic Hand Osteoarthritis Using an Automated Machine Learning Platform. ACR Open Rheumatol 2024. [PMID: 38576187 DOI: 10.1002/acr2.11665] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2023] [Revised: 01/31/2024] [Accepted: 02/09/2024] [Indexed: 04/06/2024] Open
Abstract
OBJECTIVE Automated machine learning (autoML) platforms allow health care professionals to play an active role in the development of machine learning (ML) algorithms according to scientific or clinical needs. The aim of this study was to develop and evaluate such a model for automated detection and grading of distal hand osteoarthritis (OA). METHODS A total of 13,690 hand radiographs from 2,863 patients within the Swiss Cohort of Quality Management (SCQM) and an external control data set of 346 non-SCQM patients were collected and scored for distal interphalangeal OA (DIP-OA) using the modified Kellgren/Lawrence (K/L) score. Giotto (Learn to Forecast [L2F]) was used as an autoML platform for training two convolutional neural networks for DIP joint extraction and subsequent classification according to the K/L scores. A total of 48,892 DIP joints were extracted and then used to train the classification model. Heatmaps were generated independently of the platform. User experience of a web application as a provisional user interface was investigated by rheumatologists and radiologists. RESULTS The sensitivity and specificity of this model for detecting DIP-OA were 79% and 86%, respectively. The accuracy for grading the correct K/L score was 75%, with a κ score of 0.76. The accuracy per DIP-OA class differed, with 86% for no OA (defined as K/L scores 0 and 1), 71% for a K/L score of 2, 46% for a K/L score of 3, and 67% for a K/L score of 4. Similar values were obtained in an independent external test set. Qualitative and quantitative user experience testing of the web application revealed a moderate to high demand for automated DIP-OA scoring among rheumatologists. Conversely, radiologists expressed a low demand, except for the use of heatmaps. CONCLUSION AutoML platforms are an opportunity to develop clinical end-to-end ML algorithms. Here, automated radiographic DIP-OA detection is both feasible and usable, whereas grading among individual K/L scores (eg, for clinical trials) remains challenging.
Collapse
Affiliation(s)
- Leo Caratsch
- Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
- City Hospital Waid, Zurich, Switzerland
- L2F (Learn to Forecast), Lausanne, Switzerland
| | - Christian Lechtenboehmer
- Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
- City Hospital Waid, Zurich, Switzerland
- L2F (Learn to Forecast), Lausanne, Switzerland
- University Hospital of Basel, Basel, Switzerland
| | | | - Karine Oung
- Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | - Fabio Zanchi
- Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | - Yasser Aleman
- Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | | | - Patrick Omoumi
- Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| | - Thomas Hügle
- Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
| |
Collapse
|
22
|
Nasser L, McLeod SL, Hall JN. Evaluating the Reliability of a Remote Acuity Prediction Tool in a Canadian Academic Emergency Department. Ann Emerg Med 2024; 83:373-379. [PMID: 38180398 DOI: 10.1016/j.annemergmed.2023.11.018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2023] [Revised: 11/04/2023] [Accepted: 11/17/2023] [Indexed: 01/06/2024]
Abstract
STUDY OBJECTIVE There is increasing interest in harnessing artificial intelligence to virtually triage patients seeking care. The objective was to examine the reliability of a virtual machine learning algorithm to remotely predict acuity scores for patients seeking emergency department (ED) care by applying the algorithm to retrospective ED data. METHODS This was a retrospective review of adult patients conducted at an academic tertiary care ED (annual census 65,000) from January 2021 to August 2022. Data including ED visit date and time, patient age, sex, reason for visit, presenting complaint and patient-reported pain score were used by the machine learning algorithm to predict acuity scores. The algorithm was designed to up-triage high-risk complaints to promote safety for remote use. The predicted scores were then compared to nurse-led triage scores previously derived in real time using the electronic Canadian Triage and Acuity Scale (eCTAS), an electronic triage decision-support tool used in the ED. Interrater reliability was estimated using kappa statistics with 95% confidence intervals (CIs). RESULTS In total, 21,469 unique ED patient encounters were included. Exact modal agreement was achieved for 10,396 (48.4%) patient encounters. Interrater reliability ranged from poor to fair, as estimated using unweighted kappa (0.18, 95% CI 0.17 to 0.19), linear-weighted kappa (0.25, 95% CI 0.24 to 0.26), and quadratic-weighted kappa (0.36, 95% CI 0.35 to 0.37) statistics. Using the nurse-led eCTAS score as the reference, the machine learning algorithm overtriaged 9,897 (46.1%) and undertriaged 1,176 (5.5%) cases. Some of the presenting complaints under-triaged were conditions generally requiring further probing to delineate their nature, including abnormal lab/imaging results, visual disturbance, and fever. CONCLUSION This machine learning algorithm needs further refinement before being safely implemented for patient use.
Collapse
Affiliation(s)
- Laila Nasser
- Department of Emergency Medicine, Sunnybrook Health Sciences Centre, Department of Medicine, Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada; Department for Continuing Education, Oxford University, Oxford, England.
| | - Shelley L McLeod
- Schwartz/Reisman Emergency Medicine Institute, Sinai Health, Toronto, Ontario, Canada; Division of Emergency Medicine, Department of Family and Community Medicine, Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
| | - Justin N Hall
- Department of Emergency Medicine, Sunnybrook Health Sciences Centre, Department of Medicine, Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
23
|
Hu B, Shi Z, Lu L, Miao Z, Wang H, Zhou Z, Zhang F, Wang R, Luo X, Xu F, Li S, Fang X, Wang X, Yan G, Lv F, Zhang M, Sun Q, Cui G, Liu Y, Zhang S, Pan C, Hou Z, Liang H, Pan Y, Chen X, Li X, Zhou F, Schoepf UJ, Varga-Szemes A, Garrison Moore W, Yu Y, Hu C, Zhang LJ. A deep-learning model for intracranial aneurysm detection on CT angiography images in China: a stepwise, multicentre, early-stage clinical validation study. Lancet Digit Health 2024; 6:e261-e271. [PMID: 38519154 DOI: 10.1016/s2589-7500(23)00268-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2023] [Revised: 10/23/2023] [Accepted: 12/29/2023] [Indexed: 03/24/2024]
Abstract
BACKGROUND Artificial intelligence (AI) models in real-world implementation are scarce. Our study aimed to develop a CT angiography (CTA)-based AI model for intracranial aneurysm detection, assess how it helps clinicians improve diagnostic performance, and validate its application in real-world clinical implementation. METHODS We developed a deep-learning model using 16 546 head and neck CTA examination images from 14 517 patients at eight Chinese hospitals. Using an adapted, stepwise implementation and evaluation, 120 certified clinicians from 15 geographically different hospitals were recruited. Initially, the AI model was externally validated with images of 900 digital subtraction angiography-verified CTA cases (examinations) and compared with the performance of 24 clinicians who each viewed 300 of these cases (stage 1). Next, as a further external validation a multi-reader multi-case study enrolled 48 clinicians to individually review 298 digital subtraction angiography-verified CTA cases (stage 2). The clinicians reviewed each CTA examination twice (ie, with and without the AI model), separated by a 4-week washout period. Then, a randomised open-label comparison study enrolled 48 clinicians to assess the acceptance and performance of this AI model (stage 3). Finally, the model was prospectively deployed and validated in 1562 real-world clinical CTA cases. FINDINGS The AI model in the internal dataset achieved a patient-level diagnostic sensitivity of 0·957 (95% CI 0·939-0·971) and a higher patient-level diagnostic sensitivity than clinicians (0·943 [0·921-0·961] vs 0·658 [0·644-0·672]; p<0·0001) in the external dataset. In the multi-reader multi-case study, the AI-assisted strategy improved clinicians' diagnostic performance both on a per-patient basis (the area under the receiver operating characteristic curves [AUCs]; 0·795 [0·761-0·830] without AI vs 0·878 [0·850-0·906] with AI; p<0·0001) and a per-aneurysm basis (the area under the weighted alternative free-response receiver operating characteristic curves; 0·765 [0·732-0·799] vs 0·865 [0·839-0·891]; p<0·0001). Reading time decreased with the aid of the AI model (87·5 s vs 82·7 s, p<0·0001). In the randomised open-label comparison study, clinicians in the AI-assisted group had a high acceptance of the AI model (92·6% adoption rate), and a higher AUC when compared with the control group (0·858 [95% CI 0·850-0·866] vs 0·789 [0·780-0·799]; p<0·0001). In the prospective study, the AI model had a 0·51% (8/1570) error rate due to poor-quality CTA images and recognition failure. The model had a high negative predictive value of 0·998 (0·994-1·000) and significantly improved the diagnostic performance of clinicians; AUC improved from 0·787 (95% CI 0·766-0·808) to 0·909 (0·894-0·923; p<0·0001) and patient-level sensitivity improved from 0·590 (0·511-0·666) to 0·825 (0·759-0·880; p<0·0001). INTERPRETATION This AI model demonstrated strong clinical potential for intracranial aneurysm detection with improved clinician diagnostic performance, high acceptance, and practical implementation in real-world clinical cases. FUNDING National Natural Science Foundation of China. TRANSLATION For the Chinese translation of the abstract see Supplementary Materials section.
Collapse
Affiliation(s)
- Bin Hu
- Department of Radiology, Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, China
| | - Zhao Shi
- Department of Radiology, Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, China
| | - Li Lu
- Department of Radiology, the Affiliated Hospital of Xuzhou Medical University, Xuzhou, Jiangsu, China
| | - Zhongchang Miao
- Department of Medical Imaging, the First People's Hospital of Lianyungang, Lianyungang, Jiangsu, China
| | - Hao Wang
- Deepwise Artificial Intelligence (AI) Lab, Deepwise, Beijing, China
| | - Zhen Zhou
- Deepwise Artificial Intelligence (AI) Lab, Deepwise, Beijing, China
| | - Fandong Zhang
- Deepwise Artificial Intelligence (AI) Lab, Deepwise, Beijing, China
| | - Rongpin Wang
- Department of Medical Imaging, Guizhou Province People's Hospital, Guiyang, Guizhou, China
| | - Xiao Luo
- Department of Radiology, Ma'anshan People's Hospital, Ma'anshan, Anhui, China
| | - Feng Xu
- Department of Medical Imaging, the Affiliated Suqian First People's Hospital of Nanjing Medical University, Suqian, Jiangsu, China
| | - Sheng Li
- Department of Radiology, People's Hospital, Hubei University of Medicine, Shiyan, Hubei, China
| | - Xiangming Fang
- Department of Medical Imaging, the Affiliated Wuxi People's Hospital of Nanjing Medical University, Wuxi, Jiangsu, China
| | - Xiaodong Wang
- Department of Radiology, General Hospital of Ningxia Medical University, Yinchuan, Ningxia, China
| | - Ge Yan
- Department of Medical Imaging, the First Affiliated Hospital of Xi'an Jiaotong University, Xi'an, Shaanxi, China
| | - Fajin Lv
- Department of Radiology, the First Affiliated Hospital of Chongqing Medical University, Chongqing, China
| | - Meng Zhang
- Department of Radiology, People's Hospital of Sanya, Sanya, Hainan, China
| | - Qiu Sun
- Department of Radiology, Lanzhou University Second Hospital, Lanzhou, Gansu, China
| | - Guangbin Cui
- Department of Radiology, Tangdu Hospital, Air Force Medical University (Fourth Military Medical University), Xi'an, Shaanxi, China
| | - Yubao Liu
- Medical Imaging Center, Shenzhen Hospital of Southern Medical University, Shenzhen, Guangdong, China
| | - Shu Zhang
- Deepwise Artificial Intelligence (AI) Lab, Deepwise, Beijing, China
| | - Chengwei Pan
- Institute of Artificial Intelligence, Beihang University, Beijing, China
| | - Zhibo Hou
- Department of Radiology, Medical Imaging Center, Peking University Shougang Hospital, Beijing, China
| | - Huiying Liang
- Medical Big Data Center, Guangdong Provincial People's Hospital, Guangzhou Guangdong, China
| | - Yuning Pan
- Department of Radiology, Ningbo First Hospital, Ningbo, Zhejiang, China
| | - Xiaoxia Chen
- Department of Radiology, Third Center Medical Center, Chinese PLA General Hospital, Beijing, China
| | - Xiaorong Li
- Department of Radiology, General Hospital of Southern Theater Command, PLA, Guangzhou, Guangdong, China
| | - Fei Zhou
- Department of Radiology, Central Hospital of Jilin City, Jilin, China
| | - U Joseph Schoepf
- Division of Cardiovascular Imaging, Department of Radiology and Radiological Science, Medical University of South Carolina, Charleston, SC, USA
| | - Akos Varga-Szemes
- Division of Cardiovascular Imaging, Department of Radiology and Radiological Science, Medical University of South Carolina, Charleston, SC, USA
| | - W Garrison Moore
- Division of Cardiovascular Imaging, Department of Radiology and Radiological Science, Medical University of South Carolina, Charleston, SC, USA
| | - Yizhou Yu
- Department of Computer Science, The University of Hong Kong, Hong Kong Special Administrative Region, China
| | - Chunfeng Hu
- Department of Radiology, the Affiliated Hospital of Xuzhou Medical University, Xuzhou, Jiangsu, China
| | - Long Jiang Zhang
- Department of Radiology, Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, China.
| |
Collapse
|
24
|
Ferreres AR. Ethical and legal issues regarding artificial intelligence (AI) and management of surgical data. EUROPEAN JOURNAL OF SURGICAL ONCOLOGY 2024:108279. [PMID: 38555230 DOI: 10.1016/j.ejso.2024.108279] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2024] [Revised: 03/05/2024] [Accepted: 03/17/2024] [Indexed: 04/02/2024]
Abstract
The advent of AI in surgical practice is representing a major innovation. As its role expands and due to its several implications, strict compliance with ethical, legal and regulatory good practices is mandatory. Observance of ethical principles and legal rules will be a professional imperative for the application of AI in surgical practice, both clinically and scientifically.
Collapse
Affiliation(s)
- Alberto R Ferreres
- University of Buenos Aires, Buenos Aires, Argentina; University of Washington, Seattle, USA.
| |
Collapse
|
25
|
Ong AY, Hogg HDJ, Kale AU, Taribagil P, Kras A, Dow E, Macdonald T, Liu X, Keane PA, Denniston AK. AI as a Medical Device for Ophthalmic Imaging in Europe, Australia, and the United States: Protocol for a Systematic Scoping Review of Regulated Devices. JMIR Res Protoc 2024; 13:e52602. [PMID: 38483456 PMCID: PMC10979335 DOI: 10.2196/52602] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2023] [Revised: 02/10/2024] [Accepted: 02/20/2024] [Indexed: 04/01/2024] Open
Abstract
BACKGROUND Artificial intelligence as a medical device (AIaMD) has the potential to transform many aspects of ophthalmic care, such as improving accuracy and speed of diagnosis, addressing capacity issues in high-volume areas such as screening, and detecting novel biomarkers of systemic disease in the eye (oculomics). In order to ensure that such tools are safe for the target population and achieve their intended purpose, it is important that these AIaMD have adequate clinical evaluation to support any regulatory decision. Currently, the evidential requirements for regulatory approval are less clear for AIaMD compared to more established interventions such as drugs or medical devices. There is therefore value in understanding the level of evidence that underpins AIaMD currently on the market, as a step toward identifying what the best practices might be in this area. In this systematic scoping review, we will focus on AIaMD that contributes to clinical decision-making (relating to screening, diagnosis, prognosis, and treatment) in the context of ophthalmic imaging. OBJECTIVE This study aims to identify regulator-approved AIaMD for ophthalmic imaging in Europe, Australia, and the United States; report the characteristics of these devices and their regulatory approvals; and report the available evidence underpinning these AIaMD. METHODS The Food and Drug Administration (United States), the Australian Register of Therapeutic Goods (Australia), the Medicines and Healthcare products Regulatory Agency (United Kingdom), and the European Database on Medical Devices (European Union) regulatory databases will be searched for ophthalmic imaging AIaMD through a snowballing approach. PubMed and clinical trial registries will be systematically searched, and manufacturers will be directly contacted for studies investigating the effectiveness of eligible AIaMD. Preliminary regulatory database searches, evidence searches, screening, data extraction, and methodological quality assessment will be undertaken by 2 independent review authors and arbitrated by a third at each stage of the process. RESULTS Preliminary searches were conducted in February 2023. Data extraction, data synthesis, and assessment of methodological quality commenced in October 2023. The review is on track to be completed and submitted for peer review by April 2024. CONCLUSIONS This systematic review will provide greater clarity on ophthalmic imaging AIaMD that have achieved regulatory approval as well as the evidence that underpins them. This should help adopters understand the range of tools available and whether they can be safely incorporated into their clinical workflow, and it should also support developers in navigating regulatory approval more efficiently. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) DERR1-10.2196/52602.
Collapse
Affiliation(s)
- Ariel Yuhan Ong
- Moorfields Eye Hospital NHS Foundation Trust, London, United Kingdom
- Oxford Eye Hospital, Oxford University Hospitals NHS Foundation Trust, Oxford, United Kingdom
| | - Henry David Jeffry Hogg
- Moorfields Eye Hospital NHS Foundation Trust, London, United Kingdom
- Population Health Sciences Institute, Faculty of Medical Sciences, Newcastle University, Newcastle, United Kingdom
| | - Aditya U Kale
- Department of Ophthalmology, Queen Elizabeth Hospital Birmingham, University Hospitals Birmingham NHS Foundation Trust, Birmingham, United Kingdom
- Institute of Inflammation and Ageing, University of Birmingham, Birmingham, United Kingdom
| | - Priyal Taribagil
- Moorfields Eye Hospital NHS Foundation Trust, London, United Kingdom
| | | | - Eliot Dow
- Retinal Consultants Medical Group, Sacramento, CA, United States
| | - Trystan Macdonald
- Department of Ophthalmology, Queen Elizabeth Hospital Birmingham, University Hospitals Birmingham NHS Foundation Trust, Birmingham, United Kingdom
- Institute of Inflammation and Ageing, University of Birmingham, Birmingham, United Kingdom
- NIHR Birmingham Biomedical Research Centre, Birmingham, United Kingdom
| | - Xiaoxuan Liu
- Department of Ophthalmology, Queen Elizabeth Hospital Birmingham, University Hospitals Birmingham NHS Foundation Trust, Birmingham, United Kingdom
- Institute of Inflammation and Ageing, University of Birmingham, Birmingham, United Kingdom
- NIHR Birmingham Biomedical Research Centre, Birmingham, United Kingdom
- Centre for Regulatory Science and Innovation, Birmingham Health Partners, Birmingham, United Kingdom
| | - Pearse A Keane
- Moorfields Eye Hospital NHS Foundation Trust, London, United Kingdom
- Institute of Ophthalmology, University College London, London, United Kingdom
- NIHR Moorfields Biomedical Research Centre, London, United Kingdom
| | - Alastair K Denniston
- Department of Ophthalmology, Queen Elizabeth Hospital Birmingham, University Hospitals Birmingham NHS Foundation Trust, Birmingham, United Kingdom
- Institute of Inflammation and Ageing, University of Birmingham, Birmingham, United Kingdom
- NIHR Birmingham Biomedical Research Centre, Birmingham, United Kingdom
- Centre for Regulatory Science and Innovation, Birmingham Health Partners, Birmingham, United Kingdom
| |
Collapse
|
26
|
van de Sande D, Chung EFF, Oosterhoff J, van Bommel J, Gommers D, van Genderen ME. To warrant clinical adoption AI models require a multi-faceted implementation evaluation. NPJ Digit Med 2024; 7:58. [PMID: 38448743 PMCID: PMC10918103 DOI: 10.1038/s41746-024-01064-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2023] [Accepted: 02/22/2024] [Indexed: 03/08/2024] Open
Abstract
Despite artificial intelligence (AI) technology progresses at unprecedented rate, our ability to translate these advancements into clinical value and adoption at the bedside remains comparatively limited. This paper reviews the current use of implementation outcomes in randomized controlled trials evaluating AI-based clinical decision support and found limited adoption. To advance trust and clinical adoption of AI, there is a need to bridge the gap between traditional quantitative metrics and implementation outcomes to better grasp the reasons behind the success or failure of AI systems and improve their translation into clinical value.
Collapse
Affiliation(s)
- Davy van de Sande
- Erasmus MC University Medical Center, Department of Adult Intensive Care, Rotterdam, The Netherlands
| | - Eline Fung Fen Chung
- Erasmus MC University Medical Center, Department of Adult Intensive Care, Rotterdam, The Netherlands
| | - Jacobien Oosterhoff
- Delft University of Technology, Faculty of Technology, Policy and Management, Delft, The Netherlands
| | - Jasper van Bommel
- Erasmus MC University Medical Center, Department of Adult Intensive Care, Rotterdam, The Netherlands
| | - Diederik Gommers
- Erasmus MC University Medical Center, Department of Adult Intensive Care, Rotterdam, The Netherlands
| | - Michel E van Genderen
- Erasmus MC University Medical Center, Department of Adult Intensive Care, Rotterdam, The Netherlands.
| |
Collapse
|
27
|
Yao S, Dai F, Sun P, Zhang W, Qian B, Lu H. Enhancing the fairness of AI prediction models by Quasi-Pareto improvement among heterogeneous thyroid nodule population. Nat Commun 2024; 15:1958. [PMID: 38438371 PMCID: PMC10912763 DOI: 10.1038/s41467-024-44906-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2023] [Accepted: 01/09/2024] [Indexed: 03/06/2024] Open
Abstract
Artificial Intelligence (AI) models for medical diagnosis often face challenges of generalizability and fairness. We highlighted the algorithmic unfairness in a large thyroid ultrasound dataset with significant diagnostic performance disparities across subgroups linked causally to sample size imbalances. To address this, we introduced the Quasi-Pareto Improvement (QPI) approach and a deep learning implementation (QP-Net) combining multi-task learning and domain adaptation to improve model performance among disadvantaged subgroups without compromising overall population performance. On the thyroid ultrasound dataset, our method significantly mitigated the area under curve (AUC) disparity for three less-prevalent subgroups by 0.213, 0.112, and 0.173 while maintaining the AUC for dominant subgroups; we also further confirmed the generalizability of our approach on two public datasets: the ISIC2019 skin disease dataset and the CheXpert chest radiograph dataset. Here we show the QPI approach to be widely applicable in promoting AI for equitable healthcare outcomes.
Collapse
Affiliation(s)
- Siqiong Yao
- State Key Laboratory of Microbial Metabolism, Joint International Research Laboratory of Metabolic and Developmental Sciences, School of Life Sciences and Biotechnology, Shanghai Jiao Tong University, Shanghai, 200240, PR China
- SJTU-Yale Joint Center of Biostatistics and Data Science, National Center for Translational Medicine, MoE Key Lab of Artificial Intelligence, AI Institute Shanghai Jiao Tong University, Shanghai, 200240, PR China
| | - Fang Dai
- State Key Laboratory of Microbial Metabolism, Joint International Research Laboratory of Metabolic and Developmental Sciences, School of Life Sciences and Biotechnology, Shanghai Jiao Tong University, Shanghai, 200240, PR China
| | - Peng Sun
- State Key Laboratory of Microbial Metabolism, Joint International Research Laboratory of Metabolic and Developmental Sciences, School of Life Sciences and Biotechnology, Shanghai Jiao Tong University, Shanghai, 200240, PR China
| | - Weituo Zhang
- Hongqiao International Institute of Medicine, Shanghai Tong Ren Hospital and School of Public Health, Shanghai Jiao Tong University School of Medicine, Shanghai, 200336, PR China.
| | - Biyun Qian
- Hongqiao International Institute of Medicine, Shanghai Tong Ren Hospital and School of Public Health, Shanghai Jiao Tong University School of Medicine, Shanghai, 200336, PR China.
| | - Hui Lu
- State Key Laboratory of Microbial Metabolism, Joint International Research Laboratory of Metabolic and Developmental Sciences, School of Life Sciences and Biotechnology, Shanghai Jiao Tong University, Shanghai, 200240, PR China.
- SJTU-Yale Joint Center of Biostatistics and Data Science, National Center for Translational Medicine, MoE Key Lab of Artificial Intelligence, AI Institute Shanghai Jiao Tong University, Shanghai, 200240, PR China.
- Shanghai Engineering Research Center for Big Data in Pediatric Precision Medicine, NHC Key Laboratory of Medical Embryogenesis and Developmental Molecular Biology & Shanghai Key Laboratory of Embryo and Reproduction Engineering, Shanghai, 200020, PR China.
| |
Collapse
|
28
|
Miró Catalina Q, Vidal-Alaball J, Fuster-Casanovas A, Escalé-Besa A, Ruiz Comellas A, Solé-Casals J. Real-world testing of an artificial intelligence algorithm for the analysis of chest X-rays in primary care settings. Sci Rep 2024; 14:5199. [PMID: 38431731 PMCID: PMC10908781 DOI: 10.1038/s41598-024-55792-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2023] [Accepted: 02/27/2024] [Indexed: 03/05/2024] Open
Abstract
Interpreting chest X-rays is a complex task, and artificial intelligence algorithms for this purpose are currently being developed. It is important to perform external validations of these algorithms in order to implement them. This study therefore aims to externally validate an AI algorithm's diagnoses in real clinical practice, comparing them to a radiologist's diagnoses. The aim is also to identify diagnoses the algorithm may not have been trained for. A prospective observational study for the external validation of the AI algorithm in a region of Catalonia, comparing the AI algorithm's diagnosis with that of the reference radiologist, considered the gold standard. The external validation was performed with a sample of 278 images and reports, 51.8% of which showed no radiological abnormalities according to the radiologist's report. Analysing the validity of the AI algorithm, the average accuracy was 0.95 (95% CI 0.92; 0.98), the sensitivity was 0.48 (95% CI 0.30; 0.66) and the specificity was 0.98 (95% CI 0.97; 0.99). The conditions where the algorithm was most sensitive were external, upper abdominal and cardiac and/or valvular implants. On the other hand, the conditions where the algorithm was less sensitive were in the mediastinum, vessels and bone. The algorithm has been validated in the primary care setting and has proven to be useful when identifying images with or without conditions. However, in order to be a valuable tool to help and support experts, it requires additional real-world training to enhance its diagnostic capabilities for some of the conditions analysed. Our study emphasizes the need for continuous improvement to ensure the algorithm's effectiveness in primary care.
Collapse
Affiliation(s)
- Queralt Miró Catalina
- Unitat de Suport a la Recerca de la Catalunya Central, Fundació Institut Universitari per a la Recerca a l'Atenció Primària de Salut Jordi Gol i Gurina, Sant Fruitós de Bages, Spain
- Health Promotion in Rural Areas Research Group, Gerència d'Atenció Primària i a la Comunitat de la Catalunya Central, Institut Català de la Salut, Carrer Pica d'Estats, 13-15, 08272, Sant Fruitós de Bages, Barcelona, Spain
- Faculty of Science Technology and Engineering, University of Vic-Central University of Catalonia, Vic, Spain
| | - Josep Vidal-Alaball
- Unitat de Suport a la Recerca de la Catalunya Central, Fundació Institut Universitari per a la Recerca a l'Atenció Primària de Salut Jordi Gol i Gurina, Sant Fruitós de Bages, Spain.
- Health Promotion in Rural Areas Research Group, Gerència d'Atenció Primària i a la Comunitat de la Catalunya Central, Institut Català de la Salut, Carrer Pica d'Estats, 13-15, 08272, Sant Fruitós de Bages, Barcelona, Spain.
- Faculty of Medicine, University of Vic-Central University of Catalonia, Vic, Spain.
| | - Aïna Fuster-Casanovas
- Unitat de Suport a la Recerca de la Catalunya Central, Fundació Institut Universitari per a la Recerca a l'Atenció Primària de Salut Jordi Gol i Gurina, Sant Fruitós de Bages, Spain
- Health Promotion in Rural Areas Research Group, Gerència d'Atenció Primària i a la Comunitat de la Catalunya Central, Institut Català de la Salut, Carrer Pica d'Estats, 13-15, 08272, Sant Fruitós de Bages, Barcelona, Spain
| | - Anna Escalé-Besa
- Unitat de Suport a la Recerca de la Catalunya Central, Fundació Institut Universitari per a la Recerca a l'Atenció Primària de Salut Jordi Gol i Gurina, Sant Fruitós de Bages, Spain
- Health Promotion in Rural Areas Research Group, Gerència d'Atenció Primària i a la Comunitat de la Catalunya Central, Institut Català de la Salut, Carrer Pica d'Estats, 13-15, 08272, Sant Fruitós de Bages, Barcelona, Spain
- Faculty of Medicine, University of Vic-Central University of Catalonia, Vic, Spain
| | - Anna Ruiz Comellas
- Unitat de Suport a la Recerca de la Catalunya Central, Fundació Institut Universitari per a la Recerca a l'Atenció Primària de Salut Jordi Gol i Gurina, Sant Fruitós de Bages, Spain
- Health Promotion in Rural Areas Research Group, Gerència d'Atenció Primària i a la Comunitat de la Catalunya Central, Institut Català de la Salut, Carrer Pica d'Estats, 13-15, 08272, Sant Fruitós de Bages, Barcelona, Spain
- Faculty of Medicine, University of Vic-Central University of Catalonia, Vic, Spain
| | - Jordi Solé-Casals
- Data and Signal Processing Group, Faculty of Science, Technology and Engineering, University of Vic-Central University of Catalonia, Vic, Spain.
- Department of Psychiatry, University of Cambridge, Cambridge, UK.
| |
Collapse
|
29
|
Evans RP, Bryant LD, Russell G, Absolom K. Trust and acceptability of data-driven clinical recommendations in everyday practice: A scoping review. Int J Med Inform 2024; 183:105342. [PMID: 38266426 DOI: 10.1016/j.ijmedinf.2024.105342] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2023] [Revised: 12/08/2023] [Accepted: 01/14/2024] [Indexed: 01/26/2024]
Abstract
BACKGROUND Increasing attention is being given to the analysis of large health datasets to derive new clinical decision support systems (CDSS). However, few data-driven CDSS are being adopted into clinical practice. Trust in these tools is believed to be fundamental for acceptance and uptake but to date little attention has been given to defining or evaluating trust in clinical settings. OBJECTIVES A scoping review was conducted to explore how and where acceptability and trustworthiness of data-driven CDSS have been assessed from the health professional's perspective. METHODS Medline, Embase, PsycInfo, Web of Science, Scopus, ACM Digital, IEEE Xplore and Google Scholar were searched in March 2022 using terms expanded from: "data-driven" AND "clinical decision support" AND "acceptability". Included studies focused on healthcare practitioner-facing data-driven CDSS, relating directly to clinical care. They included trust or a proxy as an outcome, or in the discussion. The preferred reporting items for systematic reviews and meta-analyses extension for scoping reviews (PRISMA-ScR) is followed in the reporting of this review. RESULTS 3291 papers were screened, with 85 primary research studies eligible for inclusion. Studies covered a diverse range of clinical specialisms and intended contexts, but hypothetical systems (24) outnumbered those in clinical use (18). Twenty-five studies measured trust, via a wide variety of quantitative, qualitative and mixed methods. A further 24 discussed themes of trust without it being explicitly evaluated, and from these, themes of transparency, explainability, and supporting evidence were identified as factors influencing healthcare practitioner trust in data-driven CDSS. CONCLUSION There is a growing body of research on data-driven CDSS, but few studies have explored stakeholder perceptions in depth, with limited focused research on trustworthiness. Further research on healthcare practitioner acceptance, including requirements for transparency and explainability, should inform clinical implementation.
Collapse
Affiliation(s)
- Ruth P Evans
- University of Leeds, Woodhouse Lane, Leeds LS2 9JT, UK.
| | | | - Gregor Russell
- Bradford District Care Trust, Bradford, New Mill, Victoria Rd, BD18 3LD, UK.
| | - Kate Absolom
- University of Leeds, Woodhouse Lane, Leeds LS2 9JT, UK.
| |
Collapse
|
30
|
Hanassab S, Abbara A, Yeung AC, Voliotis M, Tsaneva-Atanasova K, Kelsey TW, Trew GH, Nelson SM, Heinis T, Dhillo WS. The prospect of artificial intelligence to personalize assisted reproductive technology. NPJ Digit Med 2024; 7:55. [PMID: 38429464 PMCID: PMC10907618 DOI: 10.1038/s41746-024-01006-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2023] [Accepted: 01/10/2024] [Indexed: 03/03/2024] Open
Abstract
Infertility affects 1-in-6 couples, with repeated intensive cycles of assisted reproductive technology (ART) required by many to achieve a desired live birth. In ART, typically, clinicians and laboratory staff consider patient characteristics, previous treatment responses, and ongoing monitoring to determine treatment decisions. However, the reproducibility, weighting, and interpretation of these characteristics are contentious, and highly operator-dependent, resulting in considerable reliance on clinical experience. Artificial intelligence (AI) is ideally suited to handle, process, and analyze large, dynamic, temporal datasets with multiple intermediary outcomes that are generated during an ART cycle. Here, we review how AI has demonstrated potential for optimization and personalization of key steps in a reproducible manner, including: drug selection and dosing, cycle monitoring, induction of oocyte maturation, and selection of the most competent gametes and embryos, to improve the overall efficacy and safety of ART.
Collapse
Affiliation(s)
- Simon Hanassab
- Department of Metabolism, Digestion, and Reproduction, Imperial College London, London, UK
- Department of Computing, Imperial College London, London, UK
- UKRI Centre for Doctoral Training in AI for Healthcare, Imperial College London, London, UK
| | - Ali Abbara
- Department of Metabolism, Digestion, and Reproduction, Imperial College London, London, UK
- Imperial College Healthcare NHS Trust, London, UK
| | - Arthur C Yeung
- Department of Metabolism, Digestion, and Reproduction, Imperial College London, London, UK
- Imperial College Healthcare NHS Trust, London, UK
| | - Margaritis Voliotis
- Department of Mathematics and Statistics, University of Exeter, Exeter, UK
- Living Systems Institute, University of Exeter, Exeter, UK
- EPSRC Hub for Quantitative Modelling in Healthcare, University of Exeter, Exeter, UK
| | - Krasimira Tsaneva-Atanasova
- Department of Mathematics and Statistics, University of Exeter, Exeter, UK
- Living Systems Institute, University of Exeter, Exeter, UK
- EPSRC Hub for Quantitative Modelling in Healthcare, University of Exeter, Exeter, UK
| | - Tom W Kelsey
- School of Computer Science, University of St Andrews, St Andrews, UK
| | - Geoffrey H Trew
- Department of Metabolism, Digestion, and Reproduction, Imperial College London, London, UK
- The Fertility Partnership, Oxford, UK
| | - Scott M Nelson
- The Fertility Partnership, Oxford, UK
- School of Medicine, University of Glasgow, Glasgow, UK
- Biomedical Research Centre, University of Bristol, Bristol, UK
| | - Thomas Heinis
- Department of Computing, Imperial College London, London, UK
| | - Waljit S Dhillo
- Department of Metabolism, Digestion, and Reproduction, Imperial College London, London, UK.
- Imperial College Healthcare NHS Trust, London, UK.
| |
Collapse
|
31
|
Roufosse C, Naesens M, Haas M, Lefaucheur C, Mannon RB, Afrouzian M, Alachkar N, Aubert O, Bagnasco SM, Batal I, Bellamy COC, Broecker V, Budde K, Clahsen-Van Groningen M, Coley SM, Cornell LD, Dadhania D, Demetris AJ, Einecke G, Farris AB, Fogo AB, Friedewald J, Gibson IW, Horsfield C, Huang E, Husain SA, Jackson AM, Kers J, Kikić Ž, Klein A, Kozakowski N, Liapis H, Mangiola M, Montgomery RA, Nankinvell B, Neil DAH, Nickerson P, Rabant M, Randhawa P, Riella LV, Rosales I, Royal V, Sapir-Pichhadze R, Sarder P, Sarwal M, Schinstock C, Stegall M, Solez K, van der Laak J, Wiebe C, Colvin RB, Loupy A, Mengel M. The Banff 2022 Kidney Meeting Work Plan: Data-driven refinement of the Banff Classification for renal allografts. Am J Transplant 2024; 24:350-361. [PMID: 37931753 PMCID: PMC11135910 DOI: 10.1016/j.ajt.2023.10.031] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2023] [Accepted: 10/11/2023] [Indexed: 11/08/2023]
Abstract
The XVIth Banff Meeting for Allograft Pathology was held in Banff, Alberta, Canada, from September 19 to 23, 2022, as a joint meeting with the Canadian Society of Transplantation. In addition to a key focus on the impact of microvascular inflammation and biopsy-based transcript analysis on the Banff Classification, further sessions were devoted to other aspects of kidney transplant pathology, in particular T cell-mediated rejection, activity and chronicity indices, digital pathology, xenotransplantation, clinical trials, and surrogate endpoints. Although the output of these sessions has not led to any changes in the classification, the key role of Banff Working Groups in phrasing unanswered questions, and coordinating and disseminating results of investigations addressing these unanswered questions was emphasized. This paper summarizes the key Banff Meeting 2022 sessions not covered in the Banff Kidney Meeting 2022 Report paper and also provides an update on other Banff Working Group activities relevant to kidney allografts.
Collapse
Affiliation(s)
- Candice Roufosse
- Department of Immunology and Inflammation, Faculty Medicine, Imperial College London, London, UK.
| | - Maarten Naesens
- Department of Microbiology, Immunology and Transplantation, KU Leuven, Leuven, Belgium.
| | - Mark Haas
- Department of Pathology and Laboratory Medicine, Cedars-Sinai Medical Center, Los Angeles, California, USA
| | - Carmen Lefaucheur
- Université Paris Cité, INSERM, PARCC, Paris Institute for Transplantation and Organ Regeneration, France & Department of Nephrology and Transplantation, Saint-Louis Hospital, Paris, France
| | - Roslyn B Mannon
- Department of Internal Medicine, Division of Nephrology, University of Nebraska Medical Center, Omaha, Nebraska, USA
| | - Marjan Afrouzian
- Department of Pathology, University of Texas Medical Branch at Galveston, Texas, USA
| | - Nada Alachkar
- Department of Medicine, The Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
| | - Olivier Aubert
- Université Paris Cité, INSERM, PARCC, Paris Institute for Transplantation and Organ Regeneration, France & Department of Transplantation, Necker Hospital, Paris, France
| | - Serena M Bagnasco
- Department of Pathology, Johns Hopkins School of Medicine, Baltimore, Maryland, USA
| | - Ibrahim Batal
- Pathology & Cell Biology, Columbia University Irving Medical Center, New York, USA
| | | | - Verena Broecker
- Department of Clinical Pathology, Sahlgrenska University Hospital, Gothenburg, Sweden
| | - Klemens Budde
- Department of Nephrology, Charité Universitätsmedizin, Berlin, Germany
| | - Marian Clahsen-Van Groningen
- Department of Pathology and Clinical Bioinformatics, Erasmus University Center Rotterdam, Rotterdam, Netherlands; Institute of Experimental Medicine and Systems Biology, RWTH Aachen University, Aachen, Germany
| | - Shana M Coley
- Transplant Translational Research, Arkana Laboratories, Arkansas, USA
| | - Lynn D Cornell
- Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, Minnesota, USA
| | - Darshana Dadhania
- Department Medicine, Weill Cornell Medical College of Cornell University, New York, New York, USA
| | - Anthony J Demetris
- UPMC Hepatic and Transplantation Pathology, Pittsburg, Pennsylvania, USA
| | - Gunilla Einecke
- Department of Nephrology and Rheumatology, University Medical Center Göttingen, Germany
| | - Alton B Farris
- Department of Pathology and Laboratory Medicine, Emory University, USA
| | - Agnes B Fogo
- Department of Pathology, Microbiology and Immunology, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - John Friedewald
- Comprehensive Transplant Center, Northwestern University, USA
| | - Ian W Gibson
- Department of Pathology, University of Manitoba, Winnipeg, Canada
| | | | - Edmund Huang
- Department of Medicine, Division of Nephrology, Cedars-Sinai Medical Center, Los Angeles, California, USA
| | - Syed A Husain
- Division of Nephrology, Columbia University, New York, New York, USA
| | | | - Jesper Kers
- Department of Pathology, Leiden University Medical Center, Netherlands; Department of Pathology, Amsterdam UMC, University of Amsterdam, Amsterdam, Netherlands
| | - Željko Kikić
- Department of Urology, Medical University of Vienna, Vienna, Austria
| | | | | | - Helen Liapis
- Ludwig Maximillian University Munich, Nephrology Center, Germany
| | | | | | - Brian Nankinvell
- Department of Renal Medicine, Westmead Hospital, Westmead, New South Wales, Australia
| | - Desley A H Neil
- Department of Cellular Pathology, Queen Elizabeth Hospital Birmingham and Institute of Immunology and Immunotherapy, University of Birmingham, Birmingham, UK
| | - Peter Nickerson
- Department of Medicine and Department of Immunology, University of Manitoba, Winnipeg, Canada
| | - Marion Rabant
- Pathology department, Necker-Enfants Malades Hospital, Paris, France
| | - Parmjeet Randhawa
- Pathology, Thomas E. Starzl Transplant Institute, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Leonardo V Riella
- Department of Medicine, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA
| | - Ivy Rosales
- Immunopathology Research Laboratory, Department of Pathology, Massachusetts General Hospital, Boston, Massachusetts, USA
| | - Virginie Royal
- Maisonneuve-Rosemont Hospital, University of Montreal, Quebec, Canada
| | - Ruth Sapir-Pichhadze
- Division of Nephrology & Multiorgan Transplant Program, McGill University, Montreal, Quebec, Canada
| | - Pinaki Sarder
- Department of Medicine-Quantitative Health, University of Florida College of Medicine, Florida, USA
| | - Minnie Sarwal
- Division of MultiOrgan Transplantation, UCSF, San Francisco, California, USA
| | - Carrie Schinstock
- Department of Internal Medicine, Division of Nephrology and Hypertension, Mayo Clinic, Rochester, Minnesota, USA
| | - Mark Stegall
- Department Transplantation Surgery, Mayo Clinic, Rochester, Massachusetts, USA
| | - Kim Solez
- Department of Laboratory Medicine and Pathology, University of Alberta, Edmonton, Canada
| | | | - Chris Wiebe
- Department of Medicine and Department of Immunology, University of Manitoba, Winnipeg, Canada
| | - Robert B Colvin
- Department of Pathology, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA
| | - Alexandre Loupy
- Université Paris Cité, INSERM, PARCC, Paris Institute for Transplantation and Organ Regeneration, France & Department of Transplantation, Necker Hospital, Paris, France
| | - Michael Mengel
- Department of Laboratory Medicine and Pathology, University of Alberta, Edmonton, Canada
| |
Collapse
|
32
|
Fehr J, Citro B, Malpani R, Lippert C, Madai VI. A trustworthy AI reality-check: the lack of transparency of artificial intelligence products in healthcare. Front Digit Health 2024; 6:1267290. [PMID: 38455991 PMCID: PMC10919164 DOI: 10.3389/fdgth.2024.1267290] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2023] [Accepted: 02/05/2024] [Indexed: 03/09/2024] Open
Abstract
Trustworthy medical AI requires transparency about the development and testing of underlying algorithms to identify biases and communicate potential risks of harm. Abundant guidance exists on how to achieve transparency for medical AI products, but it is unclear whether publicly available information adequately informs about their risks. To assess this, we retrieved public documentation on the 14 available CE-certified AI-based radiology products of the II b risk category in the EU from vendor websites, scientific publications, and the European EUDAMED database. Using a self-designed survey, we reported on their development, validation, ethical considerations, and deployment caveats, according to trustworthy AI guidelines. We scored each question with either 0, 0.5, or 1, to rate if the required information was "unavailable", "partially available," or "fully available." The transparency of each product was calculated relative to all 55 questions. Transparency scores ranged from 6.4% to 60.9%, with a median of 29.1%. Major transparency gaps included missing documentation on training data, ethical considerations, and limitations for deployment. Ethical aspects like consent, safety monitoring, and GDPR-compliance were rarely documented. Furthermore, deployment caveats for different demographics and medical settings were scarce. In conclusion, public documentation of authorized medical AI products in Europe lacks sufficient public transparency to inform about safety and risks. We call on lawmakers and regulators to establish legally mandated requirements for public and substantive transparency to fulfill the promise of trustworthy AI for health.
Collapse
Affiliation(s)
- Jana Fehr
- Digital Health & Machine Learning, Hasso Plattner Institute, Potsdam, Germany
- Digital Engineering Faculty, University of Potsdam, Potsdam, Germany
- QUEST Center for Responsible Research, Berlin Institute of Health (BIH), Charité Universitätsmedizin Berlin, Berlin, Germany
| | - Brian Citro
- Independent Researcher, Chicago, IL, United States
| | | | - Christoph Lippert
- Digital Health & Machine Learning, Hasso Plattner Institute, Potsdam, Germany
- Digital Engineering Faculty, University of Potsdam, Potsdam, Germany
- Hasso Plattner Institute for Digital Health at Mount Sinai, Icahn School of Medicine at Mount Sinai, New York, NY, United States
| | - Vince I. Madai
- QUEST Center for Responsible Research, Berlin Institute of Health (BIH), Charité Universitätsmedizin Berlin, Berlin, Germany
- Faculty of Computing, Engineering and the Built Environment, School of Computing and Digital Technology, Birmingham City University, Birmingham, United Kingdom
| |
Collapse
|
33
|
Economou-Zavlanos NJ, Bessias S, Cary MP, Bedoya AD, Goldstein BA, Jelovsek JE, O’Brien CL, Walden N, Elmore M, Parrish AB, Elengold S, Lytle KS, Balu S, Lipkin ME, Shariff AI, Gao M, Leverenz D, Henao R, Ming DY, Gallagher DM, Pencina MJ, Poon EG. Translating ethical and quality principles for the effective, safe and fair development, deployment and use of artificial intelligence technologies in healthcare. J Am Med Inform Assoc 2024; 31:705-713. [PMID: 38031481 PMCID: PMC10873841 DOI: 10.1093/jamia/ocad221] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Revised: 10/06/2023] [Accepted: 11/03/2023] [Indexed: 12/01/2023] Open
Abstract
OBJECTIVE The complexity and rapid pace of development of algorithmic technologies pose challenges for their regulation and oversight in healthcare settings. We sought to improve our institution's approach to evaluation and governance of algorithmic technologies used in clinical care and operations by creating an Implementation Guide that standardizes evaluation criteria so that local oversight is performed in an objective fashion. MATERIALS AND METHODS Building on a framework that applies key ethical and quality principles (clinical value and safety, fairness and equity, usability and adoption, transparency and accountability, and regulatory compliance), we created concrete guidelines for evaluating algorithmic technologies at our institution. RESULTS An Implementation Guide articulates evaluation criteria used during review of algorithmic technologies and details what evidence supports the implementation of ethical and quality principles for trustworthy health AI. Application of the processes described in the Implementation Guide can lead to algorithms that are safer as well as more effective, fair, and equitable upon implementation, as illustrated through 4 examples of technologies at different phases of the algorithmic lifecycle that underwent evaluation at our academic medical center. DISCUSSION By providing clear descriptions/definitions of evaluation criteria and embedding them within standardized processes, we streamlined oversight processes and educated communities using and developing algorithmic technologies within our institution. CONCLUSIONS We developed a scalable, adaptable framework for translating principles into evaluation criteria and specific requirements that support trustworthy implementation of algorithmic technologies in patient care and healthcare operations.
Collapse
Affiliation(s)
| | - Sophia Bessias
- Duke AI Health, Duke University School of Medicine, Durham, NC 27705, United States
| | - Michael P Cary
- Duke AI Health, Duke University School of Medicine, Durham, NC 27705, United States
- Duke University School of Nursing, Durham, NC 27710, United States
| | - Armando D Bedoya
- Duke Health Technology Solutions, Duke University Health System, Durham, NC 27705, United States
- Department of Medicine, Duke University School of Medicine, Durham, NC 27710, United States
| | - Benjamin A Goldstein
- Duke AI Health, Duke University School of Medicine, Durham, NC 27705, United States
- Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, NC 27705, United States
| | - John E Jelovsek
- Department of Obstetrics and Gynecology, Duke University School of Medicine, Durham, NC 27710, United States
| | - Cara L O’Brien
- Duke Health Technology Solutions, Duke University Health System, Durham, NC 27705, United States
- Department of Medicine, Duke University School of Medicine, Durham, NC 27710, United States
| | - Nancy Walden
- Duke AI Health, Duke University School of Medicine, Durham, NC 27705, United States
| | - Matthew Elmore
- Duke AI Health, Duke University School of Medicine, Durham, NC 27705, United States
| | - Amanda B Parrish
- Office of Regulatory Affairs and Quality, Duke University School of Medicine, Durham, NC 27705, United States
| | - Scott Elengold
- Office of Counsel, Duke University, Durham, NC 27701, United States
| | - Kay S Lytle
- Duke University School of Nursing, Durham, NC 27710, United States
- Duke Health Technology Solutions, Duke University Health System, Durham, NC 27705, United States
| | - Suresh Balu
- Duke Institute for Health Innovation, Duke University, Durham, NC 27701, United States
| | - Michael E Lipkin
- Department of Urology, Duke University School of Medicine, Durham, NC 27710, United States
| | - Afreen Idris Shariff
- Department of Medicine, Duke University School of Medicine, Durham, NC 27710, United States
- Duke Endocrine-Oncology Program, Duke University Health System, Durham, NC 27710, United States
| | - Michael Gao
- Duke Institute for Health Innovation, Duke University, Durham, NC 27701, United States
| | - David Leverenz
- Department of Medicine, Duke University School of Medicine, Durham, NC 27710, United States
| | - Ricardo Henao
- Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, NC 27705, United States
- Department of Bioengineering, King Abdullah University of Science and Technology, Thuwal 23955, Saudi Arabia
| | - David Y Ming
- Department of Medicine, Duke University School of Medicine, Durham, NC 27710, United States
- Duke Department of Pediatrics, Duke University Health System, Durham, NC 27705, United States
- Department of Population Health Sciences, Duke University School of Medicine, Durham, NC 27701, United States
| | - David M Gallagher
- Department of Medicine, Duke University School of Medicine, Durham, NC 27710, United States
| | - Michael J Pencina
- Duke AI Health, Duke University School of Medicine, Durham, NC 27705, United States
- Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, NC 27705, United States
| | - Eric G Poon
- Duke Health Technology Solutions, Duke University Health System, Durham, NC 27705, United States
- Department of Medicine, Duke University School of Medicine, Durham, NC 27710, United States
- Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, NC 27705, United States
| |
Collapse
|
34
|
Li P, Xiong F, Huang X, Wen X. Construction and optimization of vending machine decision support system based on improved C4.5 decision tree. Heliyon 2024; 10:e25024. [PMID: 38318033 PMCID: PMC10838796 DOI: 10.1016/j.heliyon.2024.e25024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2023] [Revised: 12/14/2023] [Accepted: 01/18/2024] [Indexed: 02/07/2024] Open
Abstract
The intensification of market competition makes refined operation management become the focus of attention of major manufacturers. As an important branch of artificial intelligence (AI), machine learning (ML) plays a key role in it, and has its application prospect in various systems. Based on this situation, this paper takes vending machines as the research object. On the one hand, the product classification model of vending machine is constructed based on decision tree algorithm. On the other hand, based on neural network (NN), the sales forecast model of vending machines is built. Finally, based on the above research, the theoretical framework of decision support system (DSS) for vending machines is constructed. The research shows that: (1) The accuracy of C4.5 algorithm can reach 87 % at the highest and 68 % at the lowest. The accuracy of the improved C4.5 algorithm can reach 87 % at the highest and 67 % at the lowest, with little difference between them. (2) The maximum running time of the improved C4.5 algorithm is about 5500 ms, and the minimum is close to 1 ms. In addition, the running time of all seven datasets is better than that of the unmodified algorithm. (3) When the back propagation neural network (BPNN) is used to forecast the sales of vending machines, the curve of the predicted data basically coincides with the curve of the actual data, which shows that its accuracy is high. This paper aims to build a convenient and secure DSS by taking vending machines as an example. In addition, this paper also uses reinforcement learning to optimize the research methods of this paper. It can further optimize the performance and efficiency of vending machines and provide better service experience for customers. Meanwhile, the use of reinforcement learning can make the whole system more intelligent and adaptive to better cope with the changing market environment.
Collapse
Affiliation(s)
- Ping Li
- School of Information and Mechatronic Engineering, Hunan International Economics University, Changsha, 410205, China
| | - Fang Xiong
- School of Information and Mechatronic Engineering, Hunan International Economics University, Changsha, 410205, China
| | - Xibei Huang
- School of Information and Mechatronic Engineering, Hunan International Economics University, Changsha, 410205, China
| | - Xiaojun Wen
- School of Information and Mechatronic Engineering, Hunan International Economics University, Changsha, 410205, China
| |
Collapse
|
35
|
Molinski NS, Kenda M, Leithner C, Nee J, Storm C, Scheel M, Meddeb A. Deep learning-enabled detection of hypoxic-ischemic encephalopathy after cardiac arrest in CT scans: a comparative study of 2D and 3D approaches. Front Neurosci 2024; 18:1245791. [PMID: 38419661 PMCID: PMC10899383 DOI: 10.3389/fnins.2024.1245791] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2023] [Accepted: 01/31/2024] [Indexed: 03/02/2024] Open
Abstract
Objective To establish a deep learning model for the detection of hypoxic-ischemic encephalopathy (HIE) features on CT scans and to compare various networks to determine the best input data format. Methods 168 head CT scans of patients after cardiac arrest were retrospectively identified and classified into two categories: 88 (52.4%) with radiological evidence of severe HIE and 80 (47.6%) without signs of HIE. These images were randomly divided into a training and a test set, and five deep learning models based on based on Densely Connected Convolutional Networks (DenseNet121) were trained and validated using different image input formats (2D and 3D images). Results All optimized stacked 2D and 3D networks could detect signs of HIE. The networks based on the data as 2D image data stacks provided the best results (S100: AUC: 94%, ACC: 79%, S50: AUC: 93%, ACC: 79%). We provide visual explainability data for the decision making of our AI model using Gradient-weighted Class Activation Mapping. Conclusion Our proof-of-concept deep learning model can accurately identify signs of HIE on CT images. Comparing different 2D- and 3D-based approaches, most promising results were achieved by 2D image stack models. After further clinical validation, a deep learning model of HIE detection based on CT images could be implemented in clinical routine and thus aid clinicians in characterizing imaging data and predicting outcome.
Collapse
Affiliation(s)
- Noah S. Molinski
- Department for Neuroradiology, Charité – Universitätsmedizin Berlin, Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Martin Kenda
- Department of Neurology with Experimental Neurology, Charité – Universitätsmedizin Berlin, Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
- Berlin Institute of Health at Charité – Universitätsmedizin Berlin, BIH Biomedical Innovation Academy, Berlin, Germany
| | - Christoph Leithner
- Department of Neurology with Experimental Neurology, Charité – Universitätsmedizin Berlin, Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Jens Nee
- Department of Nephrology and Medical Intensive Care, Charité – Universitätsmedizin Berlin, Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Christian Storm
- Department of Nephrology and Medical Intensive Care, Charité – Universitätsmedizin Berlin, Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Michael Scheel
- Department for Neuroradiology, Charité – Universitätsmedizin Berlin, Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Aymen Meddeb
- Department for Neuroradiology, Charité – Universitätsmedizin Berlin, Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
- Berlin Institute of Health at Charité – Universitätsmedizin Berlin, BIH Biomedical Innovation Academy, Berlin, Germany
| |
Collapse
|
36
|
Yu P, Fang C, Liu X, Fu W, Ling J, Yan Z, Jiang Y, Cao Z, Wu M, Chen Z, Zhu W, Zhang Y, Abudukeremu A, Wang Y, Liu X, Wang J. Performance of ChatGPT on the Chinese Postgraduate Examination for Clinical Medicine: Survey Study. JMIR MEDICAL EDUCATION 2024; 10:e48514. [PMID: 38335017 PMCID: PMC10891494 DOI: 10.2196/48514] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/26/2023] [Revised: 10/04/2023] [Accepted: 12/11/2023] [Indexed: 02/10/2024]
Abstract
BACKGROUND ChatGPT, an artificial intelligence (AI) based on large-scale language models, has sparked interest in the field of health care. Nonetheless, the capabilities of AI in text comprehension and generation are constrained by the quality and volume of available training data for a specific language, and the performance of AI across different languages requires further investigation. While AI harbors substantial potential in medicine, it is imperative to tackle challenges such as the formulation of clinical care standards; facilitating cultural transitions in medical education and practice; and managing ethical issues including data privacy, consent, and bias. OBJECTIVE The study aimed to evaluate ChatGPT's performance in processing Chinese Postgraduate Examination for Clinical Medicine questions, assess its clinical reasoning ability, investigate potential limitations with the Chinese language, and explore its potential as a valuable tool for medical professionals in the Chinese context. METHODS A data set of Chinese Postgraduate Examination for Clinical Medicine questions was used to assess the effectiveness of ChatGPT's (version 3.5) medical knowledge in the Chinese language, which has a data set of 165 medical questions that were divided into three categories: (1) common questions (n=90) assessing basic medical knowledge, (2) case analysis questions (n=45) focusing on clinical decision-making through patient case evaluations, and (3) multichoice questions (n=30) requiring the selection of multiple correct answers. First of all, we assessed whether ChatGPT could meet the stringent cutoff score defined by the government agency, which requires a performance within the top 20% of candidates. Additionally, in our evaluation of ChatGPT's performance on both original and encoded medical questions, 3 primary indicators were used: accuracy, concordance (which validates the answer), and the frequency of insights. RESULTS Our evaluation revealed that ChatGPT scored 153.5 out of 300 for original questions in Chinese, which signifies the minimum score set to ensure that at least 20% more candidates pass than the enrollment quota. However, ChatGPT had low accuracy in answering open-ended medical questions, with only 31.5% total accuracy. The accuracy for common questions, multichoice questions, and case analysis questions was 42%, 37%, and 17%, respectively. ChatGPT achieved a 90% concordance across all questions. Among correct responses, the concordance was 100%, significantly exceeding that of incorrect responses (n=57, 50%; P<.001). ChatGPT provided innovative insights for 80% (n=132) of all questions, with an average of 2.95 insights per accurate response. CONCLUSIONS Although ChatGPT surpassed the passing threshold for the Chinese Postgraduate Examination for Clinical Medicine, its performance in answering open-ended medical questions was suboptimal. Nonetheless, ChatGPT exhibited high internal concordance and the ability to generate multiple insights in the Chinese language. Future research should investigate the language-based discrepancies in ChatGPT's performance within the health care context.
Collapse
Affiliation(s)
- Peng Yu
- Department of Endocrine, The Second Affiliated Hospital of Nanchang University, Jiangxi, China
| | - Changchang Fang
- Department of Endocrine, The Second Affiliated Hospital of Nanchang University, Jiangxi, China
| | - Xiaolin Liu
- Department of Cardiology, The Eighth Affiliated Hospital of Sun Yat-sen University, Shenzhen, China
| | - Wanying Fu
- Department of Endocrine, The Second Affiliated Hospital of Nanchang University, Jiangxi, China
| | - Jitao Ling
- Department of Endocrine, The Second Affiliated Hospital of Nanchang University, Jiangxi, China
| | - Zhiwei Yan
- College of Kinesiology, Shenyang Sport University, Shenyang, China
| | - Yuan Jiang
- Department of Cardiology, Sun Yat-sen Memorial Hospital of Sun Yat-sen University, Guangzhou, China
| | - Zhengyu Cao
- Department of Cardiology, Sun Yat-sen Memorial Hospital of Sun Yat-sen University, Guangzhou, China
| | - Maoxiong Wu
- Department of Cardiology, Sun Yat-sen Memorial Hospital of Sun Yat-sen University, Guangzhou, China
| | - Zhiteng Chen
- Department of Cardiology, Sun Yat-sen Memorial Hospital of Sun Yat-sen University, Guangzhou, China
| | - Wengen Zhu
- Department of Cardiology, The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China
| | - Yuling Zhang
- Department of Cardiology, Sun Yat-sen Memorial Hospital of Sun Yat-sen University, Guangzhou, China
| | - Ayiguli Abudukeremu
- Department of Cardiology, Sun Yat-sen Memorial Hospital of Sun Yat-sen University, Guangzhou, China
| | - Yue Wang
- Department of Cardiology, Sun Yat-sen Memorial Hospital of Sun Yat-sen University, Guangzhou, China
| | - Xiao Liu
- Department of Cardiology, Sun Yat-sen Memorial Hospital of Sun Yat-sen University, Guangzhou, China
| | - Jingfeng Wang
- Department of Cardiology, Sun Yat-sen Memorial Hospital of Sun Yat-sen University, Guangzhou, China
| |
Collapse
|
37
|
Cai Y, Cai YQ, Tang LY, Wang YH, Gong M, Jing TC, Li HJ, Li-Ling J, Hu W, Yin Z, Gong DX, Zhang GW. Artificial intelligence in the risk prediction models of cardiovascular disease and development of an independent validation screening tool: a systematic review. BMC Med 2024; 22:56. [PMID: 38317226 PMCID: PMC10845808 DOI: 10.1186/s12916-024-03273-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/16/2023] [Accepted: 01/23/2024] [Indexed: 02/07/2024] Open
Abstract
BACKGROUND A comprehensive overview of artificial intelligence (AI) for cardiovascular disease (CVD) prediction and a screening tool of AI models (AI-Ms) for independent external validation are lacking. This systematic review aims to identify, describe, and appraise AI-Ms of CVD prediction in the general and special populations and develop a new independent validation score (IVS) for AI-Ms replicability evaluation. METHODS PubMed, Web of Science, Embase, and IEEE library were searched up to July 2021. Data extraction and analysis were performed for the populations, distribution, predictors, algorithms, etc. The risk of bias was evaluated with the prediction risk of bias assessment tool (PROBAST). Subsequently, we designed IVS for model replicability evaluation with five steps in five items, including transparency of algorithms, performance of models, feasibility of reproduction, risk of reproduction, and clinical implication, respectively. The review is registered in PROSPERO (No. CRD42021271789). RESULTS In 20,887 screened references, 79 articles (82.5% in 2017-2021) were included, which contained 114 datasets (67 in Europe and North America, but 0 in Africa). We identified 486 AI-Ms, of which the majority were in development (n = 380), but none of them had undergone independent external validation. A total of 66 idiographic algorithms were found; however, 36.4% were used only once and only 39.4% over three times. A large number of different predictors (range 5-52,000, median 21) and large-span sample size (range 80-3,660,000, median 4466) were observed. All models were at high risk of bias according to PROBAST, primarily due to the incorrect use of statistical methods. IVS analysis confirmed only 10 models as "recommended"; however, 281 and 187 were "not recommended" and "warning," respectively. CONCLUSION AI has led the digital revolution in the field of CVD prediction, but is still in the early stage of development as the defects of research design, report, and evaluation systems. The IVS we developed may contribute to independent external validation and the development of this field.
Collapse
Affiliation(s)
- Yue Cai
- China Medical University, Shenyang, 110122, China
| | - Yu-Qing Cai
- China Medical University, Shenyang, 110122, China
| | - Li-Ying Tang
- China Medical University, Shenyang, 110122, China
| | - Yi-Han Wang
- China Medical University, Shenyang, 110122, China
| | - Mengchun Gong
- Digital Health China Co. Ltd, Beijing, 100089, China
| | - Tian-Ci Jing
- Smart Hospital Management Department, the First Hospital of China Medical University, Shenyang, 110001, China
| | - Hui-Jun Li
- Shenyang Medical & Film Science and Technology Co. Ltd., Shenyang, 110001, China
- Enduring Medicine Smart Innovation Research Institute, Shenyang, 110001, China
| | - Jesse Li-Ling
- Institute of Genetic Medicine, School of Life Science, State Key Laboratory of Biotherapy, Sichuan University, Chengdu, 610065, China
| | - Wei Hu
- Bayi Orthopedic Hospital, Chengdu, 610017, China
| | - Zhihua Yin
- Department of Epidemiology, School of Public Health, China Medical University, Shenyang, 110122, China.
| | - Da-Xin Gong
- Smart Hospital Management Department, the First Hospital of China Medical University, Shenyang, 110001, China.
- The Internet Hospital Branch of the Chinese Research Hospital Association, Beijing, 100006, China.
| | - Guang-Wei Zhang
- Smart Hospital Management Department, the First Hospital of China Medical University, Shenyang, 110001, China.
- The Internet Hospital Branch of the Chinese Research Hospital Association, Beijing, 100006, China.
| |
Collapse
|
38
|
Guenoun D, Zins M, Champsaur P, Thomassin-Naggara I. French community grid for the evaluation of radiological artificial intelligence solutions (DRIM France Artificial Intelligence Initiative). Diagn Interv Imaging 2024; 105:74-81. [PMID: 37749026 DOI: 10.1016/j.diii.2023.09.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2023] [Revised: 09/08/2023] [Accepted: 09/09/2023] [Indexed: 09/27/2023]
Abstract
PURPOSE The purpose of this study was to validate a national descriptive and analytical grid for artificial intelligence (AI) solutions in radiology. MATERIALS AND METHODS The RAND-UCLA Appropriateness Method was chosen by expert radiologists from the DRIM France IA group for this statement paper. The study, initiated by the radiology community, involved seven steps including literature review, template development, panel selection, pre-panel meeting survey, data extraction and analysis, second and final panel meeting, and data reporting. RESULTS The panel consisted of seven software vendors, three for bone fracture detection using conventional radiology and four for breast cancer detection using mammography. A consensus was reached on various aspects, including general target, main objective, certification marking, integration, expression of results, forensic aspects and cybersecurity, performance and scientific validation, description of the company and economic details, possible usage scenarios in the clinical workflow, database, specific objectives and targets of the AI tool. CONCLUSION The study validates a descriptive and analytical grid for radiological AI solutions consisting of ten items, using breast cancer and bone fracture as an experimental guide. This grid would assist radiologists in selecting relevant and validated AI solutions. Further developments of the grid are needed to include other organs and tasks.
Collapse
Affiliation(s)
- Daphné Guenoun
- APHM, Sainte-Marguerite Hospital, Institute for Locomotion, Department of Radiology, 13009, Marseille, France; Aix Marseille Univ, CNRS, ISM, Inst Movement Sci, 13009, Marseille, France.
| | - Marc Zins
- Department of Radiology and Medical Imaging, Saint-Joseph Hospital, 75014, Paris, France
| | - Pierre Champsaur
- APHM, Sainte-Marguerite Hospital, Institute for Locomotion, Department of Radiology, 13009, Marseille, France; Aix Marseille Univ, CNRS, ISM, Inst Movement Sci, 13009, Marseille, France
| | - Isabelle Thomassin-Naggara
- Sorbonne Université, 75005, Paris, France; Department of Diagnostic and Interventional Imaging, Assistance Publique-Hôpitaux de Paris, Hôpital Tenon, 75020 Paris, France
| |
Collapse
|
39
|
Yang F, Yan D, Wang Z. Large-Scale assessment of ChatGPT's performance in benign and malignant bone tumors imaging report diagnosis and its potential for clinical applications. J Bone Oncol 2024; 44:100525. [PMID: 38314324 PMCID: PMC10834989 DOI: 10.1016/j.jbo.2024.100525] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2023] [Revised: 01/03/2024] [Accepted: 01/07/2024] [Indexed: 02/06/2024] Open
Abstract
Objective This study was designed to delve into the complexities involved in diagnosing of benign and malignant bone tumors and to assess the potential of AI technologies like ChatGPT in improving diagnostic accuracy and efficiency. The study also explores the few-shot learning as a method to optimize ChatGPT's performance in specialized medical domains such as benign and malignant bone tumors diagnosis. Methods A total of 1366 benign and malignant bone tumors-related imaging reports were collected and diagnosed by 25 experienced physicians. The gold standard of diagnosis was established by combining clinical, imaging and pathological principles.These reports were then input into the ChatGPT model which underwent a few-shot learning method to generate diagnostic results. The diagnostic results of the physicians and the AI model were compared to evaluate the performance of ChatGPT. An experiment was conducted to assess the influence of different radiologist's reporting styles on the model's diagnostic performance. Furthermore, in-depth analysis of misdiagnosed cases was carried out, categorizing diagnostic errors and exploring possible causes. Results The diagnostic results generated by ChatGPT showed an accuracy of 0.73, sensitivity of 0.95, and specificity of 0.58. After few-shot learning, ChatGPT demonstrated significant improvement, achieving an accuracy of 0.87, sensitivity of 0.99, and specificity of 0.73, bringing it much closer to the level of physician diagnostics. In an experiment analyzing the influence of the radiologist's reporting style, the model demonstrated higher sensitivity when interpreting reports written by high-level radiologists. In 56 benign cases, ChatGPT misdiagnosed them as malignant. Among these, 35 benign lesions- fibrous dysplasia and osteofibrous dysplasia- were incorrectly identified as metastatic tumors or osteosarcomas; 8 cases of myositis ossificans were wrongly diagnosed as extraosseous osteosarcoma. 7 cases of giant cell tumor of bone at the end of long bone were misdiagnosed as osteosarcoma by intermediate doctors. Chondroblastoma was misdiagnosed as malignant tumor in 6 cases -2 osteosarcoma and 4 chondrosarcoma-In this study, 23 osteosarcoma cases were misdiagnosed by ChatGPT as osteomyelitis; Chondrosarcoma was misdiagnosed as fibrous dysplasia or aneurysmal bone cyst in 8 cases. Four cases of spinal chordoma were misdiagnosed as spinal tuberculosis. Conclusion Our findings highlight the potential of ChatGPT in the diagnosis of benign and malignant bone tumors, offering advantages like enhanced efficiency and a reduction in missed diagnoses. However, the necessity of collaborative interactions between physicians and ChatGPT in practical settings was underscored. With an examination into AI's capacity in benign and malignant bone tumors diagnosis, this study lays the groundwork for future AI advancements in medicine. Additionally, the benefits of few-shot learning in fine-tuning ChatGPT applications in specialized fields were also demonstrated.
Collapse
Affiliation(s)
- Fan Yang
- Department of Radiation, Beijing Jishuitan Hospital,Capital Medical University, Beijing 100035, China
| | - Dong Yan
- Department of Radiation, Beijing Jishuitan Hospital,Capital Medical University, Beijing 100035, China
| | - Zhixiang Wang
- Department of Ultrasound, Beijing Friendship Hospital, Capital Medical University, Beijing 100050, China
| |
Collapse
|
40
|
Dickinson H, Teltsch DY, Feifel J, Hunt P, Vallejo-Yagüe E, Virkud AV, Muylle KM, Ochi T, Donneyong M, Zabinski J, Strauss VY, Hincapie-Castillo JM. The Unseen Hand: AI-Based Prescribing Decision Support Tools and the Evaluation of Drug Safety and Effectiveness. Drug Saf 2024; 47:117-123. [PMID: 38019365 DOI: 10.1007/s40264-023-01376-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/26/2023] [Indexed: 11/30/2023]
Abstract
The use of artificial intelligence (AI)-based tools to guide prescribing decisions is full of promise and may enhance patient outcomes. These tools can perform actions such as choosing the 'safest' medication, choosing between competing medications, promoting de-prescribing or even predicting non-adherence. These tools can exist in a variety of formats; for example, they may be directly integrated into electronic medical records or they may exist in a stand-alone website accessible by a web browser. One potential impact of these tools is that they could manipulate our understanding of the benefit-risk of medicines in the real world. Currently, the benefit risk of approved medications is assessed according to carefully planned agreements covering spontaneous reporting systems and planned surveillance studies. But AI-based tools may limit or even block prescription to high-risk patients or prevent off-label use. The uptake and temporal availability of these tools may be uneven across healthcare systems and geographies, creating artefacts in data that are difficult to account for. It is also hard to estimate the 'true impact' that a tool had on a prescribing decision. International borders may also be highly porous to these tools, especially in cases where tools are available over the web. These tools already exist, and their use is likely to increase in the coming years. How they can be accounted for in benefit-risk decisions is yet to be seen.
Collapse
Affiliation(s)
| | | | - Jan Feifel
- Merck Healthcare KGaA, Darmstadt, Germany
| | - Philip Hunt
- Institute of Pharmaceutical Sciences, ETH Zurich, Zurich, Switzerland
| | - Enriqueta Vallejo-Yagüe
- AstraZeneca, Gaithersberg, MD, USA
- Institute of Primary Health Care (BIHAM), University of Bern, Bern, Switzerland
| | - Arti V Virkud
- Kidney Center School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | | | - Taichi Ochi
- Department of PharmacoTherapy, Epidemiology and Economics, Groningen Research Institute of Pharmacy, University of Groningen, Groningen, The Netherlands
- Center for Innovation in Medicine, Bucharest, Romania
| | | | | | - Victoria Y Strauss
- Boehringer Ingelheim, Binger Str. 173, 55218, Ingelheim am Rhein, Germany
| | | |
Collapse
|
41
|
Ong JCL, Seng BJJ, Law JZF, Low LL, Kwa ALH, Giacomini KM, Ting DSW. Artificial intelligence, ChatGPT, and other large language models for social determinants of health: Current state and future directions. Cell Rep Med 2024; 5:101356. [PMID: 38232690 PMCID: PMC10829781 DOI: 10.1016/j.xcrm.2023.101356] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Revised: 10/12/2023] [Accepted: 12/10/2023] [Indexed: 01/19/2024]
Abstract
This perspective highlights the importance of addressing social determinants of health (SDOH) in patient health outcomes and health inequity, a global problem exacerbated by the COVID-19 pandemic. We provide a broad discussion on current developments in digital health and artificial intelligence (AI), including large language models (LLMs), as transformative tools in addressing SDOH factors, offering new capabilities for disease surveillance and patient care. Simultaneously, we bring attention to challenges, such as data standardization, infrastructure limitations, digital literacy, and algorithmic bias, that could hinder equitable access to AI benefits. For LLMs, we highlight potential unique challenges and risks including environmental impact, unfair labor practices, inadvertent disinformation or "hallucinations," proliferation of bias, and infringement of copyrights. We propose the need for a multitiered approach to digital inclusion as an SDOH and the development of ethical and responsible AI practice frameworks globally and provide suggestions on bridging the gap from development to implementation of equitable AI technologies.
Collapse
Affiliation(s)
- Jasmine Chiat Ling Ong
- Division of Pharmacy, Singapore General Hospital, Singapore, Singapore; SingHealth Duke-NUS Medicine Academic Clinical Programme, Singapore, Singapore
| | - Benjamin Jun Jie Seng
- MOHH Holdings (Singapore) Pte., Ltd., Singapore, Singapore; SingHealth Duke-NUS Family Medicine Academic Clinical Programme, Singapore, Singapore
| | | | - Lian Leng Low
- SingHealth Duke-NUS Family Medicine Academic Clinical Programme, Singapore, Singapore; Population Health and Integrated Care Office, Singapore General Hospital, Singapore, Singapore; Centre for Population Health Research and Implementation, SingHealth Regional Health System, Singapore, Singapore; Outram Community Hospital, SingHealth Community Hospitals, Singapore, Singapore
| | - Andrea Lay Hoon Kwa
- Division of Pharmacy, Singapore General Hospital, Singapore, Singapore; SingHealth Duke-NUS Medicine Academic Clinical Programme, Singapore, Singapore; Emerging Infectious Diseases, Duke-NUS Medical School, Singapore, Singapore
| | - Kathleen M Giacomini
- Department of Bioengineering and Therapeutic Sciences, Schools of Pharmacy and Medicine, University of California, San Francisco, San Francisco, CA, USA
| | - Daniel Shu Wei Ting
- Artificial Intelligence and Digital Innovation Research Group, Singapore Eye Research, Singapore, Singapore; Duke-NUS Medical School, National University of Singapore, Singapore, Singapore; Byers Eye Institute, Stanford University, Stanford, CA, USA.
| |
Collapse
|
42
|
Bekbolatova M, Mayer J, Ong CW, Toma M. Transformative Potential of AI in Healthcare: Definitions, Applications, and Navigating the Ethical Landscape and Public Perspectives. Healthcare (Basel) 2024; 12:125. [PMID: 38255014 PMCID: PMC10815906 DOI: 10.3390/healthcare12020125] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2023] [Revised: 12/27/2023] [Accepted: 01/02/2024] [Indexed: 01/24/2024] Open
Abstract
Artificial intelligence (AI) has emerged as a crucial tool in healthcare with the primary aim of improving patient outcomes and optimizing healthcare delivery. By harnessing machine learning algorithms, natural language processing, and computer vision, AI enables the analysis of complex medical data. The integration of AI into healthcare systems aims to support clinicians, personalize patient care, and enhance population health, all while addressing the challenges posed by rising costs and limited resources. As a subdivision of computer science, AI focuses on the development of advanced algorithms capable of performing complex tasks that were once reliant on human intelligence. The ultimate goal is to achieve human-level performance with improved efficiency and accuracy in problem-solving and task execution, thereby reducing the need for human intervention. Various industries, including engineering, media/entertainment, finance, and education, have already reaped significant benefits by incorporating AI systems into their operations. Notably, the healthcare sector has witnessed rapid growth in the utilization of AI technology. Nevertheless, there remains untapped potential for AI to truly revolutionize the industry. It is important to note that despite concerns about job displacement, AI in healthcare should not be viewed as a threat to human workers. Instead, AI systems are designed to augment and support healthcare professionals, freeing up their time to focus on more complex and critical tasks. By automating routine and repetitive tasks, AI can alleviate the burden on healthcare professionals, allowing them to dedicate more attention to patient care and meaningful interactions. However, legal and ethical challenges must be addressed when embracing AI technology in medicine, alongside comprehensive public education to ensure widespread acceptance.
Collapse
Affiliation(s)
- Molly Bekbolatova
- Department of Osteopathic Manipulative Medicine, College of Osteopathic Medicine, New York Institute of Technology, Old Westbury, NY 11568, USA; (M.B.); (J.M.)
| | - Jonathan Mayer
- Department of Osteopathic Manipulative Medicine, College of Osteopathic Medicine, New York Institute of Technology, Old Westbury, NY 11568, USA; (M.B.); (J.M.)
| | - Chi Wei Ong
- School of Chemistry, Chemical Engineering, and Biotechnology, Nanyang Technological University, 62 Nanyang Drive, Singapore 637459, Singapore
| | - Milan Toma
- Department of Osteopathic Manipulative Medicine, College of Osteopathic Medicine, New York Institute of Technology, Old Westbury, NY 11568, USA; (M.B.); (J.M.)
| |
Collapse
|
43
|
Mascagni P, Alapatt D, Lapergola A, Vardazaryan A, Mazellier JP, Dallemagne B, Mutter D, Padoy N. Early-stage clinical evaluation of real-time artificial intelligence assistance for laparoscopic cholecystectomy. Br J Surg 2024; 111:znad353. [PMID: 37935636 DOI: 10.1093/bjs/znad353] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2023] [Revised: 07/24/2023] [Accepted: 08/26/2023] [Indexed: 11/09/2023]
Abstract
Lay Summary
The growing availability of surgical digital data and developments in analytics such as artificial intelligence (AI) are being harnessed to improve surgical care. However, technical and cultural barriers to real-time intraoperative AI assistance exist. This early-stage clinical evaluation shows the technical feasibility of concurrently deploying several AIs in operating rooms for real-time assistance during procedures. In addition, potentially relevant clinical applications of these AI models are explored with a multidisciplinary cohort of key stakeholders.
Collapse
Affiliation(s)
- Pietro Mascagni
- ICube, University of Strasbourg, CNRS, IHU Strasbourg, Strasbourg, France
- Department of Medical and Abdominal Surgery and Endocrine-Metabolic Science, Fondazione Policlinico Universitario A. Gemelli IRCCS, Rome, Italy
| | - Deepak Alapatt
- ICube, University of Strasbourg, CNRS, IHU Strasbourg, Strasbourg, France
| | - Alfonso Lapergola
- Department of Digestive and Endocrine Surgery, Nouvel Hôpital Civil, Hôpitaux Universitaires de Strasbourg, Strasbourg, France
| | | | | | - Bernard Dallemagne
- Institute for Research against Digestive Cancer (IRCAD), Strasbourg, France
| | - Didier Mutter
- Department of Digestive and Endocrine Surgery, Nouvel Hôpital Civil, Hôpitaux Universitaires de Strasbourg, Strasbourg, France
- Institute of Image-Guided Surgery, IHU-Strasbourg, Strasbourg, France
| | - Nicolas Padoy
- ICube, University of Strasbourg, CNRS, IHU Strasbourg, Strasbourg, France
- Institute of Image-Guided Surgery, IHU-Strasbourg, Strasbourg, France
| |
Collapse
|
44
|
Badrulhisham F, Pogatzki-Zahn E, Segelcke D, Spisak T, Vollert J. Machine learning and artificial intelligence in neuroscience: A primer for researchers. Brain Behav Immun 2024; 115:470-479. [PMID: 37972877 DOI: 10.1016/j.bbi.2023.11.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/26/2023] [Revised: 10/16/2023] [Accepted: 11/08/2023] [Indexed: 11/19/2023] Open
Abstract
Artificial intelligence (AI) is often used to describe the automation of complex tasks that we would attribute intelligence to. Machine learning (ML) is commonly understood as a set of methods used to develop an AI. Both have seen a recent boom in usage, both in scientific and commercial fields. For the scientific community, ML can solve bottle necks created by complex, multi-dimensional data generated, for example, by functional brain imaging or *omics approaches. ML can here identify patterns that could not have been found using traditional statistic approaches. However, ML comes with serious limitations that need to be kept in mind: their tendency to optimise solutions for the input data means it is of crucial importance to externally validate any findings before considering them more than a hypothesis. Their black-box nature implies that their decisions usually cannot be understood, which renders their use in medical decision making problematic and can lead to ethical issues. Here, we present an introduction for the curious to the field of ML/AI. We explain the principles as commonly used methods as well as recent methodological advancements before we discuss risks and what we see as future directions of the field. Finally, we show practical examples of neuroscience to illustrate the use and limitations of ML.
Collapse
Affiliation(s)
| | - Esther Pogatzki-Zahn
- Department of Anaesthesiology, Intensive Care and Pain Medicine, University Hospital Muenster, Muenster, Germany
| | - Daniel Segelcke
- Department of Anaesthesiology, Intensive Care and Pain Medicine, University Hospital Muenster, Muenster, Germany
| | - Tamas Spisak
- Institute of Diagnostic and Interventional Radiology and Neuroradiology, University Medicine Essen, Essen, Germany; Center for Translational Neuro- and Behavioral Sciences, Department of Neurology, University Medicine Essen, Essen, Germany
| | - Jan Vollert
- Department of Clinical and Biomedical Sciences, Faculty of Health and Life Sciences, University of Exeter, Exeter, United Kingdom; Pain Research, Department of Surgery and Cancer, Imperial College London, London, United Kingdom.
| |
Collapse
|
45
|
Föllmer B, Williams MC, Dey D, Arbab-Zadeh A, Maurovich-Horvat P, Volleberg RHJA, Rueckert D, Schnabel JA, Newby DE, Dweck MR, Guagliumi G, Falk V, Vázquez Mézquita AJ, Biavati F, Išgum I, Dewey M. Roadmap on the use of artificial intelligence for imaging of vulnerable atherosclerotic plaque in coronary arteries. Nat Rev Cardiol 2024; 21:51-64. [PMID: 37464183 DOI: 10.1038/s41569-023-00900-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 06/07/2023] [Indexed: 07/20/2023]
Abstract
Artificial intelligence (AI) is likely to revolutionize the way medical images are analysed and has the potential to improve the identification and analysis of vulnerable or high-risk atherosclerotic plaques in coronary arteries, leading to advances in the treatment of coronary artery disease. However, coronary plaque analysis is challenging owing to cardiac and respiratory motion, as well as the small size of cardiovascular structures. Moreover, the analysis of coronary imaging data is time-consuming, can be performed only by clinicians with dedicated cardiovascular imaging training, and is subject to considerable interreader and intrareader variability. AI has the potential to improve the assessment of images of vulnerable plaque in coronary arteries, but requires robust development, testing and validation. Combining human expertise with AI might facilitate the reliable and valid interpretation of images obtained using CT, MRI, PET, intravascular ultrasonography and optical coherence tomography. In this Roadmap, we review existing evidence on the application of AI to the imaging of vulnerable plaque in coronary arteries and provide consensus recommendations developed by an interdisciplinary group of experts on AI and non-invasive and invasive coronary imaging. We also outline future requirements of AI technology to address bias, uncertainty, explainability and generalizability, which are all essential for the acceptance of AI and its clinical utility in handling the anticipated growing volume of coronary imaging procedures.
Collapse
Affiliation(s)
- Bernhard Föllmer
- Department of Radiology, Charité - Universitätsmedizin Berlin, Berlin, Germany.
| | | | - Damini Dey
- Biomedical Imaging Research Institute and Department of Imaging, Medicine and Biomedical Sciences, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Armin Arbab-Zadeh
- Division of Cardiology, Department of Medicine, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Pál Maurovich-Horvat
- Department of Radiology, Medical Imaging Center, Semmelweis University, Budapest, Hungary
| | - Rick H J A Volleberg
- Department of Cardiology, Radboud University Medical Center, Nijmegen, Netherlands
| | - Daniel Rueckert
- Artificial Intelligence in Medicine and Healthcare, Technical University of Munich, Munich, Germany
- Department of Computing, Imperial College London, London, UK
| | - Julia A Schnabel
- School of Biomedical Imaging and Imaging Sciences, King's College London, London, UK
- Institute of Machine Learning in Biomedical Imaging, Helmholtz Munich, Neuherberg, Germany
- School of Computation, Information and Technology, Technical University of Munich, Munich, Germany
| | - David E Newby
- Centre for Cardiovascular Science, University of Edinburgh, Edinburgh, UK
| | - Marc R Dweck
- Centre for Cardiovascular Science, University of Edinburgh, Edinburgh, UK
| | - Giulio Guagliumi
- Division of Cardiology, IRCCS Galeazzi Sant'Ambrogio Hospital, Milan, Italy
| | - Volkmar Falk
- Department of Cardiothoracic and Vascular Surgery, Deutsches Herzzentrum der Charité, Charité Universitätsmedizin, Berlin, Germany
- Department of Health Science and Technology, ETH Zurich, Zurich, Switzerland
- Berlin Institute of Health at Charité and DZHK (German Centre for Cardiovascular Research), Partner Site, Berlin, Germany
| | | | - Federico Biavati
- Department of Radiology, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Ivana Išgum
- Department of Biomedical Engineering and Physics, Amsterdam University Medical Center, University of Amsterdam, Amsterdam, Netherlands
- Department of Radiology and Nuclear Medicine, Amsterdam UMC, University of Amsterdam, Amsterdam, Netherlands
- Informatics Institute, Faculty of Science, University of Amsterdam, Amsterdam, Netherlands
| | - Marc Dewey
- Department of Radiology, Charité - Universitätsmedizin Berlin, Berlin, Germany.
- Berlin Institute of Health, Campus Charité Mitte, Berlin, Germany.
- DZHK (German Centre for Cardiovascular Research), Partner Site Berlin and Deutsches Herzzentrum der Charité (DHZC), Charité - Universitätsmedizin Berlin, Berlin, Germany.
| |
Collapse
|
46
|
O'Reilly D, McGrath J, Martin-Loeches I. Optimizing artificial intelligence in sepsis management: Opportunities in the present and looking closely to the future. JOURNAL OF INTENSIVE MEDICINE 2024; 4:34-45. [PMID: 38263963 PMCID: PMC10800769 DOI: 10.1016/j.jointm.2023.10.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 04/22/2023] [Revised: 09/29/2023] [Accepted: 10/01/2023] [Indexed: 01/25/2024]
Abstract
Sepsis remains a major challenge internationally for healthcare systems. Its incidence is rising due to poor public awareness and delays in its recognition and subsequent management. In sepsis, mortality increases with every hour left untreated. Artificial intelligence (AI) is transforming worldwide healthcare delivery at present. This review has outlined how AI can augment strategies to address this global disease burden. AI and machine learning (ML) algorithms can analyze vast quantities of increasingly complex clinical datasets from electronic medical records to assist clinicians in diagnosing and treating sepsis earlier than traditional methods. Our review highlights how these models can predict the risk of sepsis and organ failure even before it occurs. This gives providers additional time to plan and execute treatment plans, thereby avoiding increasing complications associated with delayed diagnosis of sepsis. The potential for cost savings with AI implementation is also discussed, including improving workflow efficiencies, reducing administrative costs, and improving healthcare outcomes. Despite these advantages, clinicians have been slow to adopt AI into clinical practice. Some of the limitations posed by AI solutions include the lack of diverse data sets for model building so that they are widely applicable for routine clinical use. Furthermore, the subsequent algorithms are often based on complex mathematics leading to clinician hesitancy to embrace such technologies. Finally, we highlight the need for robust political and regulatory frameworks in this area to achieve the trust and approval of clinicians and patients to implement this transformational technology.
Collapse
Affiliation(s)
- Darragh O'Reilly
- Department of Intensive Care Medicine, Multidisciplinary Intensive Care Research Organization (MICRO), St James’ Hospital, Dublin, Ireland
| | - Jennifer McGrath
- Department of Intensive Care Medicine, Multidisciplinary Intensive Care Research Organization (MICRO), St James’ Hospital, Dublin, Ireland
| | - Ignacio Martin-Loeches
- Department of Intensive Care Medicine, Multidisciplinary Intensive Care Research Organization (MICRO), St James’ Hospital, Dublin, Ireland
- Department of Respiratory Intensive care, Hospital Clinic, Universitat de Barcelona, IDIBAPS, CIBERES, Barcelona, Spain
| |
Collapse
|
47
|
Cacciamani GE, Chen A, Gill IS, Hung AJ. Artificial intelligence and urology: ethical considerations for urologists and patients. Nat Rev Urol 2024; 21:50-59. [PMID: 37524914 DOI: 10.1038/s41585-023-00796-1] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 06/22/2023] [Indexed: 08/02/2023]
Abstract
The use of artificial intelligence (AI) in medicine and in urology specifically has increased over the past few years, during which time it has enabled optimization of patient workflow, increased diagnostic accuracy and enhanced computer analysis of radiological and pathological images. However, before further use of AI is undertaken, possible ethical issues need to be evaluated to improve understanding of this technology and to protect patients and providers. Possible ethical issues that require consideration when applying AI in clinical practice include patient safety, cybersecurity, transparency and interpretability of the data, inclusivity and equity, fostering responsibility and accountability, and the preservation of providers' decision-making and autonomy. Ethical principles for the application of AI to health care and in urology are proposed to guide urologists, patients and regulators to improve use of AI technologies and guide policy-making.
Collapse
Affiliation(s)
- Giovanni E Cacciamani
- The Catherine and Joseph Aresty Department of Urology, USC Institute of Urology, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA.
- AI Center at USC Urology, USC Institute of Urology, University of Southern California, Los Angeles, CA, USA.
- Department of Radiology, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA.
| | - Andrew Chen
- The Catherine and Joseph Aresty Department of Urology, USC Institute of Urology, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA
- AI Center at USC Urology, USC Institute of Urology, University of Southern California, Los Angeles, CA, USA
| | - Inderbir S Gill
- The Catherine and Joseph Aresty Department of Urology, USC Institute of Urology, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA
- AI Center at USC Urology, USC Institute of Urology, University of Southern California, Los Angeles, CA, USA
| | - Andrew J Hung
- The Catherine and Joseph Aresty Department of Urology, USC Institute of Urology, Keck School of Medicine, University of Southern California, Los Angeles, CA, USA
- AI Center at USC Urology, USC Institute of Urology, University of Southern California, Los Angeles, CA, USA
| |
Collapse
|
48
|
Truong ATL, Tan SB, Wang GZ, Yip AWJ, Egermark M, Yeung W, Lee VV, Chan MY, Kumar KS, Tan LWJ, Vijayakumar S, Blasiak A, Wang LYT, Ho D. CURATE.AI-assisted dose titration for anti-hypertensive personalized therapy: study protocol for a multi-arm, randomized, pilot feasibility trial using CURATE.AI (CURATE.AI ADAPT trial). EUROPEAN HEART JOURNAL. DIGITAL HEALTH 2024; 5:41-49. [PMID: 38264697 PMCID: PMC10802822 DOI: 10.1093/ehjdh/ztad063] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/10/2023] [Revised: 08/11/2023] [Accepted: 09/06/2023] [Indexed: 01/25/2024]
Abstract
Aims Artificial intelligence-driven small data platforms such as CURATE.AI hold potential for personalized hypertension care by assisting physicians in identifying personalized anti-hypertensive doses for titration. This trial aims to assess the feasibility of a larger randomized controlled trial (RCT), evaluating the efficacy of CURATE.AI-assisted dose titration intervention. We will also collect preliminary efficacy and safety data and explore stakeholder feedback in the early design process. Methods and results In this open-label, randomized, pilot feasibility trial, we aim to recruit 45 participants with primary hypertension. Participants will be randomized in 1:1:1 ratio into control (no intervention), home blood pressure monitoring (active control; HBPM), or CURATE.AI arms (intervention; HBPM and CURATE.AI-assisted dose titration). The home treatments include 1 month of two-drug anti-hypertensive regimens. Primary endpoints assess the logistical (e.g. dose adherence) and scientific (e.g. percentage of participants for which CURATE.AI profiles can be generated) feasibility, and define the progression criteria for the RCT in a 'traffic light system'. Secondary endpoints assess preliminary efficacy [e.g. mean change in office blood pressures (BPs)] and safety (e.g. hospitalization events) associated with each treatment protocol. Participants with both baseline and post-treatment BP measurements will form the intent-to-treat analysis. Following their involvement with the CURATE.AI intervention, feedback from CURATE.AI participants and healthcare providers will be collected via exit survey and interviews. Conclusion Findings from this study will inform about potential refinements of the current treatment protocols before proceeding with a larger RCT, or potential expansion to collect additional information. Positive results may suggest the potential efficacy of CURATE.AI to improve BP control. Trial registration number NCT05376683.
Collapse
Affiliation(s)
- Anh T L Truong
- The N.1 Institute for Health (N.1), National University of Singapore, 28 Medical Drive, Singapore 117456, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, 4 Engineering Drive 3, Singapore 117583, Singapore
- The Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore 117456, Singapore
| | - Shi-Bei Tan
- The N.1 Institute for Health (N.1), National University of Singapore, 28 Medical Drive, Singapore 117456, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, 4 Engineering Drive 3, Singapore 117583, Singapore
- The Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore 117456, Singapore
| | - Golda Z Wang
- Department of Pharmacy, Alexandra Hospital, Singapore 15996, Singapore
| | - Alexander W J Yip
- Department of Medicine, Alexandra Hospital, Singapore 159964, Singapore
- Department of Healthcare Redesign, Alexandra Hospital, Singapore 159964, Singapore
| | - Mathias Egermark
- The N.1 Institute for Health (N.1), National University of Singapore, 28 Medical Drive, Singapore 117456, Singapore
| | - Wesley Yeung
- National University Heart Centre, National University Hospital Singapore, Singapore 119074, Singapore
| | - V Vien Lee
- The N.1 Institute for Health (N.1), National University of Singapore, 28 Medical Drive, Singapore 117456, Singapore
- The Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore 117456, Singapore
| | - Mark Y Chan
- National University Heart Centre, National University Hospital Singapore, Singapore 119074, Singapore
| | - Kirthika S Kumar
- The N.1 Institute for Health (N.1), National University of Singapore, 28 Medical Drive, Singapore 117456, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, 4 Engineering Drive 3, Singapore 117583, Singapore
- The Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore 117456, Singapore
| | - Lester W J Tan
- The N.1 Institute for Health (N.1), National University of Singapore, 28 Medical Drive, Singapore 117456, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, 4 Engineering Drive 3, Singapore 117583, Singapore
- The Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore 117456, Singapore
| | - Smrithi Vijayakumar
- The N.1 Institute for Health (N.1), National University of Singapore, 28 Medical Drive, Singapore 117456, Singapore
- The Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore 117456, Singapore
| | - Agata Blasiak
- The N.1 Institute for Health (N.1), National University of Singapore, 28 Medical Drive, Singapore 117456, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, 4 Engineering Drive 3, Singapore 117583, Singapore
- The Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore 117456, Singapore
- Department of Pharmacology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore 117600, Singapore
| | - Laureen Y T Wang
- Department of Medicine, Alexandra Hospital, Singapore 159964, Singapore
- Department of Healthcare Redesign, Alexandra Hospital, Singapore 159964, Singapore
- National University Heart Centre, National University Hospital Singapore, Singapore 119074, Singapore
| | - Dean Ho
- The N.1 Institute for Health (N.1), National University of Singapore, 28 Medical Drive, Singapore 117456, Singapore
- Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, 4 Engineering Drive 3, Singapore 117583, Singapore
- The Institute for Digital Medicine (WisDM), Yong Loo Lin School of Medicine, National University of Singapore, Singapore 117456, Singapore
- Department of Pharmacology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore 117600, Singapore
- The Bia-Echo Asia Centre for Reproductive Longevity and Equality, Yong Loo Lin School of Medicine, National University of Singapore, Singapore 117456, Singapore
- Health District @ Queenstown, Yong Loo Lin School of Medicine, National University of Singapore, Singapore 119228, Singapore
| |
Collapse
|
49
|
Soh ZD, Tan M, Nongpiur ME, Xu BY, Friedman D, Zhang X, Leung C, Liu Y, Koh V, Aung T, Cheng CY. Assessment of angle closure disease in the age of artificial intelligence: A review. Prog Retin Eye Res 2024; 98:101227. [PMID: 37926242 DOI: 10.1016/j.preteyeres.2023.101227] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2023] [Revised: 11/02/2023] [Accepted: 11/02/2023] [Indexed: 11/07/2023]
Abstract
Primary angle closure glaucoma is a visually debilitating disease that is under-detected worldwide. Many of the challenges in managing primary angle closure disease (PACD) are related to the lack of convenient and precise tools for clinic-based disease assessment and monitoring. Artificial intelligence (AI)- assisted tools to detect and assess PACD have proliferated in recent years with encouraging results. Machine learning (ML) algorithms that utilize clinical data have been developed to categorize angle closure eyes by disease mechanism. Other ML algorithms that utilize image data have demonstrated good performance in detecting angle closure. Nonetheless, deep learning (DL) algorithms trained directly on image data generally outperformed traditional ML algorithms in detecting PACD, were able to accurately differentiate between angle status (open, narrow, closed), and automated the measurement of quantitative parameters. However, more work is required to expand the capabilities of these AI algorithms and for deployment into real-world practice settings. This includes the need for real-world evaluation, establishing the use case for different algorithms, and evaluating the feasibility of deployment while considering other clinical, economic, social, and policy-related factors.
Collapse
Affiliation(s)
- Zhi Da Soh
- Singapore Eye Research Institute, Singapore National Eye Centre, 20 College Road, 169856, Singapore; Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore, 21 Lower Kent Ridge Road, 119077, Singapore.
| | - Mingrui Tan
- Institute of High Performance Computing, Agency for Science, Technology and Research (A*Star), 1 Fusionopolis Way, 138632, Singapore.
| | - Monisha Esther Nongpiur
- Singapore Eye Research Institute, Singapore National Eye Centre, 20 College Road, 169856, Singapore; Ophthalmology & Visual Sciences Academic Clinical Programme, Academic Medicine, Duke-NUS Medical School, 8 College Road, 169857, Singapore.
| | - Benjamin Yixing Xu
- Roski Eye Institute, Keck School of Medicine, University of Southern California, 1450 San Pablo St #4400, Los Angeles, CA, 90033, USA.
| | - David Friedman
- Department of Ophthalmology, Harvard Medical School, 25 Shattuck Street, Boston, MA, 02115, USA; Massachusetts Eye and Ear, Mass General Brigham, 243 Charles Street, Boston, MA, 02114, USA.
| | - Xiulan Zhang
- State Key Laboratory of Ophthalmology, Zhongshan Ophthalmic Center, Sun Yat Sen University, No. 54 Xianlie South Road, Yuexiu District, Guangzhou, China.
| | - Christopher Leung
- Department of Ophthalmology, School of Clinical Medicine, The University of Hong Kong, Cyberport 4, 100 Cyberport Road, Hong Kong; Department of Ophthalmology, Queen Mary Hospital, 102 Pok Fu Lam Road, Hong Kong.
| | - Yong Liu
- Institute of High Performance Computing, Agency for Science, Technology and Research (A*Star), 1 Fusionopolis Way, 138632, Singapore.
| | - Victor Koh
- Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore, 21 Lower Kent Ridge Road, 119077, Singapore; Centre for Innovation and Precision Eye Health, Yong Loo Lin School of Medicine, National University of Singapore, 1E Kent Ridge Road, NUHS Tower Block, Level 7, 119228, Singapore.
| | - Tin Aung
- Singapore Eye Research Institute, Singapore National Eye Centre, 20 College Road, 169856, Singapore; Ophthalmology & Visual Sciences Academic Clinical Programme, Academic Medicine, Duke-NUS Medical School, 8 College Road, 169857, Singapore.
| | - Ching-Yu Cheng
- Singapore Eye Research Institute, Singapore National Eye Centre, 20 College Road, 169856, Singapore; Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore, 21 Lower Kent Ridge Road, 119077, Singapore; Ophthalmology & Visual Sciences Academic Clinical Programme, Academic Medicine, Duke-NUS Medical School, 8 College Road, 169857, Singapore; Centre for Innovation and Precision Eye Health, Yong Loo Lin School of Medicine, National University of Singapore, 1E Kent Ridge Road, NUHS Tower Block, Level 7, 119228, Singapore.
| |
Collapse
|
50
|
Welzel C, Cotte F, Wekenborg M, Vasey B, McCulloch P, Gilbert S. Holistic Human-Serving Digitization of Health Care Needs Integrated Automated System-Level Assessment Tools. J Med Internet Res 2023; 25:e50158. [PMID: 38117545 PMCID: PMC10765286 DOI: 10.2196/50158] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2023] [Revised: 09/01/2023] [Accepted: 10/26/2023] [Indexed: 12/21/2023] Open
Abstract
Digital health tools, platforms, and artificial intelligence- or machine learning-based clinical decision support systems are increasingly part of health delivery approaches, with an ever-greater degree of system interaction. Critical to the successful deployment of these tools is their functional integration into existing clinical routines and workflows. This depends on system interoperability and on intuitive and safe user interface design. The importance of minimizing emergent workflow stress through human factors research and purposeful design for integration cannot be overstated. Usability of tools in practice is as important as algorithm quality. Regulatory and health technology assessment frameworks recognize the importance of these factors to a certain extent, but their focus remains mainly on the individual product rather than on emergent system and workflow effects. The measurement of performance and user experience has so far been performed in ad hoc, nonstandardized ways by individual actors using their own evaluation approaches. We propose that a standard framework for system-level and holistic evaluation could be built into interacting digital systems to enable systematic and standardized system-wide, multiproduct, postmarket surveillance and technology assessment. Such a system could be made available to developers through regulatory or assessment bodies as an application programming interface and could be a requirement for digital tool certification, just as interoperability is. This would enable health systems and tool developers to collect system-level data directly from real device use cases, enabling the controlled and safe delivery of systematic quality assessment or improvement studies suitable for the complexity and interconnectedness of clinical workflows using developing digital health technologies.
Collapse
Affiliation(s)
- Cindy Welzel
- Else Kröner Fresenius Center for Digital Health, TUD Dresden University of Technology, Dresden, Germany
| | | | - Magdalena Wekenborg
- Else Kröner Fresenius Center for Digital Health, TUD Dresden University of Technology, Dresden, Germany
| | - Baptiste Vasey
- Nuffield Department of Surgical Sciences, University of Oxford, Oxford, United Kingdom
| | - Peter McCulloch
- Nuffield Department of Surgical Sciences, University of Oxford, Oxford, United Kingdom
| | - Stephen Gilbert
- Else Kröner Fresenius Center for Digital Health, TUD Dresden University of Technology, Dresden, Germany
| |
Collapse
|