1
|
Ebad SA, Alhashmi A, Amara M, Miled AB, Saqib M. Artificial Intelligence-Based Software as a Medical Device (AI-SaMD): A Systematic Review. Healthcare (Basel) 2025; 13:817. [PMID: 40218113 PMCID: PMC11988595 DOI: 10.3390/healthcare13070817] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2025] [Revised: 03/28/2025] [Accepted: 03/31/2025] [Indexed: 04/14/2025] Open
Abstract
Background/Objectives: Artificial intelligence-based software as a medical device (AI-SaMD) refers to AI-powered software used for medical purposes without being embedded in physical devices. Despite increasing approvals over the past decade, research in this domain-spanning technology, healthcare, and national security-remains limited. This research aims to bridge the existing research gap in AI-SaMD by systematically reviewing the literature from the past decade, with the aim of classifying key findings, identifying critical challenges, and synthesizing insights related to technological, clinical, and regulatory aspects of AI-SaMD. Methods: A systematic literature review based on the PRISMA framework was performed to select the relevant AI-SaMD studies published between 2015 and 2024 in order to uncover key themes such as publication venues, geographical trends, key challenges, and research gaps. Results: Most studies focus on specialized clinical settings like radiology and ophthalmology rather than general clinical practice. Key challenges to implement AI-SaMD include regulatory issues (e.g., regulatory frameworks), AI malpractice (e.g., explainability and expert oversight), and data governance (e.g., privacy and data sharing). Existing research emphasizes the importance of (1) addressing the regulatory problems through the specific duties of regulatory authorities, (2) interdisciplinary collaboration, (3) clinician training, (4) the seamless integration of AI-SaMD with healthcare software systems (e.g., electronic health records), and (5) the rigorous validation of AI-SaMD models to ensure effective implementation. Conclusions: This study offers valuable insights for diverse stakeholders, emphasizing the need to move beyond theoretical analyses and prioritize practical, experimental research to advance the real-world application of AI-SaMDs. This study concludes by outlining future research directions and emphasizing the limitations of the predominantly theoretical approaches currently available.
Collapse
Affiliation(s)
- Shouki A. Ebad
- Center for Scientific Research and Entrepreneurship, Northern Border University, Arar 73213, Saudi Arabia
| | - Asma Alhashmi
- Department of Computer Science, Faculty of Science, Northern Border University, Arar 73213, Saudi Arabia (M.A.)
| | - Marwa Amara
- Department of Computer Science, Faculty of Science, Northern Border University, Arar 73213, Saudi Arabia (M.A.)
| | - Achraf Ben Miled
- Department of Computer Science, Faculty of Science, Northern Border University, Arar 73213, Saudi Arabia (M.A.)
| | - Muhammad Saqib
- Applied College, Northern Border University, Arar 73213, Saudi Arabia
| |
Collapse
|
2
|
Dreizin D, Khatri G, Staziaki PV, Buch K, Unberath M, Mohammed M, Sodickson A, Khurana B, Agrawal A, Spann JS, Beckmann N, DelProposto Z, LeBedis CA, Davis M, Dickerson G, Lev M. Artificial intelligence in emergency and trauma radiology: ASER AI/ML expert panel Delphi consensus statement on research guidelines, practices, and priorities. Emerg Radiol 2025; 32:155-172. [PMID: 39714735 DOI: 10.1007/s10140-024-02306-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2024] [Accepted: 12/06/2024] [Indexed: 12/24/2024]
Abstract
BACKGROUND Emergency/trauma radiology artificial intelligence (AI) is maturing along all stages of technology readiness, with research and development (R&D) ranging from data curation and algorithm development to post-market monitoring and retraining. PURPOSE To develop an expert consensus document on best research practices and methodological priorities for emergency/trauma radiology AI. METHODS A Delphi consensus exercise was conducted by the ASER AI/ML expert panel between 2022-2024. In phase 1, a steering committee (7 panelists) established key themes- curation; validity; human factors; workflow; barriers; future avenues; and ethics- and generated an edited, collated long-list of statements. In phase 2, two Delphi rounds using anonymous RAND/UCLA Likert grading were conducted with web-based data capture (round 1) and a bespoke excel document with literature hyperlinks (round 2). Between rounds, editing and knowledge synthesis helped maximize consensus. Statements reaching ≥80% agreement were included in the final document. RESULTS Delphi rounds 1 and 2 consisted of 81 and 78 items, respectively.18/21 expert panelists (86%) responded to round 1, and 15 to round 2 (17% drop-out). Consensus was reached for 65 statements. Observations were summarized and contextualized. Statements with unanimous consensus centered around transparent methodologic reporting; testing for generalizability and robustness with external data; and benchmarking performance with appropriate metrics and baselines. A manuscript draft was circulated to panelists for editing and final approval. CONCLUSIONS The document is meant as a framework to foster best-practices and further discussion among researchers working on various aspects of emergency and trauma radiology AI.
Collapse
Affiliation(s)
- David Dreizin
- Emergency and Trauma Imaging, Department of Diagnostic Radiology and Nuclear Medicine, R Adams Cowley Shock Trauma Center, University of Maryland School of Medicine, Baltimore, MD, USA.
| | - Garvit Khatri
- Abdominal Imaging, Department of Radiology, University of Colorado, Denver, CO, USA
| | - Pedro V Staziaki
- Cardiothoracic imaging, Department of Radiology, University of Vermont, Larner College of Medicine, Burlington, USA
| | - Karen Buch
- Neuroradiology imaging, Department of Radiology, Massachusetts General Hospital, Boston, MA, USA
| | | | - Mohammed Mohammed
- Abdominal imaging, Department of Radiology, King Faisal Specialist Hospital and Research Center, Riyadh, Saudi Arabia
| | - Aaron Sodickson
- Mass General Brigham Enterprise Emergency Radiology, Department of Radiology, Brigham and Women's Hospital, Boston, MA, USA
| | - Bharti Khurana
- Trauma Imaging Research and innovation Center, Department of Radiology, Brigham and Women's Hospital, Boston, MA, USA
| | - Anjali Agrawal
- Department of Radiology, Teleradiology Solutions, Delhi, India
| | - James Stephen Spann
- Department of Radiology, University of Alabama at Birmingham Heersink School of Medicine, Birmingham, AL, USA
| | | | - Zachary DelProposto
- Division of Emergency Radiology, Department of Radiology, University of Michigan, Ann Arbor, MI, USA
| | | | - Melissa Davis
- Department of Radiology, Yale University, New Haven, CT, USA
| | | | - Michael Lev
- Emergency Radiology, Department of Radiology, Massachusetts General Hospial, Boston, USA
| |
Collapse
|
3
|
Ienghong K, Cheung LW, Gaysonsiri D, Apiratwarakul K. The diagnostic performance of automatic B-lines detection for evaluating pulmonary edema in the emergency department among novice point-of-care ultrasound practitioners. Emerg Radiol 2025; 32:241-246. [PMID: 39951213 PMCID: PMC11976347 DOI: 10.1007/s10140-025-02319-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2024] [Accepted: 02/04/2025] [Indexed: 03/02/2025]
Abstract
PURPOSE B-lines in lung ultrasound have been a critical clue for detecting pulmonary edema. However, distinguishing B-lines from other artifacts is a challenge, especially for novice point of care ultrasound (POCUS) practitioners. This study aimed to determine the efficacy of automatic detection of B-lines using artificial intelligence (Auto B-lines) for detecting pulmonary edema. METHODS A retrospective study was conducted on dyspnea patients treated at the emergency department between January 2023 and June 2024. Ultrasound documentation and electronic emergency department medical records were evaluated for sensitivity, specificity, positive likelihood ratio, and negative likelihood ratio of auto B-lines in detection of pulmonary edema. RESULTS Sixty-six patients with a final diagnosis of pulmonary edema were enrolled, with 54.68% having positive B-lines in lung ultrasound. Auto B-lines had 95.6% sensitivity (95% confidence interval [CI]: 0.92-0.98) and 77.2% specificity (95% CI: 0.74-0.80). Physicians demonstrated 82.7% sensitivity (95% CI: 0.79-0.97) and 63.09% sensitivity (95% CI: 0.58-0.69). CONCLUSION The auto B-lines were highly sensitive in diagnosing pulmonary edema in novice POCUS practitioners. The clinical integration of physicians and artificial intelligence enhances diagnostic capabilities.
Collapse
Affiliation(s)
- Kamonwon Ienghong
- Department of Emergency Medicine, Faculty of Medicine, Khon Kaen University, Khon Kaen, 40002, Thailand
| | - Lap Woon Cheung
- Accident & Emergency Department, Princess Margaret Hospital, Kowloon, Hong Kong, China
- Department of Emergency Medicine, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Pokfulam, Hong Kong, China
| | - Dhanu Gaysonsiri
- Department of Pharmacology, Faculty of Medicine, Khon Kaen University, Khon Kaen, Thailand
| | - Korakot Apiratwarakul
- Department of Emergency Medicine, Faculty of Medicine, Khon Kaen University, Khon Kaen, 40002, Thailand.
| |
Collapse
|
4
|
Dreizin D, Cheng CT, Liao CH, Jindal A, Colak E. Artificial intelligence for abdominopelvic trauma imaging: trends, gaps, and future directions. Abdom Radiol (NY) 2025:10.1007/s00261-025-04816-z. [PMID: 40116889 DOI: 10.1007/s00261-025-04816-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2024] [Revised: 01/13/2025] [Accepted: 01/17/2025] [Indexed: 03/23/2025]
Abstract
Abdominopelvic trauma is a major cause of morbidity and mortality, typically resulting from high-energy mechanisms such as motor vehicle collisions and penetrating injuries. Admission abdominopelvic trauma CT, performed either selectively or as part of a whole-body CT protocol, has become the workhorse screening and surgical planning modality due to improvements in speed and image quality. Radiography remains an essential element of the secondary trauma survey, and Focused Assessment with Sonography for Trauma (FAST) scanning has added value for quick assessment of non-compressible hemorrhage in hemodynamically unstable patients. Complex and severe polytrauma cases often delay radiology report turnaround times, which can potentially impede urgent clinical decision-making. Artificial intelligence (AI) computer-aided detection and diagnosis (CAD) offers promising solutions for enhanced diagnostic efficiency and accuracy in abdominopelvic trauma imaging. Although commercial AI tools for abdominopelvic trauma are currently available for only a few use cases, the literature reveals robust research and development (R&D) of prototype tools. Multiscale convolutional neural networks (CNNs) and transformer-based models are capable of detecting and quantifying solid organ injuries, fractures, and hemorrhage with a high degree of precision. Further, generalist foundation models such as multimodal vision-language models (VLMs) can be adapted and fine-tuned using imaging, clinical, and text data for a range of tasks, including detection, quantitative visualization, prognostication, and report auto-generation. Despite their promise, for most use cases in abdominopelvic trauma, AI CAD tools remain in the pilot stages of technology readiness, with persistent challenges related to data availability; the need for open-access PACS compatible software pipelines for pre-clinical shadow-testing; lack of well-designed multi-institutional validation studies; and regulatory hurdles. This narrative review provides a snapshot of the current state of AI in abdominopelvic trauma, examining existing commercial tools; research and development throughout the technology readiness pipeline; and future directions in this domain.
Collapse
Affiliation(s)
- David Dreizin
- University of Maryland Trauma Radiology AI Laboratory (TRAIL), University of Maryland School of Medicine, Baltimore, USA
- Department of Diagnostic Radiology and Nuclear Medicine, R Adams Cowley Shock Trauma Center, University of Maryland School of Medicine, Baltimore, USA
| | - Chi-Tung Cheng
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital Linkou, Linkou, Taoyuan, Taiwan
| | - Chien-Hung Liao
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital Linkou, Linkou, Taoyuan, Taiwan
| | - Ankush Jindal
- University of Maryland Trauma Radiology AI Laboratory (TRAIL), University of Maryland School of Medicine, Baltimore, USA
| | - Errol Colak
- Department of Medical Imaging, University of Toronto, Toronto, Canada.
- Department of Medical Imaging, St. Michael's Hospital, Unity Health Toronto, Toronto, Canada.
| |
Collapse
|
5
|
Cheng CT, Ooyang CH, Liao CH, Kang SC. Applications of deep learning in trauma radiology: A narrative review. Biomed J 2025; 48:100743. [PMID: 38679199 PMCID: PMC11751421 DOI: 10.1016/j.bj.2024.100743] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2023] [Revised: 03/26/2024] [Accepted: 04/24/2024] [Indexed: 05/01/2024] Open
Abstract
Diagnostic imaging is essential in modern trauma care for initial evaluation and identifying injuries requiring intervention. Deep learning (DL) has become mainstream in medical image analysis and has shown promising efficacy for classification, segmentation, and lesion detection. This narrative review provides the fundamental concepts for developing DL algorithms in trauma imaging and presents an overview of current progress in each modality. DL has been applied to detect free fluid on Focused Assessment with Sonography for Trauma (FAST), traumatic findings on chest and pelvic X-rays, and computed tomography (CT) scans, identify intracranial hemorrhage on head CT, detect vertebral fractures, and identify injuries to organs like the spleen, liver, and lungs on abdominal and chest CT. Future directions involve expanding dataset size and diversity through federated learning, enhancing model explainability and transparency to build clinician trust, and integrating multimodal data to provide more meaningful insights into traumatic injuries. Though some commercial artificial intelligence products are Food and Drug Administration-approved for clinical use in the trauma field, adoption remains limited, highlighting the need for multi-disciplinary teams to engineer practical, real-world solutions. Overall, DL shows immense potential to improve the efficiency and accuracy of trauma imaging, but thoughtful development and validation are critical to ensure these technologies positively impact patient care.
Collapse
Affiliation(s)
- Chi-Tung Cheng
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Taoyuan, Taiwan; School of Medicine, Chang Gung University, Taoyuan, Taiwan
| | - Chun-Hsiang Ooyang
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Taoyuan, Taiwan
| | - Chien-Hung Liao
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Taoyuan, Taiwan
| | - Shih-Ching Kang
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Taoyuan, Taiwan.
| |
Collapse
|
6
|
Yao J, Chu LC, Patlas M. Applications of Artificial Intelligence in Acute Abdominal Imaging. Can Assoc Radiol J 2024; 75:761-770. [PMID: 38715249 DOI: 10.1177/08465371241250197] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/12/2024] Open
Abstract
Artificial intelligence (AI) is a rapidly growing field with significant implications for radiology. Acute abdominal pain is a common clinical presentation that can range from benign conditions to life-threatening emergencies. The critical nature of these situations renders emergent abdominal imaging an ideal candidate for AI applications. CT, radiographs, and ultrasound are the most common modalities for imaging evaluation of these patients. For each modality, numerous studies have assessed the performance of AI models for detecting common pathologies, such as appendicitis, bowel obstruction, and cholecystitis. The capabilities of these models range from simple classification to detailed severity assessment. This narrative review explores the evolution, trends, and challenges in AI applications for evaluating acute abdominal pathologies. We review implementations of AI for non-traumatic and traumatic abdominal pathologies, with discussion of potential clinical impact, challenges, and future directions for the technology.
Collapse
Affiliation(s)
- Jason Yao
- Department of Radiology, McMaster University, Hamilton, ON, Canada
| | - Linda C Chu
- Department of Radiology, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Michael Patlas
- Department of Medical Imaging, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
7
|
Johansson JV, Engström E. 'Humans think outside the pixels' - Radiologists' perceptions of using artificial intelligence for breast cancer detection in mammography screening in a clinical setting. Health Informatics J 2024; 30:14604582241275020. [PMID: 39155239 DOI: 10.1177/14604582241275020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/20/2024]
Abstract
OBJECTIVE This study aimed to explore radiologists' views on using an artificial intelligence (AI) tool named ScreenTrustCAD with Philips equipment) as a diagnostic decision support tool in mammography screening during a clinical trial at Capio Sankt Göran Hospital, Sweden. METHODS We conducted semi-structured interviews with seven breast imaging radiologists, evaluated using inductive thematic content analysis. RESULTS We identified three main thematic categories: AI in society, reflecting views on AI's contribution to the healthcare system; AI-human interactions, addressing the radiologists' self-perceptions when using the AI and its potential challenges to their profession; and AI as a tool among others. The radiologists were generally positive towards AI, and they felt comfortable handling its sometimes-ambiguous outputs and erroneous evaluations. While they did not feel that it would undermine their profession, they preferred using it as a complementary reader rather than an independent one. CONCLUSION The results suggested that breast radiology could become a launch pad for AI in healthcare. We recommend that this exploratory work on subjective perceptions be complemented by quantitative assessments to generalize the findings.
Collapse
Affiliation(s)
- Jennifer Viberg Johansson
- Department of Public Health and Caring Sciences, Centre for Research Ethics & Bioethics, Uppsala University, Uppsala, Sweden
| | - Emma Engström
- Institute for Futures Studies, Stockholm, Sweden; Department of Urban Planning and Environment, KTH Royal Institute of Technology, Stockholm, Sweden
| |
Collapse
|
8
|
Cheng CT, Lin HH, Hsu CP, Chen HW, Huang JF, Hsieh CH, Fu CY, Chung IF, Liao CH. Deep Learning for Automated Detection and Localization of Traumatic Abdominal Solid Organ Injuries on CT Scans. JOURNAL OF IMAGING INFORMATICS IN MEDICINE 2024; 37:1113-1123. [PMID: 38366294 PMCID: PMC11169164 DOI: 10.1007/s10278-024-01038-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/25/2023] [Revised: 01/31/2024] [Accepted: 02/01/2024] [Indexed: 02/18/2024]
Abstract
Computed tomography (CT) is the most commonly used diagnostic modality for blunt abdominal trauma (BAT), significantly influencing management approaches. Deep learning models (DLMs) have shown great promise in enhancing various aspects of clinical practice. There is limited literature available on the use of DLMs specifically for trauma image evaluation. In this study, we developed a DLM aimed at detecting solid organ injuries to assist medical professionals in rapidly identifying life-threatening injuries. The study enrolled patients from a single trauma center who received abdominal CT scans between 2008 and 2017. Patients with spleen, liver, or kidney injury were categorized as the solid organ injury group, while others were considered negative cases. Only images acquired from the trauma center were enrolled. A subset of images acquired in the last year was designated as the test set, and the remaining images were utilized to train and validate the detection models. The performance of each model was assessed using metrics such as the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, positive predictive value, and negative predictive value based on the best Youden index operating point. The study developed the models using 1302 (87%) scans for training and tested them on 194 (13%) scans. The spleen injury model demonstrated an accuracy of 0.938 and a specificity of 0.952. The accuracy and specificity of the liver injury model were reported as 0.820 and 0.847, respectively. The kidney injury model showed an accuracy of 0.959 and a specificity of 0.989. We developed a DLM that can automate the detection of solid organ injuries by abdominal CT scans with acceptable diagnostic accuracy. It cannot replace the role of clinicians, but we can expect it to be a potential tool to accelerate the process of therapeutic decisions for trauma care.
Collapse
Affiliation(s)
- Chi-Tung Cheng
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan
| | - Hou-Hsien Lin
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan
| | - Chih-Po Hsu
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan
| | - Huan-Wu Chen
- Department of Medical Imaging & Intervention, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan
| | - Jen-Fu Huang
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan
| | - Chi-Hsun Hsieh
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan
| | - Chih-Yuan Fu
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan
| | - I-Fang Chung
- Institute of Biomedical Informatics, National Yang Ming Chiao Tung University, Taipei, Taiwan
| | - Chien-Hung Liao
- Department of Trauma and Emergency Surgery, Chang Gung Memorial Hospital, Linkou, Chang Gung University, Taoyuan, Taiwan.
| |
Collapse
|
9
|
Cohen I, Sorin V, Lekach R, Raskin D, Segev M, Klang E, Eshed I, Barash Y. Artificial intelligence for detection of effusion and lipo-hemarthrosis in X-rays and CT of the knee. Eur J Radiol 2024; 175:111460. [PMID: 38608501 DOI: 10.1016/j.ejrad.2024.111460] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2024] [Revised: 03/29/2024] [Accepted: 04/08/2024] [Indexed: 04/14/2024]
Abstract
BACKGROUND Traumatic knee injuries are challenging to diagnose accurately through radiography and to a lesser extent, through CT, with fractures sometimes overlooked. Ancillary signs like joint effusion or lipo-hemarthrosis are indicative of fractures, suggesting the need for further imaging. Artificial Intelligence (AI) can automate image analysis, improving diagnostic accuracy and help prioritizing clinically important X-ray or CT studies. OBJECTIVE To develop and evaluate an AI algorithm for detecting effusion of any kind in knee X-rays and selected CT images and distinguishing between simple effusion and lipo-hemarthrosis indicative of intra-articular fractures. METHODS This retrospective study analyzed post traumatic knee imaging from January 2016 to February 2023, categorizing images into lipo-hemarthrosis, simple effusion, or normal. It utilized the FishNet-150 algorithm for image classification, with class activation maps highlighting decision-influential regions. The AI's diagnostic accuracy was validated against a gold standard, based on the evaluations made by a radiologist with at least four years of experience. RESULTS Analysis included CT images from 515 patients and X-rays from 637 post traumatic patients, identifying lipo-hemarthrosis, simple effusion, and normal findings. The AI showed an AUC of 0.81 for detecting any effusion, 0.78 for simple effusion, and 0.83 for lipo-hemarthrosis in X-rays; and 0.89, 0.89, and 0.91, respectively, in CTs. CONCLUSION The AI algorithm effectively detects knee effusion and differentiates between simple effusion and lipo-hemarthrosis in post-traumatic patients for both X-rays and selected CT images further studies are needed to validate these results.
Collapse
Affiliation(s)
- Israel Cohen
- Department of Diagnostic Imaging, Sheba Medical Center, Tel Hashomer, Israel; Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel.
| | - Vera Sorin
- Department of Diagnostic Imaging, Sheba Medical Center, Tel Hashomer, Israel; Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel.
| | - Ruth Lekach
- Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel; Department of Nuclear Medicine, Sourasky Medical Center, Tel-Aviv, Israel.
| | - Daniel Raskin
- Department of Diagnostic Imaging, Sheba Medical Center, Tel Hashomer, Israel; Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel.
| | - Maria Segev
- Department of Diagnostic Imaging, Sheba Medical Center, Tel Hashomer, Israel; Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel.
| | - Eyal Klang
- Department of Diagnostic Imaging, Sheba Medical Center, Tel Hashomer, Israel; Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel.
| | - Iris Eshed
- Department of Diagnostic Imaging, Sheba Medical Center, Tel Hashomer, Israel; Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel.
| | - Yiftach Barash
- Department of Diagnostic Imaging, Sheba Medical Center, Tel Hashomer, Israel; Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel.
| |
Collapse
|
10
|
Sarkar N, Kumagai M, Meyr S, Pothapragada S, Unberath M, Li G, Ahmed SR, Smith EB, Davis MA, Khatri GD, Agrawal A, Delproposto ZS, Chen H, Caballero CG, Dreizin D. An ASER AI/ML expert panel formative user research study for an interpretable interactive splenic AAST grading graphical user interface prototype. Emerg Radiol 2024; 31:167-178. [PMID: 38302827 PMCID: PMC11257379 DOI: 10.1007/s10140-024-02202-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2023] [Accepted: 01/08/2024] [Indexed: 02/03/2024]
Abstract
PURPOSE The AAST Organ Injury Scale is widely adopted for splenic injury severity but suffers from only moderate inter-rater agreement. This work assesses SpleenPro, a prototype interactive explainable artificial intelligence/machine learning (AI/ML) diagnostic aid to support AAST grading, for effects on radiologist dwell time, agreement, clinical utility, and user acceptance. METHODS Two trauma radiology ad hoc expert panelists independently performed timed AAST grading on 76 admission CT studies with blunt splenic injury, first without AI/ML assistance, and after a 2-month washout period and randomization, with AI/ML assistance. To evaluate user acceptance, three versions of the SpleenPro user interface with increasing explainability were presented to four independent expert panelists with four example cases each. A structured interview consisting of Likert scales and free responses was conducted, with specific questions regarding dimensions of diagnostic utility (DU); mental support (MS); effort, workload, and frustration (EWF); trust and reliability (TR); and likelihood of future use (LFU). RESULTS SpleenPro significantly decreased interpretation times for both raters. Weighted Cohen's kappa increased from 0.53 to 0.70 with AI/ML assistance. During user acceptance interviews, increasing explainability was associated with improvement in Likert scores for MS, EWF, TR, and LFU. Expert panelists indicated the need for a combined early notification and grading functionality, PACS integration, and report autopopulation to improve DU. CONCLUSIONS SpleenPro was useful for improving objectivity of AAST grading and increasing mental support. Formative user research identified generalizable concepts including the need for a combined detection and grading pipeline and integration with the clinical workflow.
Collapse
Affiliation(s)
- Nathan Sarkar
- University of Maryland School of Medicine, 655 W. Baltimore Street, Baltimore, MD, 21201, USA
| | - Mitsuo Kumagai
- University of Maryland College Park, 4603 Calvert Rd, College Park, MD, 20740, USA
| | - Samantha Meyr
- University of Maryland College Park, 4603 Calvert Rd, College Park, MD, 20740, USA
| | - Sriya Pothapragada
- University of Maryland College Park, 4603 Calvert Rd, College Park, MD, 20740, USA
| | - Mathias Unberath
- Johns Hopkins University, 3400 N. Charles Street, Baltimore, MD, 21218, USA
| | - Guang Li
- University of Maryland School of Medicine, 655 W. Baltimore Street, Baltimore, MD, 21201, USA
| | - Sagheer Rauf Ahmed
- University of Maryland School of Medicine, 655 W. Baltimore Street, Baltimore, MD, 21201, USA
- R Adams Cowley Shock Trauma Center, 22 S Greene St, Baltimore, MD, 21201, USA
| | - Elana Beth Smith
- University of Maryland School of Medicine, 655 W. Baltimore Street, Baltimore, MD, 21201, USA
- R Adams Cowley Shock Trauma Center, 22 S Greene St, Baltimore, MD, 21201, USA
| | | | | | - Anjali Agrawal
- Teleradiology Solutions, 22 Lianfair Road Unit 6, Ardmore, PA, 19003, USA
| | | | - Haomin Chen
- Johns Hopkins University, 3400 N. Charles Street, Baltimore, MD, 21218, USA
| | | | - David Dreizin
- University of Maryland School of Medicine, 655 W. Baltimore Street, Baltimore, MD, 21201, USA.
- R Adams Cowley Shock Trauma Center, 22 S Greene St, Baltimore, MD, 21201, USA.
| |
Collapse
|
11
|
Alsulimani L, AlRasheed B, Saeed A, Alabsi H. The Competency of Emergency Medicine Residents in Interpreting Hand X-rays Across the Three Major Regions of Saudi Arabia. Cureus 2024; 16:e59270. [PMID: 38686103 PMCID: PMC11057336 DOI: 10.7759/cureus.59270] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/29/2024] [Indexed: 05/02/2024] Open
Abstract
Background Interpreting hand X-rays is crucial for emergency medicine residents to accurately diagnose traumatic injuries and conditions affecting the hand. This study aimed to assess the competency of emergency medicine residents in interpreting hand X-rays across three major regions in Saudi Arabia. Methodology We conducted a cross-sectional study involving 100 emergency medicine residents from the Central, Eastern, and Western regions of Saudi Arabia. Participants were presented with 10 clinical case scenarios each accompanied by hand X-rays and were asked to provide their interpretations. Assessment scores were calculated based on the proportion of correct answers for each case. Results Half of the participants (50 residents) fell within the age range of 25 to 27 years, with 61 male and 39 female participants, respectively. Residents in the third year of training (R3) exhibited the highest mean score of 74.83% ± 20.46%. Participants using desktops to view the images achieved the highest mean score of 75% ± 10.49% compared to those using smartphones or tablets. Significant associations were found between age (F = 4.072, p = 0.020), training level (F = 3.161, p = 0.028), and choice of viewing device (F = 7.811, p = 0.001) and assessment scores. Conclusions Our study highlighted that emergency medicine residents in Saudi Arabia demonstrate competent proficiency in interpreting hand X-rays, with higher competency observed among senior residents (R3 and R4), those aged 28 to 30 years, and those using desktops for image viewing.
Collapse
Affiliation(s)
- Loui Alsulimani
- Emergency Medicine, King Abdulaziz University Hospital, Jeddah, SAU
| | - Basma AlRasheed
- Emergency Medicine, King Abdulaziz University Hospital, Jeddah, SAU
| | - Afnan Saeed
- Emergency Medicine, King Abdulaziz University Hospital, Jeddah, SAU
| | - Hatim Alabsi
- Radiology, King Abdulaziz University Hospital, Jeddah, SAU
| |
Collapse
|
12
|
Zhang L, LaBelle W, Unberath M, Chen H, Hu J, Li G, Dreizin D. A vendor-agnostic, PACS integrated, and DICOM-compatible software-server pipeline for testing segmentation algorithms within the clinical radiology workflow. Front Med (Lausanne) 2023; 10:1241570. [PMID: 37954555 PMCID: PMC10637622 DOI: 10.3389/fmed.2023.1241570] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/20/2023] [Accepted: 10/09/2023] [Indexed: 11/14/2023] Open
Abstract
Background Reproducible approaches are needed to bring AI/ML for medical image analysis closer to the bedside. Investigators wishing to shadow test cross-sectional medical imaging segmentation algorithms on new studies in real-time will benefit from simple tools that integrate PACS with on-premises image processing, allowing visualization of DICOM-compatible segmentation results and volumetric data at the radiology workstation. Purpose In this work, we develop and release a simple containerized and easily deployable pipeline for shadow testing of segmentation algorithms within the clinical workflow. Methods Our end-to-end automated pipeline has two major components- 1. A router/listener and anonymizer and an OHIF web viewer backstopped by a DCM4CHEE DICOM query/retrieve archive deployed in the virtual infrastructure of our secure hospital intranet, and 2. An on-premises single GPU workstation host for DICOM/NIfTI conversion steps, and image processing. DICOM images are visualized in OHIF along with their segmentation masks and associated volumetry measurements (in mL) using DICOM SEG and structured report (SR) elements. Since nnU-net has emerged as a widely-used out-of-the-box method for training segmentation models with state-of-the-art performance, feasibility of our pipleine is demonstrated by recording clock times for a traumatic pelvic hematoma nnU-net model. Results Mean total clock time from PACS send by user to completion of transfer to the DCM4CHEE query/retrieve archive was 5 min 32 s (± SD of 1 min 26 s). This compares favorably to the report turnaround times for whole-body CT exams, which often exceed 30 min, and illustrates feasibility in the clinical setting where quantitative results would be expected prior to report sign-off. Inference times accounted for most of the total clock time, ranging from 2 min 41 s to 8 min 27 s. All other virtual and on-premises host steps combined ranged from a minimum of 34 s to a maximum of 48 s. Conclusion The software worked seamlessly with an existing PACS and could be used for deployment of DL models within the radiology workflow for prospective testing on newly scanned patients. Once configured, the pipeline is executed through one command using a single shell script. The code is made publicly available through an open-source license at "https://github.com/vastc/," and includes a readme file providing pipeline config instructions for host names, series filter, other parameters, and citation instructions for this work.
Collapse
Affiliation(s)
- Lei Zhang
- School of Medicine, University of Maryland, Baltimore, MD, United States
| | - Wayne LaBelle
- School of Medicine, University of Maryland, Baltimore, MD, United States
| | - Mathias Unberath
- Department of Computer Science, Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, United States
| | - Haomin Chen
- Department of Computer Science, Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, United States
| | - Jiazhen Hu
- Department of Computer Science, Whiting School of Engineering, Johns Hopkins University, Baltimore, MD, United States
| | - Guang Li
- School of Medicine, University of Maryland, Baltimore, MD, United States
| | - David Dreizin
- School of Medicine, University of Maryland, Baltimore, MD, United States
| |
Collapse
|
13
|
Sarkar N, Zhang L, Campbell P, Liang Y, Li G, Khedr M, Khetan U, Dreizin D. Pulmonary contusion: automated deep learning-based quantitative visualization. Emerg Radiol 2023; 30:435-441. [PMID: 37318609 PMCID: PMC10527354 DOI: 10.1007/s10140-023-02149-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2023] [Accepted: 06/07/2023] [Indexed: 06/16/2023]
Abstract
PURPOSE Rapid automated CT volumetry of pulmonary contusion may predict progression to Acute Respiratory Distress Syndrome (ARDS) and help guide early clinical management in at-risk trauma patients. This study aims to train and validate state-of-the-art deep learning models to quantify pulmonary contusion as a percentage of total lung volume (Lung Contusion Index, or auto-LCI) and assess the relationship between auto-LCI and relevant clinical outcomes. METHODS 302 adult patients (age ≥ 18) with pulmonary contusion were retrospectively identified from reports between 2016 and 2021. nnU-Net was trained on manual contusion and whole-lung segmentations. Point-of-care candidate variables for multivariate regression included oxygen saturation, heart rate, and systolic blood pressure on admission. Logistic regression was used to assess ARDS risk, and Cox proportional hazards models were used to determine differences in ICU length of stay and mechanical ventilation time. RESULTS Mean Volume Similarity Index and mean Dice scores were 0.82 and 0.67. Interclass correlation coefficient and Pearson r between ground-truth and predicted volumes were 0.90 and 0.91. 38 (14%) patients developed ARDS. In bivariate analysis, auto-LCI was associated with ARDS (p < 0.001), ICU admission (p < 0.001), and need for mechanical ventilation (p < 0.001). In multivariate analyses, auto-LCI was associated with ARDS (p = 0.04), longer length of stay in the ICU (p = 0.02) and longer time on mechanical ventilation (p = 0.04). AUC of multivariate regression to predict ARDS using auto-LCI and clinical variables was 0.70 while AUC using auto-LCI alone was 0.68. CONCLUSION Increasing auto-LCI values corresponded with increased risk of ARDS, longer ICU admissions, and longer periods of mechanical ventilation.
Collapse
Affiliation(s)
- Nathan Sarkar
- Department of Diagnostic Radiology and Nuclear Medicine, R Adams Cowley Shock Trauma Center, University of Maryland School of Medicine, 22 S Greene St, Baltimore, MD, 21201, USA
| | - Lei Zhang
- Department of Diagnostic Radiology and Nuclear Medicine, R Adams Cowley Shock Trauma Center, University of Maryland School of Medicine, 22 S Greene St, Baltimore, MD, 21201, USA
| | - Peter Campbell
- Department of Diagnostic Radiology and Nuclear Medicine, R Adams Cowley Shock Trauma Center, University of Maryland School of Medicine, 22 S Greene St, Baltimore, MD, 21201, USA
| | - Yuanyuan Liang
- Department of Epidemiology & Public Health, University of Maryland School of Medicine, Baltimore, MD, USA
| | - Guang Li
- Department of Diagnostic Radiology and Nuclear Medicine, R Adams Cowley Shock Trauma Center, University of Maryland School of Medicine, 22 S Greene St, Baltimore, MD, 21201, USA
| | - Mustafa Khedr
- Department of Diagnostic Radiology and Nuclear Medicine, R Adams Cowley Shock Trauma Center, University of Maryland School of Medicine, 22 S Greene St, Baltimore, MD, 21201, USA
| | - Udit Khetan
- Department of Diagnostic Radiology and Nuclear Medicine, R Adams Cowley Shock Trauma Center, University of Maryland School of Medicine, 22 S Greene St, Baltimore, MD, 21201, USA
| | - David Dreizin
- Department of Diagnostic Radiology and Nuclear Medicine, R Adams Cowley Shock Trauma Center, University of Maryland School of Medicine, 22 S Greene St, Baltimore, MD, 21201, USA.
| |
Collapse
|
14
|
Dreizin D, Zhang L, Sarkar N, Bodanapally UK, Li G, Hu J, Chen H, Khedr M, Khetan U, Campbell P, Unberath M. Accelerating voxelwise annotation of cross-sectional imaging through AI collaborative labeling with quality assurance and bias mitigation. FRONTIERS IN RADIOLOGY 2023; 3:1202412. [PMID: 37485306 PMCID: PMC10362988 DOI: 10.3389/fradi.2023.1202412] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/08/2023] [Accepted: 06/22/2023] [Indexed: 07/25/2023]
Abstract
Background precision-medicine quantitative tools for cross-sectional imaging require painstaking labeling of targets that vary considerably in volume, prohibiting scaling of data annotation efforts and supervised training to large datasets for robust and generalizable clinical performance. A straight-forward time-saving strategy involves manual editing of AI-generated labels, which we call AI-collaborative labeling (AICL). Factors affecting the efficacy and utility of such an approach are unknown. Reduction in time effort is not well documented. Further, edited AI labels may be prone to automation bias. Purpose In this pilot, using a cohort of CTs with intracavitary hemorrhage, we evaluate both time savings and AICL label quality and propose criteria that must be met for using AICL annotations as a high-throughput, high-quality ground truth. Methods 57 CT scans of patients with traumatic intracavitary hemorrhage were included. No participant recruited for this study had previously interpreted the scans. nnU-net models trained on small existing datasets for each feature (hemothorax/hemoperitoneum/pelvic hematoma; n = 77-253) were used in inference. Two common scenarios served as baseline comparison- de novo expert manual labeling, and expert edits of trained staff labels. Parameters included time effort and image quality graded by a blinded independent expert using a 9-point scale. The observer also attempted to discriminate AICL and expert labels in a random subset (n = 18). Data were compared with ANOVA and post-hoc paired signed rank tests with Bonferroni correction. Results AICL reduced time effort 2.8-fold compared to staff label editing, and 8.7-fold compared to expert labeling (corrected p < 0.0006). Mean Likert grades for AICL (8.4, SD:0.6) were significantly higher than for expert labels (7.8, SD:0.9) and edited staff labels (7.7, SD:0.8) (corrected p < 0.0006). The independent observer failed to correctly discriminate AI and human labels. Conclusion For our use case and annotators, AICL facilitates rapid large-scale curation of high-quality ground truth. The proposed quality control regime can be employed by other investigators prior to embarking on AICL for segmentation tasks in large datasets.
Collapse
Affiliation(s)
- David Dreizin
- Department of Diagnostic Radiology and Nuclear Medicine, School of Medicine, University of Maryland, Baltimore, MD, United States
| | - Lei Zhang
- Department of Diagnostic Radiology and Nuclear Medicine, School of Medicine, University of Maryland, Baltimore, MD, United States
| | - Nathan Sarkar
- Department of Diagnostic Radiology and Nuclear Medicine, School of Medicine, University of Maryland, Baltimore, MD, United States
| | - Uttam K. Bodanapally
- Department of Diagnostic Radiology and Nuclear Medicine, School of Medicine, University of Maryland, Baltimore, MD, United States
| | - Guang Li
- Department of Diagnostic Radiology and Nuclear Medicine, School of Medicine, University of Maryland, Baltimore, MD, United States
| | - Jiazhen Hu
- Johns Hopkins University, Baltimore, MD, United States
| | - Haomin Chen
- Johns Hopkins University, Baltimore, MD, United States
| | - Mustafa Khedr
- Department of Diagnostic Radiology and Nuclear Medicine, School of Medicine, University of Maryland, Baltimore, MD, United States
| | - Udit Khetan
- Department of Diagnostic Radiology and Nuclear Medicine, School of Medicine, University of Maryland, Baltimore, MD, United States
| | - Peter Campbell
- Department of Diagnostic Radiology and Nuclear Medicine, School of Medicine, University of Maryland, Baltimore, MD, United States
| | | |
Collapse
|
15
|
Dreizin D. The American Society of Emergency Radiology (ASER) AI/ML expert panel: inception, mandate, work products, and goals. Emerg Radiol 2023; 30:279-283. [PMID: 37071272 DOI: 10.1007/s10140-023-02135-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2023] [Accepted: 04/11/2023] [Indexed: 04/19/2023]
Affiliation(s)
- David Dreizin
- Emergency and Trauma Imaging, Department of Diagnostic Radiology and Nuclear Medicine, R Adams Cowley Shock Trauma , Center, University of Maryland School of Medicine, Baltimore, MD, USA.
| |
Collapse
|
16
|
Zhang L, LaBelle W, Unberath M, Chen H, Hu J, Li G, Dreizin D. A vendor-agnostic, PACS integrated, and DICOMcompatible software-server pipeline for testing segmentation algorithms within the clinical radiology workflow. RESEARCH SQUARE 2023:rs.3.rs-2837634. [PMID: 37163064 PMCID: PMC10168465 DOI: 10.21203/rs.3.rs-2837634/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/11/2023]
Abstract
Background Reproducible approaches are needed to bring AI/ML for medical image analysis closer to the bedside. Investigators wishing to shadow test cross-sectional medical imaging segmentation algorithms on new studies in real-time will benefit from simple tools that integrate PACS with on-premises image processing, allowing visualization of DICOM-compatible segmentation results and volumetric data at the radiology workstation. Purpose In this work, we develop and release a simple containerized and easily deployable pipeline for shadow testing of segmentation algorithms within the clinical workflow. Methods Our end-to-end automated pipeline has two major components-1. a router/listener and anonymizer and an OHIF web viewer backstopped by a DCM4CHEE DICOM query/retrieve archive deployed in the virtual infrastructure of our secure hospital intranet, and 2. An on-premises single GPU workstation host for DICOM/NIfTI conversion steps, and image processing. DICOM images are visualized in OHIF along with their segmentation masks and associated volumetry measurements (in mL) using DICOM SEG and structured report (SR) elements. Feasibility is demonstrated by recording clock times for a traumatic pelvic hematoma cascaded nnU-net model. Results Mean total clock time from PACS send by user to completion of transfer to the DCM4CHEE query/retrieve archive was 5 minutes 32 seconds (+/- SD of 1 min 26 sec). This compares favorably to the report turnaround times for whole-body CT exams, which often exceed 30 minutes. Inference times accounted for most of the total clock time, ranging from 2 minutes 41 seconds to 8 minutes 27 seconds. All other virtual and on-premises host steps combined ranged from a minimum of 34 seconds to a maximum of 48 seconds. Conclusion The software worked seamlessly with an existing PACS and could be used for deployment of DL models within the radiology workflow for prospective testing on newly scanned patients. Once configured, the pipeline is executed through one command using a single shell script. The code is made publicly available through an open-source license at "https://github.com/vastc/", and includes a readme file providing pipeline config instructions for host names, series filter, other parameters, and citation instructions for this work.
Collapse
Affiliation(s)
| | | | | | | | | | - Guang Li
- University of Maryland, Baltimore
| | | |
Collapse
|