1
|
Wongveerasin P, Tongdee T, Saiviroonporn P. Deep learning for tubes and lines detection in critical illness: Generalizability and comparison with residents. Eur J Radiol Open 2024; 13:100593. [PMID: 39175597 PMCID: PMC11338948 DOI: 10.1016/j.ejro.2024.100593] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2024] [Revised: 07/12/2024] [Accepted: 07/23/2024] [Indexed: 08/24/2024] Open
Abstract
Background Artificial intelligence (AI) has been proven useful for the assessment of tubes and lines on chest radiographs of general patients. However, validation on intensive care unit (ICU) patients remains imperative. Methods This retrospective case-control study evaluated the performance of deep learning (DL) models for tubes and lines classification on both an external public dataset and a local dataset comprising 303 films randomly sampled from the ICU database. The endotracheal tubes (ETTs), central venous catheters (CVCs), and nasogastric tubes (NGTs) were classified into "Normal," "Abnormal," or "Borderline" positions by DL models with and without rule-based modification. Their performance was evaluated using an experienced radiologist as the standard reference. Results The algorithm showed decreased performance on the local ICU dataset, compared to that of the external dataset, decreasing from the Area Under the Curve of Receiver (AUC) of 0.967 (95 % CI 0.965-0.973) to the AUC of 0.70 (95 % CI 0.68-0.77). Significant improvement in the ETT classification task was observed after modifications were made to the model to allow the use of the spatial relationship between line tips and reference anatomy with the improvement of the AUC, increasing from 0.71 (95 % CI 0.70 - 0.75) to 0.86 (95 % CI 0.83 - 0.94). Conclusions The externally trained model exhibited limited generalizability on the local ICU dataset. Therefore, evaluating the performance of externally trained AI before integrating it into critical care routine is crucial. Rule-based algorithm may be used in combination with DL to improve results.
Collapse
Affiliation(s)
- Pootipong Wongveerasin
- Department of Radiology, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand
| | - Trongtum Tongdee
- Department of Radiology, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand
| | - Pairash Saiviroonporn
- Department of Radiology, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand
| |
Collapse
|
2
|
Marra A. G4 & the balanced metric family - a novel approach to solving binary classification problems in medical device validation & verification studies. BioData Min 2024; 17:43. [PMID: 39444008 PMCID: PMC11515465 DOI: 10.1186/s13040-024-00402-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2024] [Accepted: 10/15/2024] [Indexed: 10/25/2024] Open
Abstract
BACKGROUND In medical device validation and verification studies, the area under the receiver operating characteristic curve (AUROC) is often used as a primary endpoint despite multiple reports showing its limitations. Hence, researchers are encouraged to consider alternative metrics as primary endpoints. A new metric called G4 is presented, which is the geometric mean of sensitivity, specificity, the positive predictive value, and the negative predictive value. G4 is part of a balanced metric family which includes the Unified Performance Measure (also known as P4) and the Matthews' Correlation Coefficient (MCC). The purpose of this manuscript is to unveil the benefits of using G4 together with the balanced metric family when analyzing the overall performance of binary classifiers. RESULTS Simulated datasets encompassing different prevalence rates of the minority class were analyzed under a multi-reader-multi-case study design. In addition, data from an independently published study that tested the performance of a unique ultrasound artificial intelligence algorithm in the context of breast cancer detection was also considered. Within each dataset, AUROC was reported alongside the balanced metric family for comparison. When the dataset prevalence and bias of the minority class approached 50%, all three balanced metrics provided equivalent interpretations of an AI's performance. As the prevalence rate increased / decreased and the data became more imbalanced, AUROC tended to overvalue / undervalue the true classifier performance, while the balanced metric family was resistant to such imbalance. Under certain circumstances where data imbalance was strong (minority-class prevalence < 10%), MCC was preferred for standalone assessments while P4 provided a stronger effect size when evaluating between-groups analyses. G4 acted as a middle ground for maximizing both standalone assessments and between-groups analyses. CONCLUSIONS Use of AUROC as the primary endpoint in binary classification problems provides misleading results as the dataset becomes more imbalanced. This is explicitly noticed when incorporating AUROC in medical device validation and verification studies. G4, P4, and MCC do not share this limitation and paint a more complete picture of a medical device's performance in a clinical setting. Therefore, researchers are encouraged to explore the balanced metric family when evaluating binary classification problems.
Collapse
Affiliation(s)
- Andrew Marra
- Clinical Biostatistician at GE Healthcare, Chicago, IL, USA.
| |
Collapse
|
3
|
Balza R, Mercaldo SF, Huang AJ, Husseini JS, Jarraya M, Simeone FJ, Vicentini JRT, Palmer WE. Impact of Patient-reported Symptom Information on the Interpretation of MRI of the Lumbar Spine. Radiology 2024; 313:e233487. [PMID: 39470429 DOI: 10.1148/radiol.233487] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/30/2024]
Abstract
Background Distinguishing lumbar pain generators from incidental findings at MRI can be difficult. Dictated reports may become lists of findings that cannot be ranked in order of diagnostic importance. Purpose To determine whether patient-reported symptom information can improve the interpretation of lumbar spine MRI by using the spine specialist as the reference standard. Materials and Methods This prospective, single-center, multireader study analyzed 240 participants who completed pre-MRI symptom questionnaires between May 2022 and February 2023. At the time of clinical MRI reporting, radiologists recorded pain generators in consecutive participants, creating two study groups by alternating interpretations with versus without symptom questionnaire results (SQR). Diagnostic certainty was recorded using a numeric scale of 0 to 100. Types, levels, and sides of pain generators were compared with reference diagnoses by calculating Cohen κ values with 95% CIs. Participant characteristics and diagnostic certainties were compared using the Wilcoxon rank sum, Pearson χ2, or Kruskal-Wallis test. Interrater agreement was analyzed. Results There was no difference in age (P = .69) or sex (P = .60) between participants using SQR (n = 120; mean age, 61.0 years; 62 female) and not using SQR (n = 120; mean age, 62.5 years; 67 female). When radiologists were compared with specialists, agreements on pain generators were almost perfect for interpretations using SQR (type: κ = 0.82 [95% CI: 0.74,0.89]; level: κ = 0.88 [95% CI: 0.80, 0.95]; side: κ = 0.84 [95% CI: 0.75, 0.92]), but only fair to moderate for interpretations not using SQR (type: κ = 0.26 [95% CI: 0.15, 0.36]; level: κ = 0.51 [95% CI: 0.39, 0.63]; side: κ = 0.30 [95% CI: 0.18, 0.42]) (all P < .001). Diagnostic certainty was higher for MRI interpretations using SQR (mean, 80.4 ± 14.9 [SD]) than MRI interpretations not using SQR (60.5 ± 17.7) (P < .001). Interrater agreements were substantial (κ = 0.65-0.78) for MRI interpretations using SQR but only fair to moderate (κ = 0.24-0.49) for MRI interpretations not using SQR (all P < .001). Conclusion Patient-reported symptom information enabled radiologists to achieve nearly perfect diagnostic agreement with clinical experts. © RSNA, 2024 See also the editorial by Isikbay and Shah in this issue.
Collapse
Affiliation(s)
- Rene Balza
- From the Department of Radiology, Massachusetts General Hospital, 55 Fruit St, YAW 6030, Boston, MA 02114
| | - Sarah F Mercaldo
- From the Department of Radiology, Massachusetts General Hospital, 55 Fruit St, YAW 6030, Boston, MA 02114
| | - Ambrose J Huang
- From the Department of Radiology, Massachusetts General Hospital, 55 Fruit St, YAW 6030, Boston, MA 02114
| | - Jad S Husseini
- From the Department of Radiology, Massachusetts General Hospital, 55 Fruit St, YAW 6030, Boston, MA 02114
| | - Mohamed Jarraya
- From the Department of Radiology, Massachusetts General Hospital, 55 Fruit St, YAW 6030, Boston, MA 02114
| | - F Joseph Simeone
- From the Department of Radiology, Massachusetts General Hospital, 55 Fruit St, YAW 6030, Boston, MA 02114
| | - Joao R T Vicentini
- From the Department of Radiology, Massachusetts General Hospital, 55 Fruit St, YAW 6030, Boston, MA 02114
| | - William E Palmer
- From the Department of Radiology, Massachusetts General Hospital, 55 Fruit St, YAW 6030, Boston, MA 02114
| |
Collapse
|
4
|
Novak A, Ather S, Gill A, Aylward P, Maskell G, Cowell GW, Espinosa Morgado AT, Duggan T, Keevill M, Gamble O, Akrama O, Belcher E, Taberham R, Hallifax R, Bahra J, Banerji A, Bailey J, James A, Ansaripour A, Spence N, Wrightson J, Jarral W, Barry S, Bhatti S, Astley K, Shadmaan A, Ghelman S, Baenen A, Oke J, Bloomfield C, Johnson H, Beggs M, Gleeson F. Evaluation of the impact of artificial intelligence-assisted image interpretation on the diagnostic performance of clinicians in identifying pneumothoraces on plain chest X-ray: a multi-case multi-reader study. Emerg Med J 2024; 41:602-609. [PMID: 39009424 PMCID: PMC11503157 DOI: 10.1136/emermed-2023-213620] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2023] [Accepted: 06/10/2024] [Indexed: 07/17/2024]
Abstract
BACKGROUND Artificial intelligence (AI)-assisted image interpretation is a fast-developing area of clinical innovation. Most research to date has focused on the performance of AI-assisted algorithms in comparison with that of radiologists rather than evaluating the algorithms' impact on the clinicians who often undertake initial image interpretation in routine clinical practice. This study assessed the impact of AI-assisted image interpretation on the diagnostic performance of frontline acute care clinicians for the detection of pneumothoraces (PTX). METHODS A multicentre blinded multi-case multi-reader study was conducted between October 2021 and January 2022. The online study recruited 18 clinician readers from six different clinical specialties, with differing levels of seniority, across four English hospitals. The study included 395 plain CXR images, 189 positive for PTX and 206 negative. The reference standard was the consensus opinion of two thoracic radiologists with a third acting as arbitrator. General Electric Healthcare Critical Care Suite (GEHC CCS) PTX algorithm was applied to the final dataset. Readers individually interpreted the dataset without AI assistance, recording the presence or absence of a PTX and a confidence rating. Following a 'washout' period, this process was repeated including the AI output. RESULTS Analysis of the performance of the algorithm for detecting or ruling out a PTX revealed an overall AUROC of 0.939. Overall reader sensitivity increased by 11.4% (95% CI 4.8, 18.0, p=0.002) from 66.8% (95% CI 57.3, 76.2) unaided to 78.1% aided (95% CI 72.2, 84.0, p=0.002), specificity 93.9% (95% CI 90.9, 97.0) without AI to 95.8% (95% CI 93.7, 97.9, p=0.247). The junior reader subgroup showed the largest improvement at 21.7% (95% CI 10.9, 32.6), increasing from 56.0% (95% CI 37.7, 74.3) to 77.7% (95% CI 65.8, 89.7, p<0.01). CONCLUSION The study indicates that AI-assisted image interpretation significantly enhances the diagnostic accuracy of clinicians in detecting PTX, particularly benefiting less experienced practitioners. While overall interpretation time remained unchanged, the use of AI improved diagnostic confidence and sensitivity, especially among junior clinicians. These findings underscore the potential of AI to support less skilled clinicians in acute care settings.
Collapse
Affiliation(s)
- Alex Novak
- Emergency Department, Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Sarim Ather
- Radiology Department, Oxford University Hospitals, Oxford, UK
| | - Avneet Gill
- Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Peter Aylward
- Report and Image Quality Control (RAIQC), London, UK, UK
| | - Giles Maskell
- Royal Cornwall Hospitals NHS Trust, Truro, Cornwall, UK
| | | | | | - Tom Duggan
- Buckinghamshire Healthcare NHS Trust, Amersham, UK
| | - Melissa Keevill
- Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Olivia Gamble
- Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Osama Akrama
- Emergency Department, Royal Berkshire NHS Foundation Trust, Reading, UK
| | | | - Rhona Taberham
- Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Rob Hallifax
- Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Jasdeep Bahra
- Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | | | - Jon Bailey
- Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Antonia James
- Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Ali Ansaripour
- Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Nathan Spence
- Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - John Wrightson
- Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Waqas Jarral
- Frimley Health NHS Foundation Trust, Frimley, UK
| | - Steven Barry
- Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Saher Bhatti
- Frimley Health NHS Foundation Trust, Frimley, UK
| | - Kerry Astley
- Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Amied Shadmaan
- GE Healthcare Diagnostic Imaging, Little Chalfont, Buckinghamshire, UK
| | | | | | - Jason Oke
- University of Oxford Greyfriars, Oxford, UK
| | | | | | - Mark Beggs
- University of Oxford, Oxford, Oxfordshire, UK
| | - Fergus Gleeson
- Radiology Department, Oxford University Hospitals, Oxford, UK
| |
Collapse
|
5
|
Jiang Y, Iuanow E, Malik B, Klock J. A Multireader Multicase (MRMC) Receiver Operating Characteristic (ROC) Study Evaluating Noninferiority of Quantitative Transmission (QT) Ultrasound to Digital Breast Tomosynthesis (DBT) on Detection and Recall of Breast Lesions. Acad Radiol 2024; 31:2248-2258. [PMID: 38290888 DOI: 10.1016/j.acra.2023.12.038] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Revised: 12/16/2023] [Accepted: 12/26/2023] [Indexed: 02/01/2024]
Abstract
RATIONALE AND OBJECTIVES Quantitative transmission (QT) imaging is an emerging volumetric ultrasound modality for women too young for mammography. QT images tissue without overlap seen in mammography, thereby can potentially improve breast mass detection and characterization and noncancer recall. We compared radiologists' interpretation of QT vs digital breast tomosynthesis (DBT) with a multireader multicase observer performance study. MATERIALS AND METHODS Study subjects received screening DBT and QT scans in HIPAA-compliant, institutional review board-approved prospective case-collection studies at four clinical sites. Twenty-four Mammography Quality Standards Act-qualified radiologists interpreted 177 cases (66 with cancer, atypia, or solid mass and 111 normal or with nonsolid benign abnormality), first QT, then 2 weeks later DBT synthesized 2D-views. Readers reported up to three findings per case and for each finding a recall or no recall decision and confidence of that decision. The study hypothesis was area under receiver operating characteristic curve (AUC) of QT was noninferior to DBT. Sensitivity and specificity were also compared. RESULTS AUC of QT (0.746 ± 0.028, mean ± SD) was noninferior to DBT (0.700 ± 0.028) for AUC difference margin of -0.05 (P < .05). AUC difference was 0.046 ± 0.028 (95% CI: [-0.008, 0.101]). Sensitivity was 70.6 ± 7.2% for QT and 85.2 ± 6.4% for DBT, specificity was 60.1 ± 12.3% vs 37.2 ± 11.0%, and both differences were statistically significant. Of a total of 21 cases of cysts, readers recommended recall, on average, in 1.1 ± 1.4 cases with QT, but not with DBT, and 10.6 ± 2.2 cases with DBT, but not with QT. CONCLUSION QT can be a potential alternative to mammography for breast cancer screening of women too young to undergo mammography.
Collapse
Affiliation(s)
- Yulei Jiang
- Department of Radiology, the University of Chicago, 5841 South Maryland Ave, MC2026, Chicago, IL 60637.
| | | | | | | |
Collapse
|
6
|
Han PL, Jiang L, Cheng JL, Shi K, Huang S, Jiang Y, Jiang L, Xia Q, Li YY, Zhu M, Li K, Yang ZG. Artificial intelligence-assisted diagnosis of congenital heart disease and associated pulmonary arterial hypertension from chest radiographs: A multi-reader multi-case study. Eur J Radiol 2024; 171:111277. [PMID: 38160541 DOI: 10.1016/j.ejrad.2023.111277] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2023] [Revised: 12/10/2023] [Accepted: 12/20/2023] [Indexed: 01/03/2024]
Abstract
OBJECTIVES To explore the possibility of automatic diagnosis of congenital heart disease (CHD) and pulmonary arterial hypertension associated with CHD (PAH-CHD) from chest radiographs using artificial intelligence (AI) technology and to evaluate whether AI assistance could improve clinical diagnostic accuracy. MATERIALS AND METHODS A total of 3255 frontal preoperative chest radiographs (1174 CHD of any type and 2081 non-CHD) were retrospectively obtained. In this study, we adopted ResNet18 pretrained with the ImageNet database to establish diagnostic models. Radiologists diagnosed CHD/PAH-CHD from 330/165 chest radiographs twice: the first time, 50% of the images were accompanied by AI-based classification; after a month, the remaining 50% were accompanied by AI-based classification. Diagnostic results were compared between the radiologists and AI models, and between radiologists with and without AI assistance. RESULTS The AI model achieved an average area under the receiver operating characteristic curve (AUC) of 0.948 (sensitivity: 0.970, specificity: 0.982) for CHD diagnoses and an AUC of 0.778 (sensitivity: 0.632, specificity: 0.925) for identifying PAH-CHD. In the 330 balanced (165 CHD and 165 non-CHD) testing set, AI achieved higher AUCs than all 5 radiologists in the identification of CHD (0.670-0.858) and PAH-CHD (0.610-0.688). With AI assistance, the mean ± standard error AUC of radiologists was significantly improved for CHD (ΔAUC + 0.096, 95 % CI: 0.001-0.190; P = 0.048) and PAH-CHD (ΔAUC + 0.066, 95 % CI: 0.010-0.122; P = 0.031) diagnosis. CONCLUSION Chest radiograph-based AI models can detect CHD and PAH-CHD automatically. AI assistance improved radiologists' diagnostic accuracy, which may facilitate a timely initial diagnosis of CHD and PAH-CHD.
Collapse
Affiliation(s)
- Pei-Lun Han
- Department of Radiology and West China Biomedical Big Data Center, West China Hospital, Sichuan University, Chengdu, China
| | - Lei Jiang
- College of Computer Science, Sichuan University, Chengdu, China
| | - Jun-Long Cheng
- College of Computer Science, Sichuan University, Chengdu, China
| | - Ke Shi
- Department of Radiology and West China Biomedical Big Data Center, West China Hospital, Sichuan University, Chengdu, China
| | - Shan Huang
- Department of Radiology and West China Biomedical Big Data Center, West China Hospital, Sichuan University, Chengdu, China
| | - Yu Jiang
- Department of Radiology and West China Biomedical Big Data Center, West China Hospital, Sichuan University, Chengdu, China
| | - Li Jiang
- Department of Radiology and West China Biomedical Big Data Center, West China Hospital, Sichuan University, Chengdu, China
| | - Qing Xia
- SenseTime Research, Beijing, China
| | - Yi-Yue Li
- Department of Radiology and West China Biomedical Big Data Center, West China Hospital, Sichuan University, Chengdu, China
| | - Min Zhu
- College of Computer Science, Sichuan University, Chengdu, China
| | - Kang Li
- Department of Radiology and West China Biomedical Big Data Center, West China Hospital, Sichuan University, Chengdu, China; Med-X Center for Informatics, Sichuan University, Chengdu, China; Shanghai Artificial Intelligence Laboratory, Shanghai, China
| | - Zhi-Gang Yang
- Department of Radiology and West China Biomedical Big Data Center, West China Hospital, Sichuan University, Chengdu, China.
| |
Collapse
|
7
|
Cau R, Pisu F, Suri JS, Mannelli L, Scaglione M, Masala S, Saba L. Artificial Intelligence Applications in Cardiovascular Magnetic Resonance Imaging: Are We on the Path to Avoiding the Administration of Contrast Media? Diagnostics (Basel) 2023; 13:2061. [PMID: 37370956 PMCID: PMC10297403 DOI: 10.3390/diagnostics13122061] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2023] [Revised: 06/10/2023] [Accepted: 06/12/2023] [Indexed: 06/29/2023] Open
Abstract
In recent years, cardiovascular imaging examinations have experienced exponential growth due to technological innovation, and this trend is consistent with the most recent chest pain guidelines. Contrast media have a crucial role in cardiovascular magnetic resonance (CMR) imaging, allowing for more precise characterization of different cardiovascular diseases. However, contrast media have contraindications and side effects that limit their clinical application in determinant patients. The application of artificial intelligence (AI)-based techniques to CMR imaging has led to the development of non-contrast models. These AI models utilize non-contrast imaging data, either independently or in combination with clinical and demographic data, as input to generate diagnostic or prognostic algorithms. In this review, we provide an overview of the main concepts pertaining to AI, review the existing literature on non-contrast AI models in CMR, and finally, discuss the strengths and limitations of these AI models and their possible future development.
Collapse
Affiliation(s)
- Riccardo Cau
- Department of Radiology, University Hospital of Cagliari, 09042 Monserrato, Italy; (R.C.); (F.P.)
| | - Francesco Pisu
- Department of Radiology, University Hospital of Cagliari, 09042 Monserrato, Italy; (R.C.); (F.P.)
| | - Jasjit S. Suri
- Stroke Monitoring and Diagnostic Division, AtheroPoint™, Roseville, CA 95661, USA;
| | | | - Mariano Scaglione
- Department of Radiology, University Hospital of Sassari, 07100 Sassari, Italy; (M.S.); (S.M.)
| | - Salvatore Masala
- Department of Radiology, University Hospital of Sassari, 07100 Sassari, Italy; (M.S.); (S.M.)
| | - Luca Saba
- Department of Radiology, University Hospital of Cagliari, 09042 Monserrato, Italy; (R.C.); (F.P.)
| |
Collapse
|
8
|
Yang W, Chen C, Yang Y, Chen L, Yang C, Gong L, Wang J, Shi F, Wu D, Yan F. Diagnostic performance of deep learning-based vessel extraction and stenosis detection on coronary computed tomography angiography for coronary artery disease: a multi-reader multi-case study. LA RADIOLOGIA MEDICA 2023; 128:307-315. [PMID: 36800112 DOI: 10.1007/s11547-023-01606-9] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/13/2022] [Accepted: 02/03/2023] [Indexed: 02/18/2023]
Abstract
BACKGROUND Post-processing and interpretation of coronary CT angiography (CCTA) imaging are time-consuming and dependent on the reader's experience. An automated deep learning (DL)-based imaging reconstruction and diagnosis system was developed to improve diagnostic accuracy and efficiency. METHODS Our study including 374 cases from five sites, inviting 12 radiologists, assessed the DL-based system in diagnosing obstructive coronary disease with regard to diagnostic performance, imaging post-processing and reporting time of radiologists, with invasive coronary angiography as a standard reference. The diagnostic performance of DL system and DL-assisted human readers was compared with the traditional method of human readers without DL system. RESULTS Comparing the diagnostic performance of human readers without DL system versus with DL system, the AUC was improved from 0.81 to 0.82 (p < 0.05) at patient level and from 0.79 to 0.81 (p < 0.05) at vessel level. An increase in AUC was observed in inexperienced radiologists (p < 0.05), but was absent in experienced radiologists. Regarding diagnostic efficiency, comparing the DL system versus human reader, the average post-processing and reporting time was decreased from 798.60 s to 189.12 s (p < 0.05). The sensitivity and specificity of using DL system alone were 93.55% and 59.57% at patient level and 83.23% and 79.97% at vessel level, respectively. CONCLUSIONS With the DL system serving as a concurrent reader, the overall post-processing and reading time was substantially reduced. The diagnostic accuracy of human readers, especially for inexperienced readers, was improved. DL-assisted human reader had the potential of being the reading mode of choice in clinical routine.
Collapse
Affiliation(s)
- Wenjie Yang
- Department of Radiology, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Chihua Chen
- Department of Radiology, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Yanzhao Yang
- Department of Radiology, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China
| | - Lei Chen
- Department of Radiology, Peking University People's Hospital, Beijing, China
| | - Changwei Yang
- Department of Radiology, Shanghai East Hospital, Tongji University School of Medicine, Shanghai, China
| | - Lianggeng Gong
- Department of Radiology, Second Affiliated Hospital of Nanchang University, Nanchang, China
| | - Jianing Wang
- Department of Radiology, Affiliated Hospital of Hebei University, Baoding, China
| | - Feng Shi
- Shanghai United Imaging Intelligence Co., Ltd, Shanghai, China
| | - Dijia Wu
- Shanghai United Imaging Intelligence Co., Ltd, Shanghai, China
| | - Fuhua Yan
- Department of Radiology, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
| |
Collapse
|
9
|
Warman R, Warman A, Warman P, Degnan A, Blickman J, Chowdhary V, Dash D, Sangal R, Vadhan J, Bueso T, Windisch T, Neves G. Deep Learning System Boosts Radiologist Detection of Intracranial Hemorrhage. Cureus 2022; 14:e30264. [PMID: 36381767 PMCID: PMC9653089 DOI: 10.7759/cureus.30264] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/13/2022] [Indexed: 01/25/2023] Open
Abstract
BACKGROUND Intracranial hemorrhage (ICH) requires emergent medical treatment for positive outcomes. While previous artificial intelligence (AI) solutions achieved rapid diagnostics, none were shown to improve the performance of radiologists in detecting ICHs. Here, we show that the Caire ICH artificial intelligence system enhances a radiologist's ICH diagnosis performance. METHODS A dataset of non-contrast-enhanced axial cranial computed tomography (CT) scans (n=532) were labeled for the presence or absence of an ICH. If an ICH was detected, its ICH subtype was identified. After a washout period, the three radiologists reviewed the same dataset with the assistance of the Caire ICH system. Performance was measured with respect to reader agreement, accuracy, sensitivity, and specificity when compared to the ground truth, defined as reader consensus. RESULTS Caire ICH improved the inter-reader agreement on average by 5.76% in a dataset with an ICH prevalence of 74.3%. Further, radiologists using Caire ICH detected an average of 18 more ICHs and significantly increased their accuracy by 6.15%, their sensitivity by 4.6%, and their specificity by 10.62%. The Caire ICH system also improved the radiologist's ability to accurately identify the ICH subtypes present. CONCLUSION The Caire ICH device significantly improves the performance of a cohort of radiologists. Such a device has the potential to be a tool that can improve patient outcomes and reduce misdiagnosis of ICH.
Collapse
Affiliation(s)
| | | | | | - Andrew Degnan
- Radiology, University of Pittsburgh Medical Center (UPMC) Children's Hospital of Pittsburgh, Pittsburgh, USA
| | | | | | - Dev Dash
- Emergency Medicine, Stanford University, Stanford, USA
| | - Rohit Sangal
- Emergency Medicine, Yale School of Medicine, New Haven, USA
| | - Jason Vadhan
- Emergency Medicine, The University of Texas Southwestern (UTSW), Dallas, USA
| | - Tulio Bueso
- Neurology, The Texas Tech University Health Sciences Center (TTUHSC), Lubbock, USA
| | | | - Gabriel Neves
- Neurology, The Texas Tech University Health Sciences Center (TTUHSC), Lubbock, USA
| |
Collapse
|
10
|
Tai HC, Chen KY, Wu MH, Chang KJ, Chen CN, Chen A. Assessing Detection Accuracy of Computerized Sonographic Features and Computer-Assisted Reading Performance in Differentiating Thyroid Cancers. Biomedicines 2022; 10:biomedicines10071513. [PMID: 35884818 PMCID: PMC9313277 DOI: 10.3390/biomedicines10071513] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Revised: 06/20/2022] [Accepted: 06/24/2022] [Indexed: 11/16/2022] Open
Abstract
For ultrasound imaging of thyroid nodules, medical guidelines are all based on findings of sonographic features to provide clinicians management recommendations. Due to the recent development of artificial intelligence and machine learning (AI/ML) technologies, there have been computer-assisted detection (CAD) software devices available for clinical use to detect and quantify the sonographic features of thyroid nodules. This study is to validate the accuracy of the computerized sonographic features (CSF) by a CAD software device, namely, AmCAD-UT, and then to assess how the reading performance of clinicians (readers) can be improved providing the computerized features. The feature detection accuracy is tested against the ground truth established by a panel of thyroid specialists and a multiple-reader multiple-case (MRMC) study is performed to assess the sequential reading performance with the assistance of the CSF. Five computerized features, including anechoic area, hyperechoic foci, hypoechoic pattern, heterogeneous texture, and indistinct margin, were tested, with AUCs ranging from 0.888~0.946, 0.825~0.913, 0.812~0.847, 0.627~0.77, and 0.676~0.766, respectively. With the five CSFs, the sequential reading performance of 18 clinicians is found significantly improved, with the AUC increasing from 0.720 without CSF to 0.776 with CSF. Our studies show that the computerized features are consistent with the clinicians’ findings and provide additional value in assisting sonographic diagnosis.
Collapse
Affiliation(s)
- Hao-Chih Tai
- Department of Surgery, National Taiwan University Hospital and College of Medicine, Taipei 100225, Taiwan; (H.-C.T.); (K.-Y.C.); (M.-H.W.); (K.-J.C.)
| | - Kuen-Yuan Chen
- Department of Surgery, National Taiwan University Hospital and College of Medicine, Taipei 100225, Taiwan; (H.-C.T.); (K.-Y.C.); (M.-H.W.); (K.-J.C.)
| | - Ming-Hsun Wu
- Department of Surgery, National Taiwan University Hospital and College of Medicine, Taipei 100225, Taiwan; (H.-C.T.); (K.-Y.C.); (M.-H.W.); (K.-J.C.)
| | - King-Jen Chang
- Department of Surgery, National Taiwan University Hospital and College of Medicine, Taipei 100225, Taiwan; (H.-C.T.); (K.-Y.C.); (M.-H.W.); (K.-J.C.)
| | - Chiung-Nien Chen
- Department of Surgery, National Taiwan University Hospital and College of Medicine, Taipei 100225, Taiwan; (H.-C.T.); (K.-Y.C.); (M.-H.W.); (K.-J.C.)
- Correspondence: (C.-N.C.); (A.C.)
| | - Argon Chen
- Graduate Institute of Industrial Engineering, National Taiwan University, Taipei 106216, Taiwan
- Correspondence: (C.-N.C.); (A.C.)
| |
Collapse
|
11
|
Obuchowski NA, Bullen J. Multireader Diagnostic Accuracy Imaging Studies: Fundamentals of Design and Analysis. Radiology 2022; 303:26-34. [PMID: 35166584 DOI: 10.1148/radiol.211593] [Citation(s) in RCA: 35] [Impact Index Per Article: 11.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022]
Abstract
The design and analysis of multireader multicase (MRMC) studies are quite challenging. These studies differ from most medical studies because they need a reference standard and sampling from two populations (ie, reader and patient populations). They are quite expensive to conduct, requiring a good deal of readers' time for image interpretation. One common problem is the use of imperfect reference standards, often correlated with the test or tests being evaluated. Another common issue is oversimplification of the multidimensional MRMC data. In this study, the fundamentals of MRMC study design and analysis are reviewed. The goal is to provide investigators with a guide to the fundamentals of MRMC design and analysis, with references to more detailed discussions. In addition, readers are updated on newer areas of research, including correction for studies with multiple diagnostic accuracy end points and adjustment for location bias.
Collapse
Affiliation(s)
- Nancy A Obuchowski
- From the Department of Quantitative Health Sciences, Cleveland Clinic Foundation, 9500 Euclid Ave, JJN3, Cleveland, OH 44195
| | - Jennifer Bullen
- From the Department of Quantitative Health Sciences, Cleveland Clinic Foundation, 9500 Euclid Ave, JJN3, Cleveland, OH 44195
| |
Collapse
|
12
|
Fuhrman JD, Gorre N, Hu Q, Li H, El Naqa I, Giger ML. A review of explainable and interpretable AI with applications in COVID-19 imaging. Med Phys 2022; 49:1-14. [PMID: 34796530 PMCID: PMC8646613 DOI: 10.1002/mp.15359] [Citation(s) in RCA: 38] [Impact Index Per Article: 12.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2021] [Revised: 10/14/2021] [Accepted: 10/25/2021] [Indexed: 12/24/2022] Open
Abstract
The development of medical imaging artificial intelligence (AI) systems for evaluating COVID-19 patients has demonstrated potential for improving clinical decision making and assessing patient outcomes during the recent COVID-19 pandemic. These have been applied to many medical imaging tasks, including disease diagnosis and patient prognosis, as well as augmented other clinical measurements to better inform treatment decisions. Because these systems are used in life-or-death decisions, clinical implementation relies on user trust in the AI output. This has caused many developers to utilize explainability techniques in an attempt to help a user understand when an AI algorithm is likely to succeed as well as which cases may be problematic for automatic assessment, thus increasing the potential for rapid clinical translation. AI application to COVID-19 has been marred with controversy recently. This review discusses several aspects of explainable and interpretable AI as it pertains to the evaluation of COVID-19 disease and it can restore trust in AI application to this disease. This includes the identification of common tasks that are relevant to explainable medical imaging AI, an overview of several modern approaches for producing explainable output as appropriate for a given imaging scenario, a discussion of how to evaluate explainable AI, and recommendations for best practices in explainable/interpretable AI implementation. This review will allow developers of AI systems for COVID-19 to quickly understand the basics of several explainable AI techniques and assist in the selection of an approach that is both appropriate and effective for a given scenario.
Collapse
Affiliation(s)
- Jordan D. Fuhrman
- Medical Imaging and Data Resource Center (MIDRC)The University of ChicagoChicagoIllinoisUSA
- Department of RadiologyThe University of ChicagoChicagoIllinoisUSA
| | - Naveena Gorre
- Medical Imaging and Data Resource Center (MIDRC)The University of ChicagoChicagoIllinoisUSA
- Department of Machine LearningMoffitt Cancer CenterTampaFloridaUSA
| | - Qiyuan Hu
- Medical Imaging and Data Resource Center (MIDRC)The University of ChicagoChicagoIllinoisUSA
- Department of RadiologyThe University of ChicagoChicagoIllinoisUSA
| | - Hui Li
- Medical Imaging and Data Resource Center (MIDRC)The University of ChicagoChicagoIllinoisUSA
- Department of RadiologyThe University of ChicagoChicagoIllinoisUSA
| | - Issam El Naqa
- Medical Imaging and Data Resource Center (MIDRC)The University of ChicagoChicagoIllinoisUSA
- Department of Machine LearningMoffitt Cancer CenterTampaFloridaUSA
| | - Maryellen L. Giger
- Medical Imaging and Data Resource Center (MIDRC)The University of ChicagoChicagoIllinoisUSA
- Department of RadiologyThe University of ChicagoChicagoIllinoisUSA
| |
Collapse
|
13
|
Upton R, Mumith A, Beqiri A, Parker A, Hawkes W, Gao S, Porumb M, Sarwar R, Marques P, Markham D, Kenworthy J, O'Driscoll JM, Hassanali N, Groves K, Dockerill C, Woodward W, Alsharqi M, McCourt A, Wilkes EH, Heitner SB, Yadava M, Stojanovski D, Lamata P, Woodward G, Leeson P. Automated Echocardiographic Detection of Severe Coronary Artery Disease Using Artificial Intelligence. JACC Cardiovasc Imaging 2021; 15:715-727. [PMID: 34922865 DOI: 10.1016/j.jcmg.2021.10.013] [Citation(s) in RCA: 53] [Impact Index Per Article: 13.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/20/2021] [Revised: 10/01/2021] [Accepted: 10/21/2021] [Indexed: 01/27/2023]
Abstract
OBJECTIVES The purpose of this study was to establish whether an artificially intelligent (AI) system can be developed to automate stress echocardiography analysis and support clinician interpretation. BACKGROUND Coronary artery disease is the leading global cause of mortality and morbidity and stress echocardiography remains one of the most commonly used diagnostic imaging tests. METHODS An automated image processing pipeline was developed to extract novel geometric and kinematic features from stress echocardiograms collected as part of a large, United Kingdom-based prospective, multicenter, multivendor study. An ensemble machine learning classifier was trained, using the extracted features, to identify patients with severe coronary artery disease on invasive coronary angiography. The model was tested in an independent U.S. STUDY How availability of an AI classification might impact clinical interpretation of stress echocardiograms was evaluated in a randomized crossover reader study. RESULTS Acceptable classification accuracy for identification of patients with severe coronary artery disease in the training data set was achieved on cross-fold validation based on 31 unique geometric and kinematic features, with a specificity of 92.7% and a sensitivity of 84.4%. This accuracy was maintained in the independent validation data set. The use of the AI classification tool by clinicians increased inter-reader agreement and confidence as well as sensitivity for detection of disease by 10% to achieve an area under the receiver-operating characteristic curve of 0.93. CONCLUSION Automated analysis of stress echocardiograms is possible using AI and provision of automated classifications to clinicians when reading stress echocardiograms could improve accuracy, inter-reader agreement, and reader confidence.
Collapse
Affiliation(s)
- Ross Upton
- Ultromics Ltd, Oxford, United Kingdom; Cardiovascular Clinical Research Facility, RDM Division of Cardiovascular Medicine, University of Oxford, Oxford, United Kingdom
| | | | | | | | | | - Shan Gao
- Ultromics Ltd, Oxford, United Kingdom
| | | | - Rizwan Sarwar
- Cardiovascular Clinical Research Facility, RDM Division of Cardiovascular Medicine, University of Oxford, Oxford, United Kingdom
| | | | | | | | - Jamie M O'Driscoll
- Ultromics Ltd, Oxford, United Kingdom; School of Human and Life Sciences, Canterbury Christ Church University, Kent, United Kingdom
| | | | | | - Cameron Dockerill
- Cardiovascular Clinical Research Facility, RDM Division of Cardiovascular Medicine, University of Oxford, Oxford, United Kingdom
| | - William Woodward
- Cardiovascular Clinical Research Facility, RDM Division of Cardiovascular Medicine, University of Oxford, Oxford, United Kingdom
| | - Maryam Alsharqi
- Cardiovascular Clinical Research Facility, RDM Division of Cardiovascular Medicine, University of Oxford, Oxford, United Kingdom
| | - Annabelle McCourt
- Cardiovascular Clinical Research Facility, RDM Division of Cardiovascular Medicine, University of Oxford, Oxford, United Kingdom
| | | | - Stephen B Heitner
- Knight Cardiovascular Institute, Oregon Health & Science University, Portland Oregon, USA
| | - Mrinal Yadava
- Knight Cardiovascular Institute, Oregon Health & Science University, Portland Oregon, USA
| | - David Stojanovski
- Department of Imaging Sciences and Biomedical Engineering, King's College London, London, United Kingdom
| | - Pablo Lamata
- Department of Imaging Sciences and Biomedical Engineering, King's College London, London, United Kingdom
| | | | - Paul Leeson
- Ultromics Ltd, Oxford, United Kingdom; Cardiovascular Clinical Research Facility, RDM Division of Cardiovascular Medicine, University of Oxford, Oxford, United Kingdom.
| |
Collapse
|
14
|
Seah JCY, Tang CHM, Buchlak QD, Holt XG, Wardman JB, Aimoldin A, Esmaili N, Ahmad H, Pham H, Lambert JF, Hachey B, Hogg SJF, Johnston BP, Bennett C, Oakden-Rayner L, Brotchie P, Jones CM. Effect of a comprehensive deep-learning model on the accuracy of chest x-ray interpretation by radiologists: a retrospective, multireader multicase study. Lancet Digit Health 2021; 3:e496-e506. [PMID: 34219054 DOI: 10.1016/s2589-7500(21)00106-0] [Citation(s) in RCA: 107] [Impact Index Per Article: 26.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2020] [Revised: 05/02/2021] [Accepted: 05/12/2021] [Indexed: 02/01/2023]
Abstract
BACKGROUND Chest x-rays are widely used in clinical practice; however, interpretation can be hindered by human error and a lack of experienced thoracic radiologists. Deep learning has the potential to improve the accuracy of chest x-ray interpretation. We therefore aimed to assess the accuracy of radiologists with and without the assistance of a deep-learning model. METHODS In this retrospective study, a deep-learning model was trained on 821 681 images (284 649 patients) from five data sets from Australia, Europe, and the USA. 2568 enriched chest x-ray cases from adult patients (≥16 years) who had at least one frontal chest x-ray were included in the test dataset; cases were representative of inpatient, outpatient, and emergency settings. 20 radiologists reviewed cases with and without the assistance of the deep-learning model with a 3-month washout period. We assessed the change in accuracy of chest x-ray interpretation across 127 clinical findings when the deep-learning model was used as a decision support by calculating area under the receiver operating characteristic curve (AUC) for each radiologist with and without the deep-learning model. We also compared AUCs for the model alone with those of unassisted radiologists. If the lower bound of the adjusted 95% CI of the difference in AUC between the model and the unassisted radiologists was more than -0·05, the model was considered to be non-inferior for that finding. If the lower bound exceeded 0, the model was considered to be superior. FINDINGS Unassisted radiologists had a macroaveraged AUC of 0·713 (95% CI 0·645-0·785) across the 127 clinical findings, compared with 0·808 (0·763-0·839) when assisted by the model. The deep-learning model statistically significantly improved the classification accuracy of radiologists for 102 (80%) of 127 clinical findings, was statistically non-inferior for 19 (15%) findings, and no findings showed a decrease in accuracy when radiologists used the deep-learning model. Unassisted radiologists had a macroaveraged mean AUC of 0·713 (0·645-0·785) across all findings, compared with 0·957 (0·954-0·959) for the model alone. Model classification alone was significantly more accurate than unassisted radiologists for 117 (94%) of 124 clinical findings predicted by the model and was non-inferior to unassisted radiologists for all other clinical findings. INTERPRETATION This study shows the potential of a comprehensive deep-learning model to improve chest x-ray interpretation across a large breadth of clinical practice. FUNDING Annalise.ai.
Collapse
Affiliation(s)
- Jarrel C Y Seah
- Annalise.ai, Sydney, NSW, Australia; Department of Radiology, Alfred Health, Melbourne, VIC, Australia
| | | | | | | | | | | | - Nazanin Esmaili
- School of Medicine, University of Notre Dame Australia, Sydney, NSW, Australia; Faculty of Engineering and IT, University of Technology Sydney, Sydney, NSW, Australia
| | | | | | | | | | | | | | - Christine Bennett
- School of Medicine, University of Notre Dame Australia, Sydney, NSW, Australia
| | - Luke Oakden-Rayner
- Australian Institute for Machine Learning, The University of Adelaide, Adelaide, SA, Australia
| | - Peter Brotchie
- Annalise.ai, Sydney, NSW, Australia; Department of Radiology, St Vincent's Health Australia, Melbourne, VIC, Australia
| | | |
Collapse
|
15
|
Fletcher JG, DeLone DR, Kotsenas AL, Campeau NG, Lehman VT, Yu L, Leng S, Holmes DR, Edwards PK, Johnson MP, Michalak GJ, Carter RE, McCollough CH. Evaluation of Lower-Dose Spiral Head CT for Detection of Intracranial Findings Causing Neurologic Deficits. AJNR Am J Neuroradiol 2019; 40:1855-1863. [PMID: 31649155 DOI: 10.3174/ajnr.a6251] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2019] [Accepted: 08/21/2019] [Indexed: 01/13/2023]
Abstract
BACKGROUND AND PURPOSE Despite the frequent use of unenhanced head CT for the detection of acute neurologic deficit, the radiation dose for this exam varies widely. Our aim was to evaluate the performance of lower-dose head CT for detection of intracranial findings resulting in acute neurologic deficit. MATERIALS AND METHODS Projection data from 83 patients undergoing unenhanced spiral head CT for suspected neurologic deficits were collected. Cases positive for infarction, intra-axial hemorrhage, mass, or extra-axial hemorrhage required confirmation by histopathology, surgery, progression of findings, or corresponding neurologic deficit; cases negative for these target diagnoses required negative assessments by two neuroradiologists and a clinical neurologist. A routine dose head CT was obtained using 250 effective mAs and iterative reconstruction. Lower-dose configurations were reconstructed (25-effective mAs iterative reconstruction, 50-effective mAs filtered back-projection and iterative reconstruction, 100-effective mAs filtered back-projection and iterative reconstruction, 200-effective mAs filtered back-projection). Three neuroradiologists circled findings, indicating diagnosis, confidence (0-100), and image quality. The difference between the jackknife alternative free-response receiver operating characteristic figure of merit at routine and lower-dose configurations was estimated. A lower 95% CI estimate of the difference greater than -0.10 indicated noninferiority. RESULTS Forty-two of 83 patients had 70 intracranial findings (29 infarcts, 25 masses, 10 extra- and 6 intra-axial hemorrhages) at routine head CT (CT dose index = 38.3 mGy). The routine-dose jackknife alternative free-response receiver operating characteristic figure of merit was 0.87 (95% CI, 0.81-0.93). Noninferiority was shown for 100-effective mAs iterative reconstruction (figure of merit difference, -0.04; 95% CI, -0.08 to 0.004) and 200-effective mAs filtered back-projection (-0.02; 95% CI, -0.06 to 0.02) but not for 100-effective mAs filtered back-projection (-0.06; 95% CI, -0.10 to -0.02) or lower-dose levels. Image quality was better at higher-dose levels and with iterative reconstruction (P < .05). CONCLUSIONS Observer performance for dose levels using 100-200 eff mAs was noninferior to that observed at 250 effective mAs with iterative reconstruction, with iterative reconstruction preserving noninferiority at a mean CT dose index of 15.2 mGy.
Collapse
Affiliation(s)
- J G Fletcher
- From the Departments of Radiology (J.G.F., D.R.D., A.L.K., N.G.C., V.T.L., L.Y., S.L., G.J.M., C.H.M.)
| | - D R DeLone
- From the Departments of Radiology (J.G.F., D.R.D., A.L.K., N.G.C., V.T.L., L.Y., S.L., G.J.M., C.H.M.)
| | - A L Kotsenas
- From the Departments of Radiology (J.G.F., D.R.D., A.L.K., N.G.C., V.T.L., L.Y., S.L., G.J.M., C.H.M.)
| | - N G Campeau
- From the Departments of Radiology (J.G.F., D.R.D., A.L.K., N.G.C., V.T.L., L.Y., S.L., G.J.M., C.H.M.)
| | - V T Lehman
- From the Departments of Radiology (J.G.F., D.R.D., A.L.K., N.G.C., V.T.L., L.Y., S.L., G.J.M., C.H.M.)
| | - L Yu
- From the Departments of Radiology (J.G.F., D.R.D., A.L.K., N.G.C., V.T.L., L.Y., S.L., G.J.M., C.H.M.)
| | - S Leng
- From the Departments of Radiology (J.G.F., D.R.D., A.L.K., N.G.C., V.T.L., L.Y., S.L., G.J.M., C.H.M.)
| | - D R Holmes
- Biomedical Imaging Resource (D.R.H., P.E.)
| | | | - M P Johnson
- Biomedical Statistics and Informatics (M.P.J.), Mayo Clinic, Rochester, Minnesota
| | - G J Michalak
- From the Departments of Radiology (J.G.F., D.R.D., A.L.K., N.G.C., V.T.L., L.Y., S.L., G.J.M., C.H.M.)
| | - R E Carter
- Health Sciences Research (R.E.C.), Mayo Clinic, Jacksonville, Florida
| | - C H McCollough
- From the Departments of Radiology (J.G.F., D.R.D., A.L.K., N.G.C., V.T.L., L.Y., S.L., G.J.M., C.H.M.)
| |
Collapse
|
16
|
Impact of Deep Learning Assistance on the Histopathologic Review of Lymph Nodes for Metastatic Breast Cancer. Am J Surg Pathol 2019; 42:1636-1646. [PMID: 30312179 PMCID: PMC6257102 DOI: 10.1097/pas.0000000000001151] [Citation(s) in RCA: 287] [Impact Index Per Article: 47.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
Abstract
Supplemental Digital Content is available in the text. Advances in the quality of whole-slide images have set the stage for the clinical use of digital images in anatomic pathology. Along with advances in computer image analysis, this raises the possibility for computer-assisted diagnostics in pathology to improve histopathologic interpretation and clinical care. To evaluate the potential impact of digital assistance on interpretation of digitized slides, we conducted a multireader multicase study utilizing our deep learning algorithm for the detection of breast cancer metastasis in lymph nodes. Six pathologists reviewed 70 digitized slides from lymph node sections in 2 reader modes, unassisted and assisted, with a wash-out period between sessions. In the assisted mode, the deep learning algorithm was used to identify and outline regions with high likelihood of containing tumor. Algorithm-assisted pathologists demonstrated higher accuracy than either the algorithm or the pathologist alone. In particular, algorithm assistance significantly increased the sensitivity of detection for micrometastases (91% vs. 83%, P=0.02). In addition, average review time per image was significantly shorter with assistance than without assistance for both micrometastases (61 vs. 116 s, P=0.002) and negative images (111 vs. 137 s, P=0.018). Lastly, pathologists were asked to provide a numeric score regarding the difficulty of each image classification. On the basis of this score, pathologists considered the image review of micrometastases to be significantly easier when interpreted with assistance (P=0.0005). Utilizing a proof of concept assistant tool, this study demonstrates the potential of a deep learning algorithm to improve pathologist accuracy and efficiency in a digital pathology workflow.
Collapse
|
17
|
Yu H, Chen X, Dong R, Zhang W, Han P, Kang L, Ma Y, Jia L, Fu L, Hou L, Yu X, Wang L, Zhu X, Yang F, Guo Q. Clinical relevance of different handgrip strength indexes and cardiovascular disease risk factors: A cross-sectional study in suburb-dwelling elderly Chinese. J Formos Med Assoc 2018; 118:1062-1072. [PMID: 30522855 DOI: 10.1016/j.jfma.2018.11.001] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2018] [Revised: 10/10/2018] [Accepted: 11/01/2018] [Indexed: 01/14/2023] Open
Abstract
BACKGROUND Reduced muscle strength, as measured by handgrip strength (HS), has been associated with an increased risk of cardiovascular disease (CVD). The aim of this study was to examine the association between different HS indexes and CVD risk factors in elderly Chinese individuals. We also determine optimal cutoffs of HS indexes for predicting CVD risk factors. METHODS Data were obtained from 603 men and 789 women aged ≥60 years (average age 66.8 ± 6.4 y). These study participants were recruited in the suburb area of Tianjin, China. An individual was considered a patient when they exhibited any one of three CVD risk factors: diabetes mellitus, hypertension and dyslipidemia. All participants were interviewed face-to-face. In addition, serum samples were collected from all participants, and all participants underwent measures of anthropometry and HS. RESULTS The optimal cutoffs were 0.376 of HS/weight in men and 0.726 of HS/body fat mass in women for predicting diabetes mellitus. The adjusted odds ratios (ORs) of at least one CVD risk factor for those with low muscle strength identified by HS/body fat mass were 2.14 (95% confidence interval [CI]: 1.53, 3.44; p < 0.001) in men and 2.32 (95% CI: 1.60, 3.29; p < 0.001) in women. CONCLUSION HS/body fat mass appear to be the index best associated with CVD risk factors except diabetes mellitus in men. The optimal cutoffs of HS indexes have the potential to identify elderly adults at risk of CVD.
Collapse
Affiliation(s)
- Hairui Yu
- Department of Rehabilitation Medicine, TEDA International Cardiovascular Hospital, Cardiovascular Clinical College of Tianjin Medical University, Tianjin, China; Department of Rehabilitation Medicine, Tianjin Medical University, Tianjin, China
| | - Xiaoyu Chen
- Department of Rehabilitation Medicine, Tianjin Medical University, Tianjin, China
| | - Renwei Dong
- Department of Rehabilitation Medicine, Tianjin Medical University, Tianjin, China
| | - Wen Zhang
- Department of Rehabilitation Medicine, Tianjin Medical University, Tianjin, China
| | - Peipei Han
- Department of Rehabilitation Medicine, Tianjin Medical University, Tianjin, China
| | - Li Kang
- Department of Rehabilitation Medicine, Tianjin Medical University, Tianjin, China
| | - Yixuan Ma
- Department of Rehabilitation Medicine, Tianjin Medical University, Tianjin, China
| | - Liye Jia
- Department of Rehabilitation Medicine, Tianjin Medical University, Tianjin, China
| | - Liyuan Fu
- Department of Rehabilitation Medicine, Tianjin Medical University, Tianjin, China
| | - Lin Hou
- Department of Rehabilitation Medicine, Tianjin Medical University, Tianjin, China
| | - Xing Yu
- Department of Rehabilitation Medicine, Tianjin Medical University, Tianjin, China
| | - Lu Wang
- Department of Rehabilitation Medicine, Tianjin Medical University, Tianjin, China
| | - Xiaodong Zhu
- Department of Neurology, Tianjin Medical University, Tianjin, China
| | - Fengying Yang
- Department of Rehabilitation Medicine, Tianjin Medical University, Tianjin, China
| | - Qi Guo
- Department of Rehabilitation Medicine, TEDA International Cardiovascular Hospital, Cardiovascular Clinical College of Tianjin Medical University, Tianjin, China; Department of Rehabilitation Medicine, Tianjin Medical University, Tianjin, China.
| |
Collapse
|
18
|
Different pixel pitch and maximum luminance of medical grade displays may result in different evaluations of digital radiography images. Radiol Med 2018; 123:586-592. [DOI: 10.1007/s11547-018-0891-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2018] [Accepted: 04/09/2018] [Indexed: 10/17/2022]
|
19
|
Hostetter J, Khanna N, Mandell JC. Integration of a Zero-footprint Cloud-based Picture Archiving and Communication System with Customizable Forms for Radiology Research and Education. Acad Radiol 2018; 25:811-818. [PMID: 29555567 DOI: 10.1016/j.acra.2018.01.031] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/25/2017] [Revised: 12/16/2017] [Accepted: 01/15/2018] [Indexed: 11/25/2022]
Abstract
RATIONALE AND OBJECTIVES The purpose of this study was to integrate web-based forms with a zero-footprint cloud-based Picture Archiving and Communication Systems (PACS) to create a tool of potential benefit to radiology research and education. MATERIALS AND METHODS Web-based forms were created with a front-end and back-end architecture utilizing common programming languages including Vue.js, Node.js and MongoDB, and integrated into an existing zero-footprint cloud-based PACS. RESULTS The web-based forms application can be accessed in any modern internet browser on desktop or mobile devices and allows the creation of customizable forms consisting of a variety of questions types. Each form can be linked to an individual DICOM examination or a collection of DICOM examinations. CONCLUSIONS Several uses are demonstrated through a series of case studies, including implementation of a research platform for multi-reader multi-case (MRMC) studies and other imaging research, and creation of an online Objective Structure Clinical Examination (OSCE) and an educational case file.
Collapse
|
20
|
Darawsheh WB. Awareness and Knowledge about Occupational Therapy in Jordan. Occup Ther Int 2018; 2018:2493584. [PMID: 29950955 PMCID: PMC5987337 DOI: 10.1155/2018/2493584] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2017] [Revised: 11/27/2017] [Accepted: 04/10/2018] [Indexed: 11/24/2022] Open
Abstract
Knowledge and awareness about occupational therapy (OT) are essential for the delivery of quality care to all clients and for occupational therapists' (OTRs) job satisfaction. OT has been a poorly understood profession in Jordan. The current study reports on the assessment of Jordanians' awareness and knowledge of occupational therapy. Convenience sampling was used. There were 829 participants (474 males, 355 females), with mean age of 32 ± 11.6 yrs. They were recruited from the three main geographical areas of Jordan (northern, central, and southern) and from all educational levels. The sample included 222 (26.8%) healthcare personnel, 146 (17.6%) clients, and 461 (55.6%) lay persons. Participants completed questionnaires, and the results revealed that 48% of the sample had poor or no knowledge about OT, while 28.3% were unaware of it. Also, OT was commonly (50%) perceived to be exclusively targeting people with disabilities (PWDs) and neurological and physical conditions (58% and 53%, resp.) in addition to exclusively providing services for the rehabilitation of the upper extremity (48%). Common misconceptions associated with OT were that OTRs prescribe medication (43%) and OTRs are physiotherapists (44%). These preliminary findings suggest that efforts need to be directed by OTRs, the Jordanian Society of Occupational Therapy (JSOT), and the Ministry of Health to preserve the OT identity and value and promote knowledge about OT in the public and among members of interdisciplinary teams. More interprofessional learning needs to be incorporated within the curricula and placements of all healthcare personnels.
Collapse
Affiliation(s)
- Wesam Barakat Darawsheh
- Department of Occupational Therapy, School of Rehabilitation Sciences, The University of Jordan, Amman, Jordan
| |
Collapse
|
21
|
Obuchowski NA, Bullen JA. Receiver operating characteristic (ROC) curves: review of methods with applications in diagnostic medicine. ACTA ACUST UNITED AC 2018; 63:07TR01. [DOI: 10.1088/1361-6560/aab4b1] [Citation(s) in RCA: 149] [Impact Index Per Article: 21.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/26/2022]
|
22
|
Darawsheh WB, Natour YS, Sada EG. Applicability of the Arabic version of Vocal Tract Discomfort Scale (VTDS) with student singers as professional voice users. LOGOP PHONIATR VOCO 2017; 43:80-91. [DOI: 10.1080/14015439.2017.1363282] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Affiliation(s)
- Wesam B. Darawsheh
- Department of Occupational Therapy, School of Rehabilitation Sciences, University of Jordan, Amman, Jordan
| | - Yaser S. Natour
- Department of Hearing and Speech Sciences, School of Rehabilitation Sciences, University of Jordan, Amman, Jordan
| | - Eve G. Sada
- Music Education, University of Oklahoma, Norman, OK, USA
| |
Collapse
|
23
|
Ferranti C, Primolevo A, Cartia F, Cavatorta C, Ciniselli CM, Lualdi M, Meroni S, Pignoli E, Plebani M, Siciliano C, Verderio P, Scaperrotta G. How Does the Display Luminance Level Affect Detectability of Breast Microcalcifications and Spiculated Lesions in Digital Breast Tomosynthesis (DBT) Images? Acad Radiol 2017; 24:795-801. [PMID: 28189505 DOI: 10.1016/j.acra.2017.01.014] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2016] [Revised: 01/18/2017] [Accepted: 01/20/2017] [Indexed: 10/20/2022]
Abstract
RATIONALE AND OBJECTIVES This study evaluates the influence of the calibrated luminance level of medical displays in the detectability of microcalcifications and spiculated lesions in digital breast tomosynthesis images. MATERIALS AND METHODS Four models of medical displays with calibrated maximum and minimum luminance, respectively, ranging from 500 to 1000 cd/m2 and from 0.5 to 1.0 cd/m2, were investigated. Forty-eight studies were selected by a senior radiologist: 16 with microcalcifications, 16 with spiculated lesions, and 16 without lesions. All images were anonymized and blindly evaluated by one senior and two junior radiologists. For each study, lesion presence or absence and localization statements, interpretative difficulty level, and overall quality were reported. Cohen's kappa statistic was computed between monitors and within or between radiologists to estimate the reproducibility in correctly identifying lesions; for multireader-multicase analysis, the weighted jackknife alternative free-response receiver operating characteristic statistical tool was applied. RESULTS Intraradiologist reproducibility ranged from 0.75 to 1.00. Interreader as well as reader-truth agreement values were >0.80 and higher with the two 1000 cd/m2 luminance displays than with the lower luminance displays for each radiologist. Performances in the detectability of breast lesions were significantly greater with the 1000 cd/m2 luminance displays when compared to the display with the lowest luminance value (P value <0.001). CONCLUSIONS Our findings highlight the role of display luminance level on the accuracy of detecting breast lesions.
Collapse
|
24
|
Radiological Medical Device Innovation: Approvals via the Premarket Approval Pathway From 2000 to 2015. J Am Coll Radiol 2017; 14:24-33. [DOI: 10.1016/j.jacr.2016.08.014] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2016] [Revised: 08/07/2016] [Accepted: 08/16/2016] [Indexed: 11/20/2022]
|
25
|
Affiliation(s)
| | - Jennifer A. Bullen
- Quantitative Health Sciences, Cleveland Clinic Foundation, Cleveland, OH, USA
| | - Nancy A. Obuchowski
- Quantitative Health Sciences, Cleveland Clinic Foundation, Cleveland, OH, USA
| |
Collapse
|