1
|
Nowroozi A, Salehi MA, Shobeiri P, Agahi S, Momtazmanesh S, Kaviani P, Kalra MK. Artificial intelligence diagnostic accuracy in fracture detection from plain radiographs and comparing it with clinicians: a systematic review and meta-analysis. Clin Radiol 2024:S0009-9260(24)00200-9. [PMID: 38772766 DOI: 10.1016/j.crad.2024.04.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2024] [Revised: 04/09/2024] [Accepted: 04/15/2024] [Indexed: 05/23/2024]
Abstract
PURPOSE Fracture detection is one of the most commonly used and studied aspects of artificial intelligence (AI) in medicine. In this systematic review and meta-analysis, we aimed to summarize available literature and data regarding AI performance in fracture detection on plain radiographs and various factors affecting it. METHODS We systematically reviewed studies evaluating AI algorithms in detecting bone fractures in plain radiographs, combined their performance using meta-analysis (a bivariate regression approach), and compared it with that of clinicians. We also analyzed the factors potentially affecting algorithm performance using meta-regression. RESULTS Our analysis included 100 studies. In 83 studies with confusion matrices, AI algorithms showed a sensitivity of 91.43% and a specificity of 92.12% (Area under the summary receiver operator curve = 0.968). After adjustment and false discovery rate correction, tibia/fibula (excluding ankle) fractures were associated with higher (7.0%, p=0.004) AI sensitivity, while more recent publications (5.5%, p=0.003) and Xception architecture (6.6%, p<0.001) were associated with higher specificity. Clinicians and AI showed similar specificity in fracture identification, although AI leaned to higher sensitivity (7.6%, p=0.07). Radiologists, on the other hand, were more specific than AI overall and in several subgroups, and more sensitive to hip fractures before FDR correction. CONCLUSIONS Currently available AI aids could result in a significant improvement in care where radiologists are not readily available. Moreover, identifying factors affecting algorithm performance could guide AI development teams in their process of optimizing their products.
Collapse
Affiliation(s)
- A Nowroozi
- School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - M A Salehi
- School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - P Shobeiri
- School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - S Agahi
- School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - S Momtazmanesh
- School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - P Kaviani
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA 02114, USA
| | - M K Kalra
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA 02114, USA.
| |
Collapse
|
2
|
Karout L, Digumarthy SR, Savage C, Fahimi R, Garza-Frias E, Kaviani P, Dasegowda G, Kalra MK. Low Contrast Volume Protocol in Routine Chest CT Amid the Global Contrast Shortage: A Single Institution Experience. Acad Radiol 2023; 30:2913-2920. [PMID: 37164818 DOI: 10.1016/j.acra.2023.03.020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2022] [Revised: 03/16/2023] [Accepted: 03/17/2023] [Indexed: 05/12/2023]
Abstract
OBJECTIVE To assess the effectiveness of low contrast volume (LCV) chest CT performed with multiple contrast agents on multivendor CT with varying scanning techniques. METHODS The study included 361 patients (65 ± 15 years; M: F 173:188) who underwent LCV chest CT on one of the six 64-256 detector-row CT scanners using single-energy (SECT) or dual-energy (DECT) modes. All patients were scanned with either a fixed-LCV (LCVf, n = 103) or weight-based LCV (LCVw, n = 258) protocol. Two thoracic radiologists independently assessed all LCV CT and patients' prior standard contrast volume (SCV, n = 263) chest CT for optimality of contrast enhancement in thoracic vasculature, cardiac chambers, and in pleuro-parenchymal and mediastinal abnormalities. CT attenuations were recorded in the main pulmonary trunk, ascending, and descending thoracic aorta. To assess the interobserver agreement, pulmonary arterial enhancement was divided into two groups: optimal or suboptimal. RESULTS There was no significant difference among patients' BMI (p = 0.883) in the three groups. DECT had a significantly higher aortic arterial enhancement (250 ± 99HU vs 228 ± 76 HU for SECT, p < 0.001). Optimal enhancement was present in 558 of 624 chest CT (89.4%), whereas 66 of 624 chest CT with suboptimal enhancement was noted in 48 of 258 LCVw (18.6%) and 14 of 103 LCVf (13.6%). Most patients with suboptimal enhancement with LCVw injection protocol were overweight/obese (30/48; 62.5%), (p < 0.001). CONCLUSION LCV chest CT can be performed across complex multivendor, multicontrast media, multiscanner, and multiprotocol CT practices. However, LCV chest CT examinations can result in suboptimal contrast enhancement in patients with larger body habitus.
Collapse
Affiliation(s)
- Lina Karout
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts, 75 Blossom Court, Boston, Massachusetts, 02114
| | - Subba R Digumarthy
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts, 75 Blossom Court, Boston, Massachusetts, 02114
| | - Cristy Savage
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts
| | - Roshan Fahimi
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts, 75 Blossom Court, Boston, Massachusetts, 02114
| | - Emiliano Garza-Frias
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts, 75 Blossom Court, Boston, Massachusetts, 02114
| | - Parisa Kaviani
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts, 75 Blossom Court, Boston, Massachusetts, 02114
| | - Giridhar Dasegowda
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts, 75 Blossom Court, Boston, Massachusetts, 02114
| | - Mannudeep K Kalra
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts, 75 Blossom Court, Boston, Massachusetts, 02114.
| |
Collapse
|
3
|
Dasegowda G, Bizzo BC, Gupta RV, Kaviani P, Ebrahimian S, Ricciardelli D, Abedi-Tari F, Neumark N, Digumarthy SR, Kalra MK, Dreyer KJ. Radiologist-Trained AI Model for Identifying Suboptimal Chest-Radiographs. Acad Radiol 2023; 30:2921-2930. [PMID: 37019698 DOI: 10.1016/j.acra.2023.03.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2023] [Revised: 02/28/2023] [Accepted: 03/06/2023] [Indexed: 04/05/2023]
Abstract
RATIONALE AND OBJECTIVES Suboptimal chest radiographs (CXR) can limit interpretation of critical findings. Radiologist-trained AI models were evaluated for differentiating suboptimal(sCXR) and optimal(oCXR) chest radiographs. MATERIALS AND METHODS Our IRB-approved study included 3278 CXRs from adult patients (mean age 55 ± 20 years) identified from a retrospective search of CXR in radiology reports from 5 sites. A chest radiologist reviewed all CXRs for the cause of suboptimality. The de-identified CXRs were uploaded into an AI server application for training and testing 5 AI models. The training set consisted of 2202 CXRs (n = 807 oCXR; n = 1395 sCXR) while 1076 CXRs (n = 729 sCXR; n = 347 oCXR) were used for testing. Data were analyzed with the Area under the curve (AUC) for the model's ability to classify oCXR and sCXR correctly. RESULTS For the two-class classification into sCXR or oCXR from all sites, for CXR with missing anatomy, AI had sensitivity, specificity, accuracy, and AUC of 78%, 95%, 91%, 0.87(95% CI 0.82-0.92), respectively. AI identified obscured thoracic anatomy with 91% sensitivity, 97% specificity, 95% accuracy, and 0.94 AUC (95% CI 0.90-0.97). Inadequate exposure with 90% sensitivity, 93% specificity, 92% accuracy, and AUC of 0.91 (95% CI 0.88-0.95). The presence of low lung volume was identified with 96% sensitivity, 92% specificity, 93% accuracy, and 0.94 AUC (95% CI 0.92-0.96). The sensitivity, specificity, accuracy, and AUC of AI in identifying patient rotation were 92%, 96%, 95%, and 0.94 (95% CI 0.91-0.98), respectively. CONCLUSION The radiologist-trained AI models can accurately classify optimal and suboptimal CXRs. Such AI models at the front end of radiographic equipment can enable radiographers to repeat sCXRs when necessary.
Collapse
Affiliation(s)
- Giridhar Dasegowda
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA 02114; Mass General Brigham Data Science Office (DSO), 100 Cambridge St, Boston, MA, US 02114
| | - Bernardo C Bizzo
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA 02114; Mass General Brigham Data Science Office (DSO), 100 Cambridge St, Boston, MA, US 02114
| | - Reya V Gupta
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA 02114
| | - Parisa Kaviani
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA 02114; Mass General Brigham Data Science Office (DSO), 100 Cambridge St, Boston, MA, US 02114
| | - Shadi Ebrahimian
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA 02114; Mass General Brigham Data Science Office (DSO), 100 Cambridge St, Boston, MA, US 02114
| | - Debra Ricciardelli
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA 02114
| | - Faezeh Abedi-Tari
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA 02114
| | - Nir Neumark
- Mass General Brigham Data Science Office (DSO), 100 Cambridge St, Boston, MA, US 02114
| | - Subba R Digumarthy
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA 02114; Mass General Brigham Data Science Office (DSO), 100 Cambridge St, Boston, MA, US 02114
| | - Mannudeep K Kalra
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA 02114; Mass General Brigham Data Science Office (DSO), 100 Cambridge St, Boston, MA, US 02114.
| | - Keith J Dreyer
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA 02114; Mass General Brigham Data Science Office (DSO), 100 Cambridge St, Boston, MA, US 02114
| |
Collapse
|
4
|
Dasegowda G, Bizzo BC, Kaviani P, Karout L, Ebrahimian S, Digumarthy SR, Neumark N, Hillis JM, Kalra MK, Dreyer KJ. Auto-Detection of Motion Artifacts on CT Pulmonary Angiograms with a Physician-Trained AI Algorithm. Diagnostics (Basel) 2023; 13:778. [PMID: 36832266 PMCID: PMC9955317 DOI: 10.3390/diagnostics13040778] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2022] [Revised: 02/02/2023] [Accepted: 02/16/2023] [Indexed: 02/22/2023] Open
Abstract
Purpose: Motion-impaired CT images can result in limited or suboptimal diagnostic interpretation (with missed or miscalled lesions) and patient recall. We trained and tested an artificial intelligence (AI) model for identifying substantial motion artifacts on CT pulmonary angiography (CTPA) that have a negative impact on diagnostic interpretation. Methods: With IRB approval and HIPAA compliance, we queried our multicenter radiology report database (mPower, Nuance) for CTPA reports between July 2015 and March 2022 for the following terms: "motion artifacts", "respiratory motion", "technically inadequate", and "suboptimal" or "limited exam". All CTPA reports were from two quaternary (Site A, n = 335; B, n = 259) and a community (C, n = 199) healthcare sites. A thoracic radiologist reviewed CT images of all positive hits for motion artifacts (present or absent) and their severity (no diagnostic effect or major diagnostic impairment). Coronal multiplanar images from 793 CTPA exams were de-identified and exported offline into an AI model building prototype (Cognex Vision Pro, Cognex Corporation) to train an AI model to perform two-class classification ("motion" or "no motion") with data from the three sites (70% training dataset, n = 554; 30% validation dataset, n = 239). Separately, data from Site A and Site C were used for training and validating; testing was performed on the Site B CTPA exams. A five-fold repeated cross-validation was performed to evaluate the model performance with accuracy and receiver operating characteristics analysis (ROC). Results: Among the CTPA images from 793 patients (mean age 63 ± 17 years; 391 males, 402 females), 372 had no motion artifacts, and 421 had substantial motion artifacts. The statistics for the average performance of the AI model after five-fold repeated cross-validation for the two-class classification included 94% sensitivity, 91% specificity, 93% accuracy, and 0.93 area under the ROC curve (AUC: 95% CI 0.89-0.97). Conclusion: The AI model used in this study can successfully identify CTPA exams with diagnostic interpretation limiting motion artifacts in multicenter training and test datasets. Clinical relevance: The AI model used in the study can help alert technologists about the presence of substantial motion artifacts on CTPA, where a repeat image acquisition can help salvage diagnostic information.
Collapse
Affiliation(s)
- Giridhar Dasegowda
- Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114, USA
- Mass General Brigham Data Science Office, Boston, MA 02114, USA
| | - Bernardo C. Bizzo
- Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114, USA
- Mass General Brigham Data Science Office, Boston, MA 02114, USA
| | - Parisa Kaviani
- Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114, USA
- Mass General Brigham Data Science Office, Boston, MA 02114, USA
| | - Lina Karout
- Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114, USA
- Mass General Brigham Data Science Office, Boston, MA 02114, USA
| | - Shadi Ebrahimian
- Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114, USA
| | - Subba R. Digumarthy
- Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114, USA
| | - Nir Neumark
- Mass General Brigham Data Science Office, Boston, MA 02114, USA
| | - James M. Hillis
- Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114, USA
- Mass General Brigham Data Science Office, Boston, MA 02114, USA
| | - Mannudeep K. Kalra
- Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114, USA
- Mass General Brigham Data Science Office, Boston, MA 02114, USA
| | - Keith J. Dreyer
- Massachusetts General Hospital, Harvard Medical School, Boston, MA 02114, USA
- Mass General Brigham Data Science Office, Boston, MA 02114, USA
| |
Collapse
|
5
|
Taheri SV, Roshan B, Roshan K, Kaviani P, Pezeshgi A. On the occassion of world kidney day 2023; renal impacts of COVID-19. J Nephropathol 2023. [DOI: 10.34172/jnp.2023.21430] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/17/2023] Open
Abstract
World kidney day is an international campaign focused on bringing awareness to kidney health throughout the world and reducing the incidence of renal disease and its related medical complications. This mini-review sought to take a short look on the renal impact of SARS-CoV-2, with a particular focus on post-COVID-19 nephropathy as a new dilemma in the era of nephrology, which can be a new concern for nephrologists that requires more attention and particular strategies.
Collapse
Affiliation(s)
- Seyed Vahid Taheri
- Student Research Committee, Semnan University of Medical Sciences, Semnan, Iran
| | - Bijan Roshan
- Division of Nephrology, Scripps Clinic, La Jolla, California, USA
| | - Keyan Roshan
- University of California, Santa Barbara, California, USA
| | - Parisa Kaviani
- Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
| | - Aiyoub Pezeshgi
- Zanjan Metabolic Diseases Research Center, Zanjan University of Medical Science, Zanjan, Iran
| |
Collapse
|
6
|
Kaviani P, Primak A, Bizzo B, Ebrahimian S, Saini S, Dreyer KJ, Kalra MK. Performance of threshold-based stone segmentation and radiomics for determining the composition of kidney stones from single-energy CT. Jpn J Radiol 2023; 41:194-200. [PMID: 36331701 DOI: 10.1007/s11604-022-01349-z] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2022] [Accepted: 09/30/2022] [Indexed: 11/06/2022]
Abstract
PURPOSE Knowledge of kidney stone composition can help in patient management; urine composition analysis and dual-energy CT are frequently used to assess stone type. We assessed if threshold-based stone segmentation and radiomics can determine the composition of kidney stones from single-energy, non-contrast abdomen-pelvis CT. METHODS With IRB approval, we identified 218 consecutive patients (mean age 64 ± 13 years; male:female 138:80) with the presence of kidney stones on non-contrast, abdomen-pelvis CT and surgical or biochemical proof of their stone composition. CT examinations were performed on one of the seven multidetector-row scanners from four vendors (GE, Philips, Siemens, Toshiba). Deidentified CT images were processed with a radiomics prototype (Frontier, Siemens Healthineers) to segment the entire kidney volumes with an AI-based organ segmentation tool. We applied a threshold of 130 HU to isolate stones in the segmented kidneys and to estimate radiomics over the segmented stone volume. A coinvestigator verified kidney stone segmentation and adjusted the volume of interest to include the entire stone volume when necessary. We applied multiple logistic regression tests with precision recall plots to obtain area under the curve (AUC) using a built-in R statistical program. RESULTS The threshold-based stone segmentation successfully isolated kidney stones (uric acid: n = 102 patients, calcium oxalate/phosphate: n = 116 patients) in all patients. Radiomics differentiated between calcium and uric acid stones with an AUC of 0.78 (p < 0.01, 95% CI 0.73-0.83), 0.79 sensitivity, and 0.90 specificity regardless of CT vendors (GE CT: AUC = 0.82, p < 0.01, 95% CI 0.740-0896; Siemens CT: AUC = 0.77, 95% CI 0.700-0.846, p < 0.01). CONCLUSION Automated threshold-based stone segmentation and radiomics can differentiate between calcium oxalate/phosphate and urate stones from non-contrast, single-energy abdomen CT.
Collapse
Affiliation(s)
- Parisa Kaviani
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA, 02114, USA
| | - Andrew Primak
- Siemens Medical Solutions USA Inc, Malvern, PA, 19355, USA
| | - Bernardo Bizzo
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA, 02114, USA.,MGH and BWH Center for Clinical Data Science, Boston, USA
| | - Shadi Ebrahimian
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA, 02114, USA
| | - Sanjay Saini
- Department of Radiology, Massachusetts General Hospital, 55 Fruit St, Boston, MA, 02114, USA
| | - Keith J Dreyer
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA, 02114, USA.,MGH and BWH Center for Clinical Data Science, Boston, USA.,Department of Radiology, Massachusetts General Hospital, 25 New Chardon Street, Boston, MA, 02114, USA
| | - Mannudeep K Kalra
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, 75 Blossom Court, Suite 248, Boston, MA, 02114, USA.
| |
Collapse
|
7
|
Gupta RV, Kalra MK, Ebrahimian S, Kaviani P, Primak A, Bizzo B, Dreyer KJ. Complex Relationship Between Artificial Intelligence and CT Radiation Dose. Acad Radiol 2022; 29:1709-1719. [PMID: 34836775 DOI: 10.1016/j.acra.2021.10.024] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2021] [Revised: 10/15/2021] [Accepted: 10/17/2021] [Indexed: 12/22/2022]
Abstract
Concerns over need for CT radiation dose optimization and reduction led to improved scanner efficiency and introduction of several reconstruction techniques and image processing-based software. The latest technologies use artificial intelligence (AI) for CT dose optimization and image quality improvement. While CT dose optimization has and can benefit from AI, variations in scanner technologies, reconstruction methods, and scan protocols can lead to substantial variations in radiation doses and image quality across and within different scanners. These variations in turn can influence performance of AI algorithms being deployed for tasks such as detection, segmentation, characterization, and quantification. We review the complex relationship between AI and CT radiation dose.
Collapse
Affiliation(s)
- Reya V Gupta
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, 75 Blossom Court, Suite 248, Boston, Massachusetts
| | - Mannudeep K Kalra
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, 75 Blossom Court, Suite 248, Boston, Massachusetts.
| | - Shadi Ebrahimian
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, 75 Blossom Court, Suite 248, Boston, Massachusetts
| | - Parisa Kaviani
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, 75 Blossom Court, Suite 248, Boston, Massachusetts
| | - Andrew Primak
- Siemens Medical Solutions USA Inc, Malvern, Pennsylvania
| | - Bernardo Bizzo
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, 75 Blossom Court, Suite 248, Boston, Massachusetts; MGH & BWH Center for Clinical Data Science, Boston, Massachusetts
| | - Keith J Dreyer
- Department of Radiology, Massachusetts General Hospital and Harvard Medical School, 75 Blossom Court, Suite 248, Boston, Massachusetts; MGH & BWH Center for Clinical Data Science, Boston, Massachusetts
| |
Collapse
|
8
|
Ahn JS, Ebrahimian S, McDermott S, Lee S, Naccarato L, Di Capua JF, Wu MY, Zhang EW, Muse V, Miller B, Sabzalipour F, Bizzo BC, Dreyer KJ, Kaviani P, Digumarthy SR, Kalra MK. Association of Artificial Intelligence-Aided Chest Radiograph Interpretation With Reader Performance and Efficiency. JAMA Netw Open 2022; 5:e2229289. [PMID: 36044215 PMCID: PMC9434361 DOI: 10.1001/jamanetworkopen.2022.29289] [Citation(s) in RCA: 28] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/01/2023] Open
Abstract
IMPORTANCE The efficient and accurate interpretation of radiologic images is paramount. OBJECTIVE To evaluate whether a deep learning-based artificial intelligence (AI) engine used concurrently can improve reader performance and efficiency in interpreting chest radiograph abnormalities. DESIGN, SETTING, AND PARTICIPANTS This multicenter cohort study was conducted from April to November 2021 and involved radiologists, including attending radiologists, thoracic radiology fellows, and residents, who independently participated in 2 observer performance test sessions. The sessions included a reading session with AI and a session without AI, in a randomized crossover manner with a 4-week washout period in between. The AI produced a heat map and the image-level probability of the presence of the referrable lesion. The data used were collected at 2 quaternary academic hospitals in Boston, Massachusetts: Beth Israel Deaconess Medical Center (The Medical Information Mart for Intensive Care Chest X-Ray [MIMIC-CXR]) and Massachusetts General Hospital (MGH). MAIN OUTCOMES AND MEASURES The ground truths for the labels were created via consensual reading by 2 thoracic radiologists. Each reader documented their findings in a customized report template, in which the 4 target chest radiograph findings and the reader confidence of the presence of each finding was recorded. The time taken for reporting each chest radiograph was also recorded. Sensitivity, specificity, and area under the receiver operating characteristic curve (AUROC) were calculated for each target finding. RESULTS A total of 6 radiologists (2 attending radiologists, 2 thoracic radiology fellows, and 2 residents) participated in the study. The study involved a total of 497 frontal chest radiographs-247 from the MIMIC-CXR data set (demographic data for patients were not available) and 250 chest radiographs from MGH (mean [SD] age, 63 [16] years; 133 men [53.2%])-from adult patients with and without 4 target findings (pneumonia, nodule, pneumothorax, and pleural effusion). The target findings were found in 351 of 497 chest radiographs. The AI was associated with higher sensitivity for all findings compared with the readers (nodule, 0.816 [95% CI, 0.732-0.882] vs 0.567 [95% CI, 0.524-0.611]; pneumonia, 0.887 [95% CI, 0.834-0.928] vs 0.673 [95% CI, 0.632-0.714]; pleural effusion, 0.872 [95% CI, 0.808-0.921] vs 0.889 [95% CI, 0.862-0.917]; pneumothorax, 0.988 [95% CI, 0.932-1.000] vs 0.792 [95% CI, 0.756-0.827]). AI-aided interpretation was associated with significantly improved reader sensitivities for all target findings, without negative impacts on the specificity. Overall, the AUROCs of readers improved for all 4 target findings, with significant improvements in detection of pneumothorax and nodule. The reporting time with AI was 10% lower than without AI (40.8 vs 36.9 seconds; difference, 3.9 seconds; 95% CI, 2.9-5.2 seconds; P < .001). CONCLUSIONS AND RELEVANCE These findings suggest that AI-aided interpretation was associated with improved reader performance and efficiency for identifying major thoracic findings on a chest radiograph.
Collapse
Affiliation(s)
| | - Shadi Ebrahimian
- Division of Thoracic Imaging, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts
- Internal Medicine, Icahn School of Medicine at Mount Sinai, Elmhurst Hospital Center, Elmhurst, New York
| | - Shaunagh McDermott
- Division of Thoracic Imaging, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts
| | | | - Laura Naccarato
- Division of Thoracic Imaging, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts
| | - John F. Di Capua
- Division of Thoracic Imaging, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts
| | - Markus Y. Wu
- Division of Thoracic Imaging, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts
| | - Eric W. Zhang
- Division of Thoracic Imaging, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts
| | - Victorine Muse
- Division of Thoracic Imaging, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts
| | - Benjamin Miller
- Division of Thoracic Imaging, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts
- Data Science Office, Mass General Brigham, Boston, Massachusetts
| | - Farid Sabzalipour
- Division of Thoracic Imaging, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts
- Data Science Office, Mass General Brigham, Boston, Massachusetts
| | - Bernardo C. Bizzo
- Division of Thoracic Imaging, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts
- Data Science Office, Mass General Brigham, Boston, Massachusetts
| | - Keith J. Dreyer
- Division of Thoracic Imaging, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts
- Data Science Office, Mass General Brigham, Boston, Massachusetts
| | - Parisa Kaviani
- Division of Thoracic Imaging, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts
| | - Subba R. Digumarthy
- Division of Thoracic Imaging, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts
| | - Mannudeep K. Kalra
- Division of Thoracic Imaging, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, Massachusetts
- Data Science Office, Mass General Brigham, Boston, Massachusetts
| |
Collapse
|
9
|
Kaviani P, Mokhtari J, Mirvakili S. Integral form of the control rod calibration curve in the new core configuration of HWZPR using rod insertion method. Progress in Nuclear Energy 2020. [DOI: 10.1016/j.pnucene.2020.103375] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|