1
|
Park SH, Han K, Lee JG. Conceptual review of outcome metrics and measures used in clinical evaluation of artificial intelligence in radiology. LA RADIOLOGIA MEDICA 2024:10.1007/s11547-024-01886-9. [PMID: 39225919 DOI: 10.1007/s11547-024-01886-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/01/2024] [Accepted: 08/21/2024] [Indexed: 09/04/2024]
Abstract
Artificial intelligence (AI) has numerous applications in radiology. Clinical research studies to evaluate the AI models are also diverse. Consequently, diverse outcome metrics and measures are employed in the clinical evaluation of AI, presenting a challenge for clinical radiologists. This review aims to provide conceptually intuitive explanations of the outcome metrics and measures that are most frequently used in clinical research, specifically tailored for clinicians. While we briefly discuss performance metrics for AI models in binary classification, detection, or segmentation tasks, our primary focus is on less frequently addressed topics in published literature. These include metrics and measures for evaluating multiclass classification; those for evaluating generative AI models, such as models used in image generation or modification and large language models; and outcome measures beyond performance metrics, including patient-centered outcome measures. Our explanations aim to guide clinicians in the appropriate use of these metrics and measures.
Collapse
Affiliation(s)
- Seong Ho Park
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, 88, Olympic-Ro 43-Gil, Songpa-Gu, Seoul, 05505, South Korea.
| | - Kyunghwa Han
- Department of Radiology, Research Institute of Radiological Science and Center for Clinical Imaging Data Science, Yonsei University College of Medicine, Seoul, South Korea
| | - June-Goo Lee
- Biomedical Engineering Research Center, Asan Institute for Life Sciences, University of Ulsan College of Medicine, Seoul, South Korea
| |
Collapse
|
2
|
Hwang MH, Kang S, Lee JW, Lee G. Deep Learning-Based Reconstruction Algorithm With Lung Enhancement Filter for Chest CT: Effect on Image Quality and Ground Glass Nodule Sharpness. Korean J Radiol 2024; 25:833-842. [PMID: 39197828 PMCID: PMC11361802 DOI: 10.3348/kjr.2024.0472] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2024] [Revised: 07/04/2024] [Accepted: 07/18/2024] [Indexed: 09/01/2024] Open
Abstract
OBJECTIVE To assess the effect of a new lung enhancement filter combined with deep learning image reconstruction (DLIR) algorithm on image quality and ground-glass nodule (GGN) sharpness compared to hybrid iterative reconstruction or DLIR alone. MATERIALS AND METHODS Five artificial spherical GGNs with various densities (-250, -350, -450, -550, and -630 Hounsfield units) and 10 mm in diameter were placed in a thorax anthropomorphic phantom. Four scans at four different radiation dose levels were performed using a 256-slice CT (Revolution Apex CT, GE Healthcare). Each scan was reconstructed using three different reconstruction algorithms: adaptive statistical iterative reconstruction-V at a level of 50% (AR50), Truefidelity (TF), which is a DLIR method, and TF with a lung enhancement filter (TF + Lu). Thus, 12 sets of reconstructed images were obtained and analyzed. Image noise, signal-to-noise ratio, and contrast-to-noise ratio were compared among the three reconstruction algorithms. Nodule sharpness was compared among the three reconstruction algorithms using the full-width at half-maximum value. Furthermore, subjective image quality analysis was performed. RESULTS AR50 demonstrated the highest level of noise, which was decreased by using TF + Lu and TF alone (P = 0.001). TF + Lu significantly improved nodule sharpness at all radiation doses compared to TF alone (P = 0.001). The nodule sharpness of TF + Lu was similar to that of AR50. Using TF alone resulted in the lowest nodule sharpness. CONCLUSION Adding a lung enhancement filter to DLIR (TF + Lu) significantly improved the nodule sharpness compared to DLIR alone (TF). TF + Lu can be an effective reconstruction technique to enhance image quality and GGN evaluation in ultralow-dose chest CT scans.
Collapse
Affiliation(s)
- Min-Hee Hwang
- Department of Radiology and Medical Research Institute, Pusan National University Hospital, Pusan National University School of Medicine, Busan, Republic of Korea
| | | | - Ji Won Lee
- Department of Radiology and Medical Research Institute, Pusan National University Hospital, Pusan National University School of Medicine, Busan, Republic of Korea
| | - Geewon Lee
- Department of Radiology and Medical Research Institute, Pusan National University Hospital, Pusan National University School of Medicine, Busan, Republic of Korea.
| |
Collapse
|
3
|
Cho HH, Lee SM, You SK. Assessment of deep learning image reconstruction (DLIR) on image quality in pediatric cardiac CT datasets type of manuscript: Original research. PLoS One 2024; 19:e0300090. [PMID: 39186484 PMCID: PMC11346658 DOI: 10.1371/journal.pone.0300090] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2024] [Accepted: 05/28/2024] [Indexed: 08/28/2024] Open
Abstract
BAKGROUND To evaluate the quantitative and qualitative image quality using deep learning image reconstruction (DLIR) of pediatric cardiac computed tomography (CT) compared with conventional image reconstruction methods. METHODS Between January 2020 and December 2022, 109 pediatric cardiac CT scans were included in this study. The CT scans were reconstructed using an adaptive statistical iterative reconstruction-V (ASiR-V) with a blending factor of 80% and three levels of DLIR with TrueFidelity (low-, medium-, and high-strength settings). Quantitative image quality was measured using signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR). The edge rise distance (ERD) and angle between 25% and 75% of the line density profile were drawn to evaluate sharpness. Qualitative image quality was assessed using visual grading analysis scores. RESULTS A gradual improvement in the SNR and CNR was noted among the strength levels of the DLIR in sequence from low to high. Compared to ASiR-V, high-level DLIR showed significantly improved SNR and CNR (P<0.05). ERD decreased with increasing angle as the level of DLIR increased. CONCLUSION High-level DLIR showed improved SNR and CNR compared to ASiR-V, with better sharpness on pediatric cardiac CT scans.
Collapse
Affiliation(s)
- Hyun-Hae Cho
- Department of Radiology and Medical Research Institute, College of Medicine, Ewha Womans University Seoul Hospital, Seoul, Republic of Korea
| | - So Mi Lee
- Department of Radiology, School of Medicine, Kyungpook National University, Kyungpook National University Hospital, Daegu, South Korea
| | - Sun Kyoung You
- Department of Radiology, Chungnam National University Hospital, Daejeon, Republic of Korea
| |
Collapse
|
4
|
Muthukrishnan V, Jaipurkar S, Damodaran N. Continuum topological derivative - a novel application tool for denoising CT and MRI medical images. BMC Med Imaging 2024; 24:182. [PMID: 39048968 PMCID: PMC11267933 DOI: 10.1186/s12880-024-01341-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2023] [Accepted: 06/18/2024] [Indexed: 07/27/2024] Open
Abstract
BACKGROUND CT and MRI modalities are important diagnostics tools for exploring the anatomical and tissue properties, respectively of the human beings. Several advancements like HRCT, FLAIR and Propeller have advantages in diagnosing the diseases very accurately, but still have enough space for improvements due to the presence of inherent and instrument noises. In the case of CT and MRI, the quantum mottle and the Gaussian and Rayleigh noises, respectively are still present in their advanced modalities of imaging. This paper addresses the denoising problem with continuum topological derivative technique and proved its trustworthiness based on the comparative study with other traditional filtration methods such as spatial, adaptive, frequency and transformation techniques using measures like visual inspection and performance metrics. METHODS This research study focuses on identifying a novel method for denoising by testing different filters on HRCT (High-Resolution Computed Tomography) and MR (Magnetic Resonance) images. The images were acquired from the Image Art Radiological Scan Centre using the SOMATOM CT and SIGNA Explorer (operating at 1.5 Tesla) machines. To compare the performance of the proposed CTD (Continuum Topological Derivative) method, various filters were tested on both HRCT and MR images. The filters tested for comparison were Gaussian (2D convolution operator), Wiener (deconvolution operator), Laplacian and Laplacian diagonal (2nd order partial differential operator), Average, Minimum, and Median (ordinary spatial operators), PMAD (Anisotropic diffusion operator), Kuan (statistical operator), Frost (exponential convolution operator), and HAAR Wavelet (time-frequency operator). The purpose of the study was to evaluate the effectiveness of the CTD method in removing noise compared to the other filters. The performance metrics were analyzed to assess the diligence of noise removal achieved by the CTD method. The primary outcome of the study was the removal of quantum mottle noise in HRCT images, while the secondary outcome focused on removing Gaussian (foreground) and Rayleigh (background) noise in MR images. The study aimed to observe the dynamics of noise removal by examining the values of the performance metrics. In summary, this study aimed to assess the denoising ability of various filters in HRCT and MR images, with the CTD method being the proposed approach. The study evaluated the performance of each filter using specific metrics and compared the results to determine the effectiveness of the CTD method in removing noise from the images. RESULTS Based on the calculated performance metric values, it has been observed that the CTD method successfully removed quantum mottle noise in HRCT images and Gaussian as well as Rayleigh noise in MRI. This can be evidenced by the PSNR (Peak Signal-to-Noise Ratio) metric, which consistently exhibited values ranging from 50 to 65 for all the tested images. Additionally, the CTD method demonstrated remarkably low residual values, typically on the order of e-09, which is a distinctive characteristic across all the images. Furthermore, the performance metrics of the CTD method consistently outperformed those of the other tested methods. Consequently, the results of this study have significant implications for the quality, structural similarity, and contrast of HRCT and MR images, enabling clinicians to obtain finer details for diagnostic purposes. CONCLUSION Continuum topological derivative algorithm is found to be constructive in removing prominent noises in both CT and MRI images and can serve as a potential tool for recognition of anatomical details in case of diseased and normal ones. The results obtained from this research work are highly inspiring and offer great promise in obtaining accurate diagnostic information for critical cases such as Thoracic Cavity Carina, Brain SPI Globe Lens 4th Ventricle, Brain-Middle Cerebral Artery, Brain-Middle Cerebral Artery and neoplastic lesions. These findings lay the foundation for implementing the proposed CTD technique in routine clinical diagnosis.
Collapse
Affiliation(s)
- Viswanath Muthukrishnan
- Central Instrumentation & Service Laboratory, Guindy Campus, University of Madras, Chennai, India
| | | | - Nedumaran Damodaran
- Central Instrumentation & Service Laboratory, Guindy Campus, University of Madras, Chennai, India.
| |
Collapse
|
5
|
Choi HU, Cho J, Hwang J, Lee S, Chang W, Park JH, Lee KH. Diagnostic performance and image quality of an image-based denoising algorithm applied to radiation dose-reduced CT in diagnosing acute appendicitis. Abdom Radiol (NY) 2024; 49:1839-1849. [PMID: 38411690 PMCID: PMC11213764 DOI: 10.1007/s00261-024-04246-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/10/2023] [Revised: 02/07/2024] [Accepted: 02/07/2024] [Indexed: 02/28/2024]
Abstract
PURPOSE To evaluate diagnostic performance and image quality of ultralow-dose CT (ULDCT) in diagnosing acute appendicitis with an image-based deep-learning denoising algorithm (IDLDA). METHODS This retrospective multicenter study included 180 patients (mean ± standard deviation, 29 ± 9 years; 91 female) who underwent contrast-enhanced 2-mSv CT for suspected appendicitis from February 2014 to August 2016. We simulated ULDCT from 2-mSv CT, reducing the dose by at least 50%. Then we applied an IDLDA on ULDCT to produce denoised ULDCT (D-ULDCT). Six radiologists with different experience levels (three board-certified radiologists and three residents) independently reviewed the ULDCT and D-ULDCT. They rated the likelihood of appendicitis and subjective image qualities (subjective image noise, diagnostic acceptability, and artificial sensation). One radiologist measured image noise, signal-to-noise ratio (SNR), and contrast-to-noise ratio (CNR). We used the receiver operating characteristic (ROC) analyses, Wilcoxon's signed-rank tests, and paired t-tests. RESULTS The area under the ROC curves (AUC) for diagnosing appendicitis ranged 0.90-0.97 for ULDCT and 0.94-0.97 for D-ULDCT. The AUCs of two residents were significantly higher on D-ULDCT (AUC difference = 0.06 [95% confidence interval, 0.01-0.11; p = .022] and 0.05 [0.00-0.10; p = .046], respectively). D-ULDCT provided better subjective image noise and diagnostic acceptability to all six readers. However, the response of board-certified radiologists and residents differed in artificial sensation (all p ≤ .003). D-ULDCT showed significantly lower image noise, higher SNR, and higher CNR (all p < .001). CONCLUSION An IDLDA can provide better ULDCT image quality and enhance diagnostic performance for less-experienced radiologists.
Collapse
Affiliation(s)
- Hyeon Ui Choi
- Department of Radiology, Seoul National University Bundang Hospital, Seongnam-si, Gyeonggi-do, Korea
| | - Jungheum Cho
- Department of Radiology, Seoul National University Bundang Hospital, Seongnam-si, Gyeonggi-do, Korea.
| | - Jinhee Hwang
- Department of Radiology, Seoul National University Bundang Hospital, Seongnam-si, Gyeonggi-do, Korea
- Department of Applied Bioengineering, Graduate School of Convergence Science and Technology, Seoul National University, Seoul, Korea
| | - Seungjae Lee
- Department of Radiology, Seoul National University Bundang Hospital, Seongnam-si, Gyeonggi-do, Korea
- Institute of Health and Environment, Seoul National University, Seoul, Korea
| | - Won Chang
- Department of Radiology, Seoul National University Bundang Hospital, Seongnam-si, Gyeonggi-do, Korea
| | - Ji Hoon Park
- Department of Radiology, Seoul National University Bundang Hospital, Seongnam-si, Gyeonggi-do, Korea
- Department of Applied Bioengineering, Graduate School of Convergence Science and Technology, Seoul National University, Seoul, Korea
- Department of Radiology, Seoul National University College of Medicine, Seoul National University Bundang Hospital, Seongnam-si, Gyeonggi-do, Korea
- Department of Medical Device Development, Seoul National University College of Medicine, Seoul, Korea
| | - Kyoung Ho Lee
- Department of Radiology, Seoul National University Bundang Hospital, Seongnam-si, Gyeonggi-do, Korea
- Department of Applied Bioengineering, Graduate School of Convergence Science and Technology, Seoul National University, Seoul, Korea
- Department of Radiology, Seoul National University College of Medicine, Seoul National University Bundang Hospital, Seongnam-si, Gyeonggi-do, Korea
- Department of Medical Device Development, Seoul National University College of Medicine, Seoul, Korea
| |
Collapse
|
6
|
Im JY, Halliburton SS, Mei K, Perkins AE, Wong E, Roshkovan L, Sandvold OF, Liu LP, Gang GJ, Noël PB. Patient-derived PixelPrint phantoms for evaluating clinical imaging performance of a deep learning CT reconstruction algorithm. Phys Med Biol 2024; 69:115009. [PMID: 38604190 PMCID: PMC11097966 DOI: 10.1088/1361-6560/ad3dba] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2023] [Revised: 03/22/2024] [Accepted: 04/11/2024] [Indexed: 04/13/2024]
Abstract
Objective. Deep learning reconstruction (DLR) algorithms exhibit object-dependent resolution and noise performance. Thus, traditional geometric CT phantoms cannot fully capture the clinical imaging performance of DLR. This study uses a patient-derived 3D-printed PixelPrint lung phantom to evaluate a commercial DLR algorithm across a wide range of radiation dose levels.Method. The lung phantom used in this study is based on a patient chest CT scan containing ground glass opacities and was fabricated using PixelPrint 3D-printing technology. The phantom was placed inside two different size extension rings to mimic a small- and medium-sized patient and was scanned on a conventional CT scanner at exposures between 0.5 and 20 mGy. Each scan was reconstructed using filtered back projection (FBP), iterative reconstruction, and DLR at five levels of denoising. Image noise, contrast to noise ratio (CNR), root mean squared error, structural similarity index (SSIM), and multi-scale SSIM (MS SSIM) were calculated for each image.Results.DLR demonstrated superior performance compared to FBP and iterative reconstruction for all measured metrics in both phantom sizes, with better performance for more aggressive denoising levels. DLR was estimated to reduce dose by 25%-83% in the small phantom and by 50%-83% in the medium phantom without decreasing image quality for any of the metrics measured in this study. These dose reduction estimates are more conservative compared to the estimates obtained when only considering noise and CNR.Conclusion. DLR has the capability of producing diagnostic image quality at up to 83% lower radiation dose, which can improve the clinical utility and viability of lower dose CT scans. Furthermore, the PixelPrint phantom used in this study offers an improved testing environment with more realistic tissue structures compared to traditional CT phantoms, allowing for structure-based image quality evaluation beyond noise and contrast-based assessments.
Collapse
Affiliation(s)
- Jessica Y Im
- Department of Radiology, University of Pennsylvania, Philadelphia, PA, United States of America
- Department of Bioengineering, University of Pennsylvania, Philadelphia, PA, United States of America
| | | | - Kai Mei
- Department of Radiology, University of Pennsylvania, Philadelphia, PA, United States of America
| | - Amy E Perkins
- Philips Healthcare, Cleveland, OH, United States of America
| | - Eddy Wong
- Philips Healthcare, Cleveland, OH, United States of America
| | - Leonid Roshkovan
- Department of Radiology, University of Pennsylvania, Philadelphia, PA, United States of America
| | - Olivia F Sandvold
- Department of Radiology, University of Pennsylvania, Philadelphia, PA, United States of America
- Department of Bioengineering, University of Pennsylvania, Philadelphia, PA, United States of America
| | - Leening P Liu
- Department of Radiology, University of Pennsylvania, Philadelphia, PA, United States of America
- Department of Bioengineering, University of Pennsylvania, Philadelphia, PA, United States of America
| | - Grace J Gang
- Department of Radiology, University of Pennsylvania, Philadelphia, PA, United States of America
| | - Peter B Noël
- Department of Radiology, University of Pennsylvania, Philadelphia, PA, United States of America
| |
Collapse
|
7
|
Jhang H, Park SJ, Sul AR, Jang HY, Park SH. Survey on Value Elements Provided by Artificial Intelligence and Their Eligibility for Insurance Coverage With an Emphasis on Patient-Centered Outcomes. Korean J Radiol 2024; 25:414-425. [PMID: 38627874 PMCID: PMC11058425 DOI: 10.3348/kjr.2023.1281] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2023] [Revised: 01/27/2024] [Accepted: 02/04/2024] [Indexed: 05/01/2024] Open
Abstract
OBJECTIVE This study aims to explore the opinions on the insurance coverage of artificial intelligence (AI), as categorized based on the distinct value elements offered by AI, with a specific focus on patient-centered outcomes (PCOs). PCOs are distinguished from traditional clinical outcomes and focus on patient-reported experiences and values such as quality of life, functionality, well-being, physical or emotional status, and convenience. MATERIALS AND METHODS We classified the value elements provided by AI into four dimensions: clinical outcomes, economic aspects, organizational aspects, and non-clinical PCOs. The survey comprised three sections: 1) experiences with PCOs in evaluating AI, 2) opinions on the coverage of AI by the National Health Insurance of the Republic of Korea when AI demonstrated benefits across the four value elements, and 3) respondent characteristics. The opinions regarding AI insurance coverage were assessed dichotomously and semi-quantitatively: non-approval (0) vs. approval (on a 1-10 weight scale, with 10 indicating the strongest approval). The survey was conducted from July 4 to 26, 2023, using a web-based method. Responses to PCOs and other value elements were compared. RESULTS Among 200 respondents, 44 (22%) were patients/patient representatives, 64 (32%) were industry/developers, 60 (30%) were medical practitioners/doctors, and 32 (16%) were government health personnel. The level of experience with PCOs regarding AI was low, with only 7% (14/200) having direct experience and 10% (20/200) having any experience (either direct or indirect). The approval rate for insurance coverage for PCOs was 74% (148/200), significantly lower than the corresponding rates for other value elements (82.5%-93.5%; P ≤ 0.034). The approval strength was significantly lower for PCOs, with a mean weight ± standard deviation of 5.1 ± 3.5, compared to other value elements (P ≤ 0.036). CONCLUSION There is currently limited demand for insurance coverage for AI that demonstrates benefits in terms of non-clinical PCOs.
Collapse
Affiliation(s)
- Hoyol Jhang
- Division of Healthcare Research, National Evidence-Based Healthcare Collaborating Agency, Seoul, Republic of Korea
| | - So Jin Park
- Division of Healthcare Research, National Evidence-Based Healthcare Collaborating Agency, Seoul, Republic of Korea
| | - Ah-Ram Sul
- Division of Healthcare Research, National Evidence-Based Healthcare Collaborating Agency, Seoul, Republic of Korea.
| | - Hye Young Jang
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | - Seong Ho Park
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea.
| |
Collapse
|
8
|
Lyu P, Li Z, Chen Y, Wang H, Liu N, Liu J, Zhan P, Liu X, Shang B, Wang L, Gao J. Deep learning reconstruction CT for liver metastases: low-dose dual-energy vs standard-dose single-energy. Eur Radiol 2024; 34:28-38. [PMID: 37532899 DOI: 10.1007/s00330-023-10033-3] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/26/2022] [Revised: 05/28/2023] [Accepted: 06/05/2023] [Indexed: 08/04/2023]
Abstract
OBJECTIVES To assess image quality and liver metastasis detection of reduced-dose dual-energy CT (DECT) with deep learning image reconstruction (DLIR) compared to standard-dose single-energy CT (SECT) with DLIR or iterative reconstruction (IR). METHODS In this prospective study, two groups of 40 participants each underwent abdominal contrast-enhanced scans with full-dose SECT (120-kVp images, DLIR and IR algorithms) or reduced-dose DECT (40- to 60-keV virtual monochromatic images [VMIs], DLIR algorithm), with 122 and 106 metastases, respectively. Groups were matched by age, sex ratio, body mass index, and cross-sectional area. Noise power spectrum of liver images and task-based transfer function of metastases were calculated to assess the noise texture and low-contrast resolution. The image noise, signal-to-noise ratios (SNR) of liver and portal vein, liver-to-lesion contrast-to-noise ratio (LLR), lesion conspicuity, lesion detection rate, and the subjective image quality metrics were compared between groups on 1.25-mm reconstructed images. RESULTS Compared to 120-kVp images with IR, 40- and 50-keV VMIs with DLIR showed similar noise texture and LLR, similar or higher image noise and low-contrast resolution, improved SNR and lesion conspicuity, and similar or better perceptual image quality. When compared to 120-kVp images with DLIR, 50-keV VMIs with DLIR had similar low-contrast resolution, SNR, LLR, lesion conspicuity, and perceptual image quality but lower frequency noise texture and higher image noise. For the detection of hepatic metastases, reduced-dose DECT by 34% maintained observer lesion detection rates. CONCLUSION DECT assisted with DLIR enables a 34% dose reduction for detecting hepatic metastases while maintaining comparable perceptual image quality to full-dose SECT. CLINICAL RELEVANCE STATEMENT Reduced-dose dual-energy CT with deep learning image reconstruction is as accurate as standard-dose single-energy CT for the detection of liver metastases and saves more than 30% of the radiation dose. KEY POINTS • The 40- and 50-keV virtual monochromatic images (VMIs) with deep learning image reconstruction (DLIR) improved lesion conspicuity compared with 120-kVp images with iterative reconstruction while providing similar or better perceptual image quality. • The 50-keV VMIs with DLIR provided comparable perceptual image quality and lesion conspicuity to 120-kVp images with DLIR. • The reduction of radiation by 34% by DLIR in low-keV VMIs is clinically sufficient for detecting low-contrast hepatic metastases.
Collapse
Affiliation(s)
- Peijie Lyu
- Department of Radiology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, Henan Province, China
| | - Zhen Li
- Department of Interventional Radiology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, Henan Province, China
| | - Yan Chen
- Department of Radiology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, Henan Province, China
| | - Huixia Wang
- Department of Radiology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, Henan Province, China
| | - Nana Liu
- Department of Radiology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, Henan Province, China
| | - Jie Liu
- Department of Radiology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, Henan Province, China
| | - Pengchao Zhan
- Department of Radiology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, Henan Province, China
| | - Xing Liu
- Department of Radiology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, Henan Province, China
| | - Bo Shang
- Department of Radiology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, Henan Province, China
| | - Luotong Wang
- CT Imaging Research Center, GE Healthcare China, Beijing, China
| | - Jianbo Gao
- Department of Radiology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, Henan Province, China.
| |
Collapse
|
9
|
Im JY, Halliburton SS, Mei K, Perkins AE, Wong E, Roshkovan L, Sandvold OF, Liu LP, Gang GJ, Noël PB. Patient-derived PixelPrint phantoms for evaluating clinical imaging performance of a deep learning CT reconstruction algorithm. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.12.07.23299625. [PMID: 38106064 PMCID: PMC10723564 DOI: 10.1101/2023.12.07.23299625] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/19/2023]
Abstract
Objective Deep learning reconstruction (DLR) algorithms exhibit object-dependent resolution and noise performance. Thus, traditional geometric CT phantoms cannot fully capture the clinical imaging performance of DLR. This study uses a patient-derived 3D-printed PixelPrint lung phantom to evaluate a commercial DLR algorithm across a wide range of radiation dose levels. Approach The lung phantom used in this study is based on a patient chest CT scan containing ground glass opacities and was fabricated using PixelPrint 3D-printing technology. The phantom was placed inside two different sized extension rings to mimic a small and medium sized patient and was scanned on a conventional CT scanner at exposures between 0.5 and 20 mGy. Each scan was reconstructed using filtered back projection (FBP), iterative reconstruction, and DLR at five levels of denoising. Image noise, contrast to noise ratio (CNR), root mean squared error (RMSE), structural similarity index (SSIM), and multi-scale SSIM (MS SSIM) were calculated for each image. Main Results DLR demonstrated superior performance compared to FBP and iterative reconstruction for all measured metrics in both phantom sizes, with better performance for more aggressive denoising levels. DLR was estimated to reduce dose by 25-83% in the small phantom and by 50-83% in the medium phantom without decreasing image quality for any of the metrics measured in this study. These dose reduction estimates are more conservative compared to the estimates obtained when only considering noise and CNR with a non-anatomical physics phantom. Significance DLR has the capability of producing diagnostic image quality at up to 83% lower radiation dose which can improve the clinical utility and viability of lower dose CT scans. Furthermore, the PixelPrint phantom used in this study offers an improved testing environment with more realistic tissue structures compared to traditional CT phantoms, allowing for structure-based image quality evaluation beyond noise and contrast-based assessments.
Collapse
|
10
|
Kim DY, Oh HW, Suh CH. Reporting Quality of Research Studies on AI Applications in Medical Images According to the CLAIM Guidelines in a Radiology Journal With a Strong Prominence in Asia. Korean J Radiol 2023; 24:1179-1189. [PMID: 38016678 PMCID: PMC10701000 DOI: 10.3348/kjr.2023.1027] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2023] [Revised: 10/25/2023] [Accepted: 10/26/2023] [Indexed: 11/30/2023] Open
Abstract
OBJECTIVE We aimed to evaluate the reporting quality of research articles that applied deep learning to medical imaging. Using the Checklist for Artificial Intelligence in Medical Imaging (CLAIM) guidelines and a journal with prominence in Asia as a sample, we intended to provide an insight into reporting quality in the Asian region and establish a journal-specific audit. MATERIALS AND METHODS A total of 38 articles published in the Korean Journal of Radiology between June 2018 and January 2023 were analyzed. The analysis included calculating the percentage of studies that adhered to each CLAIM item and identifying items that were met by ≤ 50% of the studies. The article review was initially conducted independently by two reviewers, and the consensus results were used for the final analysis. We also compared adherence rates to CLAIM before and after December 2020. RESULTS Of the 42 items in the CLAIM guidelines, 12 items (29%) were satisfied by ≤ 50% of the included articles. None of the studies reported handling missing data (item #13). Only one study respectively presented the use of de-identification methods (#12), intended sample size (#19), robustness or sensitivity analysis (#30), and full study protocol (#41). Of the studies, 35% reported the selection of data subsets (#10), 40% reported registration information (#40), and 50% measured inter and intrarater variability (#18). No significant changes were observed in the rates of adherence to these 12 items before and after December 2020. CONCLUSION The reporting quality of artificial intelligence studies according to CLAIM guidelines, in our study sample, showed room for improvement. We recommend that the authors and reviewers have a solid understanding of the relevant reporting guidelines and ensure that the essential elements are adequately reported when writing and reviewing the manuscripts for publication.
Collapse
Affiliation(s)
- Dong Yeong Kim
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea
| | | | - Chong Hyun Suh
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea.
| |
Collapse
|
11
|
Park SH. KJR Honors Most Impactful Article and Distinguished Reviewers for 2023. Korean J Radiol 2023; 24:949-951. [PMID: 37724592 PMCID: PMC10550748 DOI: 10.3348/kjr.2023.0810] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/27/2023] [Accepted: 08/27/2023] [Indexed: 09/21/2023] Open
Affiliation(s)
- Seong Ho Park
- Department of Radiology and Research Institute of Radiology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea.
| |
Collapse
|
12
|
Shim JH, Choi SY, Chang IH, Park SB. Dose Optimization Using a Deep Learning Tool in Various CT Protocols for Urolithiasis: A Physical Human Phantom Study. MEDICINA (KAUNAS, LITHUANIA) 2023; 59:1677. [PMID: 37763796 PMCID: PMC10538199 DOI: 10.3390/medicina59091677] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/06/2023] [Revised: 09/12/2023] [Accepted: 09/14/2023] [Indexed: 09/29/2023]
Abstract
Background and Objectives: We attempted to determine the optimal radiation dose to maintain image quality using a deep learning application in a physical human phantom. Materials and Methods: Three 5 × 5 × 5 mm3 uric acid stones were placed in a physical human phantom in various locations. Three tube voltages (120, 100, and 80 kV) and four current-time products (100, 70, 30, and 15 mAs) were implemented in 12 scans. Each scan was reconstructed with filtered back projection (FBP), statistical iterative reconstruction (IR, iDose), and knowledge-based iterative model reconstruction (IMR). By applying deep learning to each image, we took 12 more scans. Objective image assessments were calculated using the standard deviation of the Hounsfield unit (HU). Subjective image assessments were performed by one radiologist and one urologist. Two radiologists assessed the subjective assessment and found the stone under the absence of information. We used this data to calculate the diagnostic accuracy. Results: Objective image noise was decreased after applying a deep learning tool in all images of FBP, iDose, and IMR. There was no statistical difference between iDose and deep learning-applied FBP images (10.1 ± 11.9, 9.5 ± 18.5 HU, p = 0.583, respectively). At a 100 kV-30 mAs setting, deep learning-applied FBP obtained a similar objective noise in approximately one third of the radiation doses compared to FBP. In radiation doses with settings lower than 100 kV-30 mAs, the subject image assessment (image quality, confidence level, and noise) showed deteriorated scores. Diagnostic accuracy was increased when the deep learning setting was lower than 100 kV-30 mAs, except for at 80 kV-15 mAs. Conclusions: At the setting of 100 kV-30 mAs or higher, deep learning-applied FBP did not differ in image quality compared to IR. At the setting of 100 kV-30 mAs, the radiation dose can decrease by about one third while maintaining objective noise.
Collapse
Affiliation(s)
- Jae Hun Shim
- Department of Urology, Chung-Ang University Hospital, Chung-Ang University College of Medicine, Seoul 06973, Republic of Korea
| | - Se Young Choi
- Department of Urology, Chung-Ang University Hospital, Chung-Ang University College of Medicine, Seoul 06973, Republic of Korea
| | - In Ho Chang
- Department of Urology, Chung-Ang University Hospital, Chung-Ang University College of Medicine, Seoul 06973, Republic of Korea
| | - Sung Bin Park
- Department of Radiology, Chung-Ang University Hospital, Chung-Ang University College of Medicine, Seoul 06973, Republic of Korea
| |
Collapse
|
13
|
Lell M, Kachelrieß M. Computed Tomography 2.0: New Detector Technology, AI, and Other Developments. Invest Radiol 2023; 58:587-601. [PMID: 37378467 PMCID: PMC10332658 DOI: 10.1097/rli.0000000000000995] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Revised: 05/04/2023] [Indexed: 06/29/2023]
Abstract
ABSTRACT Computed tomography (CT) dramatically improved the capabilities of diagnostic and interventional radiology. Starting in the early 1970s, this imaging modality is still evolving, although tremendous improvements in scan speed, volume coverage, spatial and soft tissue resolution, as well as dose reduction have been achieved. Tube current modulation, automated exposure control, anatomy-based tube voltage (kV) selection, advanced x-ray beam filtration, and iterative image reconstruction techniques improved image quality and decreased radiation exposure. Cardiac imaging triggered the demand for high temporal resolution, volume acquisition, and high pitch modes with electrocardiogram synchronization. Plaque imaging in cardiac CT as well as lung and bone imaging demand for high spatial resolution. Today, we see a transition of photon-counting detectors from experimental and research prototype setups into commercially available systems integrated in patient care. Moreover, with respect to CT technology and CT image formation, artificial intelligence is increasingly used in patient positioning, protocol adjustment, and image reconstruction, but also in image preprocessing or postprocessing. The aim of this article is to give an overview of the technical specifications of up-to-date available whole-body and dedicated CT systems, as well as hardware and software innovations for CT systems in the near future.
Collapse
|
14
|
Shehata MA, Saad AM, Kamel S, Stanietzky N, Roman-Colon AM, Morani AC, Elsayes KM, Jensen CT. Deep-learning CT reconstruction in clinical scans of the abdomen: a systematic review and meta-analysis. Abdom Radiol (NY) 2023; 48:2724-2756. [PMID: 37280374 DOI: 10.1007/s00261-023-03966-2] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2023] [Revised: 05/13/2023] [Accepted: 05/16/2023] [Indexed: 06/08/2023]
Abstract
OBJECTIVE To perform a systematic literature review and meta-analysis of the two most common commercially available deep-learning algorithms for CT. METHODS We used PubMed, Scopus, Embase, and Web of Science to conduct systematic searches for studies assessing the most common commercially available deep-learning CT reconstruction algorithms: True Fidelity (TF) and Advanced intelligent Clear-IQ Engine (AiCE) in the abdomen of human participants since only these two algorithms currently have adequate published data for robust systematic analysis. RESULTS Forty-four articles fulfilled inclusion criteria. 32 studies evaluated TF and 12 studies assessed AiCE. DLR algorithms produced images with significantly less noise (22-57.3% less than IR) but preserved a desirable noise texture with increased contrast-to-noise ratios and improved lesion detectability on conventional CT. These improvements with DLR were similarly noted in dual-energy CT which was only assessed for a single vendor. Reported radiation reduction potential was 35.1-78.5%. Nine studies assessed observer performance with the two dedicated liver lesion studies being performed on the same vendor reconstruction (TF). These two studies indicate preserved low contrast liver lesion detection (> 5 mm) at CTDIvol 6.8 mGy (BMI 23.5 kg/m2) to 12.2 mGy (BMI 29 kg/m2). If smaller lesion detection and improved lesion characterization is needed, a CTDIvol of 13.6-34.9 mGy is needed in a normal weight to obese population. Mild signal loss and blurring have been reported at high DLR reconstruction strengths. CONCLUSION Deep learning reconstructions significantly improve image quality in CT of the abdomen. Assessment of other dose levels and clinical indications is needed. Careful choice of radiation dose levels is necessary, particularly for small liver lesion assessment.
Collapse
Affiliation(s)
- Mostafa A Shehata
- Department of Abdominal Imaging, The University of Texas MD Anderson Cancer Center, 1400 Pressler Street, Unit 1473, Houston, TX, 77030-4009, USA
| | | | - Serageldin Kamel
- Department of Lymphoma and Myeloma, The University of Texas MD Anderson Cancer Center, 1400 Pressler Street, Unit 1473, Houston, TX, 77030-4009, USA
| | - Nir Stanietzky
- Department of Abdominal Imaging, The University of Texas MD Anderson Cancer Center, 1400 Pressler Street, Unit 1473, Houston, TX, 77030-4009, USA
| | | | - Ajaykumar C Morani
- Department of Abdominal Imaging, The University of Texas MD Anderson Cancer Center, 1400 Pressler Street, Unit 1473, Houston, TX, 77030-4009, USA
| | - Khaled M Elsayes
- Department of Abdominal Imaging, The University of Texas MD Anderson Cancer Center, 1400 Pressler Street, Unit 1473, Houston, TX, 77030-4009, USA
| | - Corey T Jensen
- Department of Abdominal Imaging, The University of Texas MD Anderson Cancer Center, 1400 Pressler Street, Unit 1473, Houston, TX, 77030-4009, USA.
| |
Collapse
|
15
|
Zhong J, Wang L, Shen H, Li J, Lu W, Shi X, Xing Y, Hu Y, Ge X, Ding D, Yan F, Du L, Yao W, Zhang H. Improving lesion conspicuity in abdominal dual-energy CT with deep learning image reconstruction: a prospective study with five readers. Eur Radiol 2023; 33:5331-5343. [PMID: 36976337 DOI: 10.1007/s00330-023-09556-6] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2022] [Revised: 02/07/2023] [Accepted: 02/17/2023] [Indexed: 03/29/2023]
Abstract
OBJECTIVES To evaluate image quality, diagnostic acceptability, and lesion conspicuity in abdominal dual-energy CT (DECT) using deep learning image reconstruction (DLIR) compared to those using adaptive statistical iterative reconstruction-V (Asir-V) at 50% blending (AV-50), and to identify potential factors impacting lesion conspicuity. METHODS The portal-venous phase scans in abdominal DECT of 47 participants with 84 lesions were prospectively included. The raw data were reconstructed to virtual monoenergetic image (VMI) at 50 keV using filtered back-projection (FBP), AV-50, and DLIR at low (DLIR-L), medium (DLIR-M), and high strength (DLIR-H). A noise power spectrum (NPS) was generated. CT number and standard deviation values of eight anatomical sites were measured. Signal-to-noise (SNR), and contrast-to-noise ratio (CNR) values were calculated. Five radiologists assessed image quality in terms of image contrast, image noise, image sharpness, artificial sensation, and diagnostic acceptability, and evaluated the lesion conspicuity. RESULTS DLIR further reduced image noise (p < 0.001) compared to AV-50 while better preserved the average NPS frequency (p < 0.001). DLIR maintained CT number values (p > 0.99) and improved SNR and CNR values compared to AV-50 (p < 0.001). DLIR-H and DLIR-M showed higher ratings in all image quality analyses than AV-50 (p < 0.001). DLIR-H provided significantly better lesion conspicuity than AV-50 and DLIR-M regardless of lesion size, relative CT attenuation to surrounding tissue, or clinical purpose (p < 0.05). CONCLUSIONS DLIR-H could be safely recommended for routine low-keV VMI reconstruction in daily contrast-enhanced abdominal DECT to improve image quality, diagnostic acceptability, and lesion conspicuity. KEY POINTS • DLIR is superior to AV-50 in noise reduction, with less shifts of the average spatial frequency of NPS towards low frequency, and larger improvements of NPS noise, noise peak, SNR, and CNR values. • DLIR-M and DLIR-H generate better image quality in terms of image contrast, noise, sharpness, artificial sensation, and diagnostic acceptability than AV-50, while DLIR-H provides better lesion conspicuity than AV-50 and DLIR-M. • DLIR-H could be safely recommended as a new standard for routine low-keV VMI reconstruction in contrast-enhanced abdominal DECT to provide better lesion conspicuity and better image quality than the standard AV-50.
Collapse
Affiliation(s)
- Jingyu Zhong
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200336, China
| | - Lingyun Wang
- Department of Radiology, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200025, China
| | - Hailin Shen
- Department of Radiology, Suzhou Kowloon Hospital, Shanghai Jiao Tong University School of Medicine, Suzhou, 215028, China
| | - Jianying Li
- Computed Tomography Research Center, GE Healthcare, Beijing, 100176, China
| | - Wei Lu
- Computed Tomography Research Center, GE Healthcare, Shanghai, 201203, China
| | - Xiaomeng Shi
- Department of Materials, Imperial College London, South Kensington Campus, London, SW7 2AZ, UK
| | - Yue Xing
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200336, China
| | - Yangfan Hu
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200336, China
| | - Xiang Ge
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200336, China
| | - Defang Ding
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200336, China
| | - Fuhua Yan
- Department of Radiology, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200025, China
| | - Lianjun Du
- Department of Radiology, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200025, China.
| | - Weiwu Yao
- Department of Imaging, Tongren Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200336, China.
| | - Huan Zhang
- Department of Radiology, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200025, China.
| |
Collapse
|