1
|
Pehrson LM, Li D, Mayar A, Fraccaro M, Bonnevie R, Sørensen PJ, Rykkje AM, Andersen TT, Steglich-Arnholm H, Stærk DMR, Borgwardt L, Darkner S, Carlsen JF, Nielsen MB, Ingala S. Clinicians' Agreement on Extrapulmonary Radiographic Findings in Chest X-Rays Using a Diagnostic Labelling Scheme. Diagnostics (Basel) 2025; 15:902. [PMID: 40218252 PMCID: PMC11988848 DOI: 10.3390/diagnostics15070902] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2025] [Revised: 03/18/2025] [Accepted: 03/20/2025] [Indexed: 04/14/2025] Open
Abstract
Objective: Reliable reading and annotation of chest X-ray (CXR) images are essential for both clinical decision-making and AI model development. While most of the literature emphasizes pulmonary findings, this study evaluates the consistency and reliability of annotations for extrapulmonary findings, using a labelling scheme. Methods: Six clinicians with varying experience levels (novice, intermediate, and experienced) annotated 100 CXR images using a diagnostic labelling scheme, in two rounds, separated by a three-week washout period. Annotation consistency was assessed using Randolph's free-marginal kappa (RK), prevalence- and bias-adjusted kappa (PABAK), proportion positive agreement (PPA), and proportion negative agreement (PNA). Pairwise comparisons and the McNemar's test were conducted to assess inter-reader and intra-reader agreement. Results: PABAK values indicated high overall grouped labelling agreement (novice: 0.86, intermediate: 0.90, experienced: 0.91). PNA values demonstrated strong agreement on negative findings, while PPA values showed moderate-to-low consistency in positive findings. Significant differences in specific agreement emerged between novice and experienced clinicians for eight labels, but there were no significant variations in RK across experience levels. The McNemar's test confirmed annotation stability between rounds. Conclusions: This study demonstrates that clinician annotations of extrapulmonary findings in CXR are consistent and reliable across different experience levels using a pre-defined diagnostic labelling scheme. These insights aid in optimizing training strategies for both clinicians and AI models.
Collapse
Affiliation(s)
- Lea Marie Pehrson
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
| | - Dana Li
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
| | - Alyas Mayar
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
| | | | | | - Peter Jagd Sørensen
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
| | - Alexander Malcom Rykkje
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
| | - Tobias Thostrup Andersen
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
| | - Henrik Steglich-Arnholm
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
| | - Dorte Marianne Rohde Stærk
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
| | - Lotte Borgwardt
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
| | - Sune Darkner
- Department of Computer Science, University of Copenhagen, 2100 Copenhagen, Denmark
| | - Jonathan Frederik Carlsen
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
| | - Michael Bachmann Nielsen
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, 2100 Copenhagen, Denmark
| | - Silvia Ingala
- Department of Diagnostic Radiology, Copenhagen University Hospital, Rigshospitalet, 2100 Copenhagen, Denmark
- Department of Diagnostic Radiology, Copenhagen University Hospital Herlev and Gentofte, 2730 Copenhagen, Denmark
- Cerebriu A/S, 1434 Copenhagen, Denmark
| |
Collapse
|
2
|
Kumar AA, Valakkada J, Ayyappan A, Kannath S. Basic Statistics for Radiologists: Part 1-Basic Data Interpretation and Inferential Statistics. Indian J Radiol Imaging 2025; 35:S58-S73. [PMID: 39802725 PMCID: PMC11717466 DOI: 10.1055/s-0044-1796644] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/16/2025] Open
Abstract
A systematic approach to statistical analysis is essential for accurate data interpretation and informed decision-making in the rapidly evolving field of radiology. This review provides a comprehensive overview of the fundamental statistical concepts for radiologists and clinicians. The first part of this series introduces foundational elements such as data types, distributions, descriptive and inferential statistics, hypothesis testing, and sampling methods. These are crucial for understanding the underlying structure of research data. The second part of this series delves deeper into advanced topics, including correlation and causality, regression analysis, survival curves, and the analysis of diagnostic tests using contingency tables and receiver operator characteristic (ROC) curves. These tools are vital for evaluating the efficacy of imaging techniques and drawing valid conclusions from clinical studies. As radiology continues to push the boundaries of technology and therapeutic interventions, mastering these statistical principles will empower radiologists to critically assess literature, conduct rigorous research, and contribute to evidence-based practices. Despite the pivotal role of statistics in radiology, formal training in these methodologies is still limited to a certain extent. This primer aims to bridge that gap, providing radiologists with the necessary tools to enhance diagnostic accuracy, optimize patient outcomes, and advance the field through robust research.
Collapse
Affiliation(s)
- Adarsh Anil Kumar
- Department of Imaging Sciences and Interventional Radiology, Sree Chitra Institute of Medical Sciences, Trivandrum, Kerala, India
| | - Jineesh Valakkada
- Department of Imaging Sciences and Interventional Radiology, Sree Chitra Institute of Medical Sciences, Trivandrum, Kerala, India
| | - Anoop Ayyappan
- Department of Imaging Sciences and Interventional Radiology, Sree Chitra Institute of Medical Sciences, Trivandrum, Kerala, India
| | - Santhosh Kannath
- Department of Imaging Sciences and Interventional Radiology, Sree Chitra Institute of Medical Sciences, Trivandrum, Kerala, India
| |
Collapse
|
3
|
Liu J, Xiao R, Yin H, Hu Y, Zhen S, Zhou S, Han D. Meta-analysis and systematic review of the diagnostic value of contrast-enhanced spectral mammography for the detection of breast cancer. BMJ Open 2024; 14:e069788. [PMID: 39231551 PMCID: PMC11407215 DOI: 10.1136/bmjopen-2022-069788] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/09/2022] [Accepted: 07/15/2024] [Indexed: 09/06/2024] Open
Abstract
OBJECTIVE The objective is to evaluate the diagnostic effectiveness of contrast-enhanced spectral mammography (CESM) in the diagnosis of breast cancer. DESIGN DATA SOURCES: PubMed, Embase and Cochrane libraries up to 18 June 2022. ELIGIBILITY CRITERIA FOR SELECTING STUDIES We included trials studies, compared the results of different researchers on CESM in the diagnosis of breast cancer, and calculated the diagnostic value of CESM for breast cancer. DATA EXTRACTION AND SYNTHESIS Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2) evaluated the methodological quality of all the included studies. The study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses specification. In addition to sensitivity and specificity, other important parameters were explored in an analysis of CESM accuracy for breast cancer diagnosis. For overall accuracy estimation, summary receiver operating characteristic curves were calculated. STATA V.14.0 was used for all analyses. RESULTS This meta-analysis included a total of 12 studies. According to the summary estimates for CESM in the diagnosis of breast cancer, the pooled sensitivity and specificity were 0.97 (95% CI 0.92 to 0.98) and 0.76 (95% CI 0.64 to 0.85), respectively. Positive likelihood ratio was 4.03 (95% CI 2.65 to 6.11), negative likelihood ratio was 0.05 (95% CI 0.02 to 0.09) and the diagnostic odds ratio was 89.49 (95% CI 45.78 to 174.92). Moreover, there was a 0.95 area under the curve. CONCLUSIONS The CESM has high sensitivity and good specificity when it comes to evaluating breast cancer, particularly in women with dense breasts. Thus, provide more information for clinical diagnosis and treatment.
Collapse
Affiliation(s)
- Jiulin Liu
- Department of Magnetic Resonance Imaging (MRI), The First Affiliated Hospital of Xinxiang Medical University, Weihui, Henan, China
- Department of Radiology, Luoyang Orthopedic-Traumatological Hospital of Henan Province (Henan Provincial Orthopedic Hospital), Zhengzhou, Henan, China
| | - Ran Xiao
- Department of Respiratory Medicine, The First Affiliated Hospital of Xinxiang Medical University, Weihui, Henan, China
| | - Huijia Yin
- Department of Magnetic Resonance Imaging (MRI), The First Affiliated Hospital of Xinxiang Medical University, Weihui, Henan, China
| | - Ying Hu
- Department of Magnetic Resonance Imaging (MRI), The First Affiliated Hospital of Xinxiang Medical University, Weihui, Henan, China
| | - Siyu Zhen
- Department of Magnetic Resonance Imaging (MRI), The First Affiliated Hospital of Xinxiang Medical University, Weihui, Henan, China
| | - Shihao Zhou
- Department of Magnetic Resonance Imaging (MRI), The First Affiliated Hospital of Xinxiang Medical University, Weihui, Henan, China
- Department of Radiology, Luoyang Orthopedic-Traumatological Hospital of Henan Province (Henan Provincial Orthopedic Hospital), Zhengzhou, Henan, China
| | - Dongming Han
- Department of Magnetic Resonance Imaging (MRI), The First Affiliated Hospital of Xinxiang Medical University, Weihui, Henan, China
| |
Collapse
|
4
|
Atzen SL. Top 10 Tips for Writing Materials and Methods in Radiology: A Brief Guide for Authors. Radiology 2024; 310:e240306. [PMID: 38501956 DOI: 10.1148/radiol.240306] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/20/2024]
Affiliation(s)
- Sarah L Atzen
- From the Radiological Society of North America, 820 Jorie Blvd, Oak Brook, IL 60523
| |
Collapse
|
5
|
Park S, Lee Y, Kim TS, Kim SK, Han JY. Response evaluation after immunotherapy in NSCLC: Early response assessment using FDG PET/CT. Medicine (Baltimore) 2020; 99:e23815. [PMID: 33371161 PMCID: PMC7748304 DOI: 10.1097/md.0000000000023815] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/04/2020] [Accepted: 11/19/2020] [Indexed: 11/26/2022] Open
Abstract
The present study aimed to evaluate the role of early F-18 2-deoxy-2-[fluorine-18] fluoro-D-glucose positron emission tomography/computed tomography (FDG PET/CT) in non-small cell lung cancer patients undergoing immune checkpoint inhibitor (ICI) treatment.Twenty-four non-small cell lung cancer patients who received nivolumab or pembrolizumab and underwent FDG PET/CT as an interim analysis after 2 or 3 cycles of ICI treatment were retrospectively enrolled. Tumor response was assessed using the PET Response Criteria in Solid Tumors 1.0 (PERCIST) and the European Organization for Research and Treatment of Cancer (EORTC) criteria after 2 or 3 cycles of ICI treatment (SCAN-1) and after an additional 2 cycles of ICI treatment (SCAN-2). The best overall response was determined by FDG PET/CT or chest CT at ≥ 3 months after therapy initiation, and the clinical benefit was investigated. progression-free survival was investigated, and its correlation with clinicopathologic and metabolic parameters was examined using a Cox multivariate proportional hazards model.In the interim analysis, 4 patients achieved a complete metabolic response (CMR), 1 patient exhibited a partial metabolic response (PMR), and 14 patients had Progressive metabolic disease (PMD) according to the PERCIST and EORTC criteria. Four patients showed stable metabolic disease (SMD) according to the PERCIST criteria, and 2 patients showed different responses (i.e., PMR) according to the EORTC criteria. Patients with a CMR or PMR at SCAN-1 had a clinical benefit. Among the 4 patients with SMD at SCAN-1, only 1 experienced a clinical benefit regardless of the percent change in the peak standardized uptake value. Two patients with discordant response assessments between the PERCIST and EORTC criteria showed conflicting clinical benefits. Among the 14 patients with PMD, none experienced any clinical benefit. Only metabolic parameters were significant factors for predicting progression in the multivariate analysis (peak standardized uptake value and metabolic tumor volume, HRs of 1.18 and 1.00, respectively).Based on early F-18 FDG PET/CT after ICI treatment, metabolic parameters could predict post-treatment progression. Responses after ICI treatment were correctly assessed in patients with a CMR, a PMR, and PMD, but patients with SMD required a meticulous follow-up because of varying clinical benefits.
Collapse
Affiliation(s)
- Sohyun Park
- Department of Nuclear Medicine, Guro Hospital, Korea University College of Medicine, Goyang, Republic of Korea
- Department of Nuclear Medicine
| | | | | | - Seok-ki Kim
- Department of Nuclear Medicine
- Molecular Imaging Branch, Research Institute, National Cancer Center, Goyang-si, Gyeonggi-do, Korea
| | | |
Collapse
|
6
|
Arm J, Ribbons K, Lechner-Scott J, Ramadan S. Evaluation of MS related central fatigue using MR neuroimaging methods: Scoping review. J Neurol Sci 2019; 400:52-71. [PMID: 30903860 DOI: 10.1016/j.jns.2019.03.007] [Citation(s) in RCA: 56] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2018] [Revised: 02/17/2019] [Accepted: 03/11/2019] [Indexed: 11/30/2022]
Abstract
BACKGROUND Fatigue is a common and debilitating symptom in multiple sclerosis (MS). Over the past decade, a growing body of research has focussed on the pathophysiological mechanisms underlying central (cognitive and physical) fatigue in MS. The precise mechanisms causing fatigue in MS patients are complex and poorly understood, and may differ between patients. Advanced quantitative magnetic resonance imaging (MRI) techniques allow for objective assessment of disease pathology and have been used to characterise the pathophysiology of central fatigue in MS. OBJECTIVE To systematically review the existing literature of MRI-based studies assessing the pathophysiological mechanisms of MS-related central fatigue. METHODS A systematic literature search of four major databases (PubMed, Medline, Embase, Scopus and Google Scholar) was conducted to identify MRI-based studies of MS-related fatigue published in the past 20 years. Studies using the following MRI-based methods were included: structural (lesion load/atrophy), T1 relaxation time/magnetisation transfer ratio (MTR), diffusion tensor imaging (DTI), functional MRI (fMRI) and magnetic resonance spectroscopy (MRS). RESULTS A total of 92 studies were identified as meeting the search criteria and included for review. Structurally, regional gray/white matter atrophy, cortical thinning, decreased T1 relaxation times and reduced fractional anisotropy were associated with central fatigue in MS. Functionally, hyperactivity and reduced functional connectivity in several regional areas of frontal, parietal, occipital, temporal and cerebellum were suggested as causes of central fatigue. Biochemically, a reduction in N-acetyl aspartate/creatine and increased (glutamine+glutamate)/creatine ratios were correlated with fatigue severity in MS. CONCLUSION Several advanced quantitative MRI methods have been employed in the study of central fatigue in MS. Central fatigue in MS is associated with macro/microstructural and functional changes within specific brain regions (frontal, parietal, temporal and deep gray matter) and specific pathways/networks (cortico-cortical and cortico-subcortical). Alternations in the cortico-striatal-thalamocortical (CSTC) loop are correlated with the development of fatigue in MS patients.
Collapse
Affiliation(s)
- Jameen Arm
- School of Health Sciences, Faculty of Health and Medicine, University of Newcastle, Callaghan, NSW 2308, Australia
| | - Karen Ribbons
- Department of Neurology, John Hunter Hospital, Lookout Road, New Lambton Heights, NSW 2305, Australia
| | - Jeannette Lechner-Scott
- School of Medicine and Public Health, Faculty of Health and Medicine, University of Newcastle, Callaghan, NSW 2308, Australia; Department of Neurology, John Hunter Hospital, Lookout Road, New Lambton Heights, NSW 2305, Australia; Hunter Medical Research Institute, Kookaburra Circuit, New Lambton Heights, NSW 2305, Australia
| | - Saadallah Ramadan
- School of Health Sciences, Faculty of Health and Medicine, University of Newcastle, Callaghan, NSW 2308, Australia; Hunter Medical Research Institute, Kookaburra Circuit, New Lambton Heights, NSW 2305, Australia.
| |
Collapse
|
7
|
Cho SY, Lipson EJ, Im HJ, Rowe SP, Gonzalez EM, Blackford A, Chirindel A, Pardoll DM, Topalian SL, Wahl RL. Prediction of Response to Immune Checkpoint Inhibitor Therapy Using Early-Time-Point 18F-FDG PET/CT Imaging in Patients with Advanced Melanoma. J Nucl Med 2017; 58:1421-1428. [PMID: 28360208 DOI: 10.2967/jnumed.116.188839] [Citation(s) in RCA: 168] [Impact Index Per Article: 21.0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2016] [Accepted: 03/13/2017] [Indexed: 12/25/2022] Open
Abstract
The purpose of this study was to evaluate 18F-FDG PET/CT scanning as an early predictor of response to immune checkpoint inhibitors (ICIs) in patients with advanced melanoma. Methods: Twenty patients with advanced melanoma receiving ICI prospectively underwent 18F-FDG PET/CT at 3 scan intervals: before treatment initiation (SCAN-1), at days 21-28 (SCAN-2), and at 4 mo (SCAN-3). This study was approved by the institutional review board, and informed consent was received from all patients who were enrolled between April 2012 and December 2013. Tumor response at each posttreatment time point was assessed according to RECIST 1.1, immune-related response criteria, PERCIST (PERCIST 1.0), and European Organization for Research and Treatment of Cancer (EORTC) criteria. Performance characteristics of each metric to predict best overall response (BOR) at ≥ 4 mo were assessed. Results: Twenty evaluable patients were treated with ipilimumab (n = 16), BMS-936559 (n = 3), or nivolumab (n = 1). BOR at ≥ 4 mo included complete response (n = 2), partial response (n = 2), stable disease (n = 1), and progressive disease (n = 15). Response evaluations at SCAN-2 using RECIST 1.1, immune-related response criteria, PERCIST, and EORTC criteria demonstrated accuracies of 75%, 70%, 70%, and 65%, respectively, to predict BOR at ≥ 4 mo. Interestingly, the optimal PERCIST and EORTC threshold values at SCAN-2 to predict BOR were >15.5% and >14.7%, respectively. By combining anatomic and functional imaging data collected at SCAN-2, we developed criteria to predict eventual response to ICI with 100% sensitivity, 93% specificity, and 95% accuracy. Conclusion: Combining functional and anatomic imaging parameters from 18F-FDG PET/CT scans performed early in ICI appears predictive for eventual response in patients with advanced melanoma. These findings require validation in larger cohorts.
Collapse
Affiliation(s)
- Steve Y Cho
- Johns Hopkins University School of Medicine and Sidney Kimmel Comprehensive Cancer Center, Baltimore, Maryland .,University of Wisconsin School of Medicine and Public Health and Carbone Comprehensive Cancer Center, Madison, Wisconsin
| | - Evan J Lipson
- Johns Hopkins University School of Medicine and Sidney Kimmel Comprehensive Cancer Center, Baltimore, Maryland
| | - Hyung-Jun Im
- University of Wisconsin School of Medicine and Public Health and Carbone Comprehensive Cancer Center, Madison, Wisconsin.,Department of Transdisciplinary Studies, Graduate School of Convergence Science and Technology, Seoul National University, Seoul, Korea; and
| | - Steven P Rowe
- Johns Hopkins University School of Medicine and Sidney Kimmel Comprehensive Cancer Center, Baltimore, Maryland
| | - Esther Mena Gonzalez
- Johns Hopkins University School of Medicine and Sidney Kimmel Comprehensive Cancer Center, Baltimore, Maryland
| | - Amanda Blackford
- Johns Hopkins University School of Medicine and Sidney Kimmel Comprehensive Cancer Center, Baltimore, Maryland
| | - Alin Chirindel
- Johns Hopkins University School of Medicine and Sidney Kimmel Comprehensive Cancer Center, Baltimore, Maryland
| | - Drew M Pardoll
- Johns Hopkins University School of Medicine and Sidney Kimmel Comprehensive Cancer Center, Baltimore, Maryland
| | - Suzanne L Topalian
- Johns Hopkins University School of Medicine and Sidney Kimmel Comprehensive Cancer Center, Baltimore, Maryland
| | - Richard L Wahl
- Johns Hopkins University School of Medicine and Sidney Kimmel Comprehensive Cancer Center, Baltimore, Maryland.,Mallinckrodt Institute of Radiology, Washington University School of Medicine, St. Louis, Missouri
| |
Collapse
|
8
|
Iranmahboob AK, Kierans AS, Huang C, Ream JM, Rosenkrantz AB. Preliminary investigation of whole-pancreas 3D histogram ADC metrics for predicting progression of acute pancreatitis. Clin Imaging 2017; 42:172-177. [DOI: 10.1016/j.clinimag.2016.12.007] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2016] [Revised: 11/23/2016] [Accepted: 12/14/2016] [Indexed: 12/24/2022]
|