1
|
Al-Mousa DS, Rawashdeh M, Alakhras M, Spuur KM, AbuTaimai R, Brennan PC. Does mammographic density remain a radiological challenge in the digital era? Acta Radiol 2021; 62:707-714. [PMID: 32623914 DOI: 10.1177/0284185120938367] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
BACKGROUND The low subject contrast between cancerous and fibroglandular tissue could obscure breast abnormalities. PURPOSE To investigate radiologists' performance for detection of breast cancer in low and high mammographic density (MD) when cases are digitally acquired. MATERIAL AND METHODS A test set of 60 digital mammography cases, of which 20 were cancerous, were examined by 17 radiologists. Mammograms were categorized as low (≤50%) or high (>50%) MD and rated for suspicion of malignancy using the Royal Australian and New Zealand College of Radiology (RANZCR) classification system. Radiologist demographics including cases read per year, age, subspecialty, and years of reporting were recorded. Radiologist performance was analyzed by the following metrics: sensitivity; specificity; area under the receiver operating characteristic (ROC) curve (AUC), location sensitivity, and jackknife free-response ROC (JAFROC) figure of merit (FOM). RESULTS Comparing high to low MD cases, radiologists showed a significantly higher sensitivity (P = 0.015), AUC (P = 0.003), location sensitivity (P = 0.002), and JAFROC FOM (P = 0.001). In high compared to low MD cases, radiologists with <1000 annual reads and radiologists with no mammographic subspecialty had significantly higher AUC, location sensitivity, and JAFROC FOM. Radiologists with ≥1000 annual reads and radiologists with mammography subspecialty demonstrated a significant increase in location sensitivity in high compared to low MD cases. CONCLUSION In this experimental situation, radiologists' performance was higher when reading cases with high compared to low MD. Experienced radiologists were able to precisely localize lesions in breasts with higher MD. Further studies in unselected screening materials are needed to verify the results.
Collapse
Affiliation(s)
- Dana S Al-Mousa
- Department of Allied Medical Sciences, Faculty of Applied Medical Sciences, Jordan University of Science and Technology, Irbid, Jordan
| | - Mohammad Rawashdeh
- Department of Allied Medical Sciences, Faculty of Applied Medical Sciences, Jordan University of Science and Technology, Irbid, Jordan
| | - Maram Alakhras
- Department of Allied Medical Sciences, Faculty of Applied Medical Sciences, Jordan University of Science and Technology, Irbid, Jordan
| | - Kelly M Spuur
- School of Dentistry and Health Sciences, Charles Sturt University, Wagga Wagga, NSW, Australia
| | | | - Patrick C Brennan
- Medical Image Optimisation and Perception Group (MIOPeG), Discipline of Medical Imaging and Radiation Sciences, Faculty of Health Sciences, The University of Sydney, Sydney, NSW, Australia
| |
Collapse
|
2
|
The pop-up research centre - Challenges and opportunities. Radiography (Lond) 2019; 25 Suppl 1:S19-S24. [PMID: 31481183 DOI: 10.1016/j.radi.2019.05.009] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/14/2019] [Revised: 05/25/2019] [Accepted: 05/30/2019] [Indexed: 11/24/2022]
Abstract
OBJECTIVES This article sets out to describe the concept of the "pop-up" research centre as a means to promote and develop radiography research locally, nationally and internationally, and to empower professional colleagues to set up similar initiatives in the future. KEY FINDINGS A detailed overview of the development and management of "pop-up" research is provided based on the experiences of the authors, including specific examples. Matters such as study design, approvals, equipment and software, environment, participant recruitment and management, research teams and activity costs are discussed. Quantifiable benefits of "pop-up" research such as resultant peer reviewed publications, development of researchers' skills and potential collaborations are described. A number of "soft skill" benefits are also apparent and include enhanced organisational profiles, team building and the development of leadership skills. CONCLUSIONS "Pop-up" research centres are a valuable option for conducting research and offer the radiography profession an achievable mechanism to increase and enhance research activity. However, careful planning and execution are essential.
Collapse
|
3
|
Woznitza N, Piper K, Burke S, Bothamley G. Chest X-ray Interpretation by Radiographers Is Not Inferior to Radiologists: A Multireader, Multicase Comparison Using JAFROC (Jack-knife Alternative Free-response Receiver Operating Characteristics) Analysis. Acad Radiol 2018; 25:1556-1563. [PMID: 29724674 DOI: 10.1016/j.acra.2018.03.026] [Citation(s) in RCA: 30] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2017] [Revised: 03/08/2018] [Accepted: 03/29/2018] [Indexed: 12/25/2022]
Abstract
RATIONALE AND OBJECTIVES Chest X-rays (CXR) are one of the most frequently requested imaging examinations and are fundamental to many patient pathways. The aim of this study was to investigate the diagnostic accuracy of CXR interpretation by reporting radiographers (technologists). METHODS A cohort of consultant radiologists (n = 10) and reporting radiographers (technologists; n = 11) interpreted a bank (n = 106) of adult CXRs that contained a range of pathologies. Jack-knife alternate free-response receiver operating characteristic (JAFROC) methodology was used to determine the performance of the observers (JAFROC v4.2). A noninferiority approach was used, with a predefined margin of clinical insignificance of 10% of average consultant radiologist diagnostic accuracy. RESULTS The diagnostic accuracy of the reporting radiographers (figure of merit = 0.828, 95% confidence interval 0.808-0.847) was noninferior to the consultant radiologists (figure of merit = 0.788, 95% confidence interval 0.766-0.811), P < .0001. CONCLUSIONS With appropriate postgraduate education, reporting radiographers are able to interpret CXRs at a level comparable to consultant radiologists.
Collapse
Affiliation(s)
- Nick Woznitza
- Radiology Department, Homerton University Hospital, Homerton Row, London E9 6SR, United Kingdom; School of Allied Health Professions, Canterbury Christ Church University, North Holmes Road, Canterbury, Kent CT1 1QU, United Kingdom.
| | - Keith Piper
- School of Allied Health Professions, Canterbury Christ Church University, North Holmes Road, Canterbury, Kent CT1 1QU, United Kingdom
| | - Stephen Burke
- Radiology Department, Homerton University Hospital, Homerton Row, London E9 6SR, United Kingdom
| | - Graham Bothamley
- Department of Respiratory Medicine, Homerton University Hospital, London, United Kingdom
| |
Collapse
|
4
|
Carrigan AJ, Wardle SG, Rich AN. Finding cancer in mammograms: if you know it's there, do you know where? Cogn Res Princ Implic 2018; 3:10. [PMID: 29707615 PMCID: PMC5904219 DOI: 10.1186/s41235-018-0096-5] [Citation(s) in RCA: 23] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2017] [Accepted: 02/27/2018] [Indexed: 11/23/2022] Open
Abstract
Humans can extract considerable information from scenes, even when these are presented extremely quickly. The ability of an experienced radiologist to rapidly detect an abnormality on a mammogram may build upon this general capacity. Although radiologists have been shown to be able to detect an abnormality 'above chance' at short durations, the extent to which abnormalities can be localised at brief presentations is less clear. Extending previous work, we presented radiologists with unilateral mammograms, 50% containing a mass, for 250 or 1000 ms. As the female breast varies with respect to the level of normal fibroglandular tissue, the images were categorised into high and low density (50% of each), resulting in difficult and easy searches, respectively. Participants were asked to decide whether there was an abnormality (detection) and then to locate the mass on a blank outline of the mammogram (localisation). We found both detection and localisation information for all conditions. Although there may be a dissociation between detection and localisation on a small proportion of trials, we find a number of factors that lead to the underestimation of localisation including stimulus variability, response imprecision and participant guesses. We emphasise the importance of taking these factors into account when interpreting results. The effect of density on detection and localisation highlights the importance of considering breast density in medical screening.
Collapse
Affiliation(s)
- Ann J. Carrigan
- Perception in Action Research Centre & Department of Cognitive Science, Macquarie University, Sydney, Australia
- ARC Centre of Excellence in Cognition & Its Disorders, Macquarie University, Sydney, Australia
- Centre for Elite Performance, Expertise, and Training, Macquarie University, Sydney, Australia
| | - Susan G. Wardle
- Perception in Action Research Centre & Department of Cognitive Science, Macquarie University, Sydney, Australia
- ARC Centre of Excellence in Cognition & Its Disorders, Macquarie University, Sydney, Australia
| | - Anina N. Rich
- Perception in Action Research Centre & Department of Cognitive Science, Macquarie University, Sydney, Australia
- ARC Centre of Excellence in Cognition & Its Disorders, Macquarie University, Sydney, Australia
- Centre for Elite Performance, Expertise, and Training, Macquarie University, Sydney, Australia
| |
Collapse
|
5
|
Abbey CK, Samuelson FW, Zeng R, Boone JM, Eckstein MP, Myers K. Classification images for localization performance in ramp-spectrum noise. Med Phys 2018. [PMID: 29532479 DOI: 10.1002/mp.12857] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022] Open
Abstract
PURPOSE This study investigates forced localization of targets in simulated images with statistical properties similar to trans-axial sections of x-ray computed tomography (CT) volumes. A total of 24 imaging conditions are considered, comprising two target sizes, three levels of background variability, and four levels of frequency apodization. The goal of the study is to better understand how human observers perform forced-localization tasks in images with CT-like statistical properties. METHODS The transfer properties of CT systems are modeled by a shift-invariant transfer function in addition to apodization filters that modulate high spatial frequencies. The images contain noise that is the combination of a ramp-spectrum component, simulating the effect of acquisition noise in CT, and a power-law component, simulating the effect of normal anatomy in the background, which are modulated by the apodization filter as well. Observer performance is characterized using two psychophysical techniques: efficiency analysis and classification image analysis. Observer efficiency quantifies how much diagnostic information is being used by observers to perform a task, and classification images show how that information is being accessed in the form of a perceptual filter. RESULTS Psychophysical studies from five subjects form the basis of the results. Observer efficiency ranges from 29% to 77% across the different conditions. The lowest efficiency is observed in conditions with uniform backgrounds, where significant effects of apodization are found. The classification images, estimated using smoothing windows, suggest that human observers use center-surround filters to perform the task, and these are subjected to a number of subsequent analyses. When implemented as a scanning linear filter, the classification images appear to capture most of the observer variability in efficiency (r2 = 0.86). The frequency spectra of the classification images show that frequency weights generally appear bandpass in nature, with peak frequency and bandwidth that vary with statistical properties of the images. CONCLUSIONS In these experiments, the classification images appear to capture important features of human-observer performance. Frequency apodization only appears to have a significant effect on performance in the absence of anatomical variability, where the observers appear to underweight low spatial frequencies that have relatively little noise. Frequency weights derived from the classification images generally have a bandpass structure, with adaptation to different conditions seen in the peak frequency and bandwidth. The classification image spectra show relatively modest changes in response to different levels of apodization, with some evidence that observers are attempting to rebalance the apodized spectrum presented to them.
Collapse
Affiliation(s)
- Craig K Abbey
- Department of Psychological and Brain Sciences, University of California Santa Barbara, Santa Barbara, CA, 93106, USA
| | - Frank W Samuelson
- Division of Imaging Diagnostics and Software Reliability, United States Food and Drug Administration, White Oaks, MD, 20993, USA
| | - Rongping Zeng
- Division of Imaging Diagnostics and Software Reliability, United States Food and Drug Administration, White Oaks, MD, 20993, USA
| | - John M Boone
- Departments of Radiology and Biomedical Engineering, University of California Davis, Sacramento, CA, 95817, USA
| | - Miguel P Eckstein
- Department of Psychological and Brain Sciences, University of California Santa Barbara, Santa Barbara, CA, 93106, USA
| | - Kyle Myers
- Division of Imaging Diagnostics and Software Reliability, United States Food and Drug Administration, White Oaks, MD, 20993, USA
| |
Collapse
|
6
|
Salvagnini E, Bosmans H, Van Ongeval C, Van Steen A, Michielsen K, Cockmartin L, Struelens L, Marshall NW. Impact of compressed breast thickness and dose on lesion detectability in digital mammography: FROC study with simulated lesions in real mammograms. Med Phys 2017; 43:5104. [PMID: 27587041 DOI: 10.1118/1.4960630] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/20/2023] Open
Abstract
PURPOSE The aim of this work was twofold: (1) to examine whether, with standard automatic exposure control (AEC) settings that maintain pixel values in the detector constant, lesion detectability in clinical images decreases as a function of breast thickness and (2) to verify whether a new AEC setup can increase lesion detectability at larger breast thicknesses. METHODS Screening patient images, acquired on two identical digital mammography systems, were collected over a period of 2 yr. Mammograms were acquired under standard AEC conditions (part 1) and subsequently with a new AEC setup (part 2), programmed to use the standard AEC settings for compressed breast thicknesses ≤49 mm, while a relative dose increase was applied above this thickness. The images were divided into four thickness groups: T1 ≤ 29 mm, T2 = 30-49 mm, T3 = 50-69 mm, and T4 ≥ 70 mm, with each thickness group containing 130 randomly selected craniocaudal lesion-free images. Two measures of density were obtained for every image: a BI-RADS score and a map of volumetric breast density created with a software application (VolparaDensity, Matakina, NZ). This information was used to select subsets of four images, containing one image from each thickness group, matched to a (global) BI-RADS score and containing a region with the same (local) volpara volumetric density value. One selected lesion (a microcalcification cluster or a mass) was simulated into each of the four images. This process was repeated so that, for a given thickness group, half the images contained a single lesion and half were lesion-free. The lesion templates created and inserted in groups T3 and T4 for the first part of the study were then inserted into the images of thickness groups T3 and T4 acquired with higher dose settings. Finally, all images were visualized using the ViewDEX software and scored by four radiologists performing a free search study. A statistical jackknife-alternative free-response receiver operating characteristic analysis was applied. RESULTS For part 1, the alternative free-response receiver operating characteristic curves for the four readers were 0.80, 0.65, 0.55 and 0.56 in going from T1 to T4, indicating a decrease in detectability with increasing breast thickness. P-values and the 95% confidence interval showed no significant difference for the T3-T4 comparison (p = 0.78) while all the other differences were significant (p < 0.05). Separate analysis of microcalcification clusters presented the same results while for mass detection, the only significant difference came when comparing T1 to the other thickness groups. Comparing the scores of part 1 and part 2, results for the T3 group acquired with the new AEC setup and T3 group at standard AEC doses were significantly different (p = 0.0004), indicating improved detection. For this group a subanalysis for microcalcification detection gave the same results while no significant difference was found for mass detection. CONCLUSIONS These data using clinical images confirm results found in simple QA tests for many mammography systems that detectability falls as breast thickness increases. Results obtained with the AEC setup for constant detectability above 49 mm showed an increase in lesion detection with compressed breast thickness, bringing detectability of lesions to the same level.
Collapse
Affiliation(s)
- Elena Salvagnini
- Department of Imaging and Pathology, Radiology, KUL, Herestraat 49, Leuven B-3000, Belgium and SCK•CEN, Boeretang 200, Mol 2400, Belgium
| | - Hilde Bosmans
- Department of Imaging and Pathology, Radiology, KUL, Herestraat 49, Leuven B-3000, Belgium and Department of Radiology, Radiology, UZ Gasthuisberg, Herestraat 49, Leuven B-3000, Belgium
| | - Chantal Van Ongeval
- Department of Radiology, Radiology, UZ Gasthuisberg, Herestraat 49, Leuven B-3000, Belgium
| | - Andreas Van Steen
- Department of Radiology, Radiology, UZ Gasthuisberg, Herestraat 49, Leuven B-3000, Belgium
| | - Koen Michielsen
- Department of Imaging and Pathology, Nuclear Medicine and Molecular Imaging, KUL, Herestraat 49, Leuven B-3000, Belgium
| | - Lesley Cockmartin
- Department of Radiology, Radiology, UZ Gasthuisberg, Herestraat 49, Leuven B-3000, Belgium
| | | | - Nicholas W Marshall
- Department of Imaging and Pathology, Radiology, KUL, Herestraat 49, Leuven B-3000, Belgium and Department of Radiology, Radiology, UZ Gasthuisberg, Herestraat 49, Leuven B-3000, Belgium
| |
Collapse
|
7
|
A method to determine the impact of reduced visual function on nodule detection performance. Radiography (Lond) 2017; 23:19-24. [DOI: 10.1016/j.radi.2016.07.002] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2016] [Revised: 07/12/2016] [Accepted: 07/13/2016] [Indexed: 11/20/2022]
|
8
|
Littlefair S, Brennan P, Reed W, Mello-Thoms C. Does Expectation of Abnormality Affect the Search Pattern of Radiologists When Looking for Pulmonary Nodules? J Digit Imaging 2016; 30:55-62. [PMID: 27659798 DOI: 10.1007/s10278-016-9908-7] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022] Open
Abstract
This experiment investigated whether there might be an effect on the visual search strategy of radiologists during image interpretation of the same adult chest radiographs when given different clinical information. Each of 17 experienced radiologists was asked to interpret a set of 57 (10 abnormal) posteroanterior chest images to identify the presence of pulmonary lesions using differing clinical information (leading to unknown, low and high expectations of prevalence). Eye position metrics (search time, dwell time and time to first fixation) were compared for normal and abnormal images, as well as between conditions. For all images, there was a significantly longer search time at high prevalence expectation compared to low prevalence expectation (W = 75.19, P = <0.0001). Mann-Whitney analysis of the abnormal images demonstrated that the dwell time on correctly identified lesions was significantly shorter at low prevalence expectation compared to both unknown (U = 364.5, P = 0.02) and high prevalence expectation (U = 397.0, P = 0.0002). Visual search patterns of radiologists appear to be affected by changing a priori information where such information fosters an expectation of abnormality.
Collapse
Affiliation(s)
- Stephen Littlefair
- Medical Image Optimisation and Perception Group (MIOPeG), Discipline of Medical Radiation Sciences, Faculty of Health Sciences, University of Sydney, Room M213, 75, East Street, Sydney, NSW, 2141, Australia.
| | - Patrick Brennan
- Medical Image Optimisation and Perception Group (MIOPeG), Discipline of Medical Radiation Sciences, Faculty of Health Sciences, University of Sydney, Room M213, 75, East Street, Sydney, NSW, 2141, Australia
| | - Warren Reed
- Medical Image Optimisation and Perception Group (MIOPeG), Discipline of Medical Radiation Sciences, Faculty of Health Sciences, University of Sydney, Room M213, 75, East Street, Sydney, NSW, 2141, Australia
| | - Claudia Mello-Thoms
- Medical Image Optimisation and Perception Group (MIOPeG), Discipline of Medical Radiation Sciences, Faculty of Health Sciences, University of Sydney, Room M213, 75, East Street, Sydney, NSW, 2141, Australia
| |
Collapse
|
9
|
Thompson JD, Thomas NB, Manning DJ, Hogg P. The impact of greyscale inversion for nodule detection in an anthropomorphic chest phantom: a free-response observer study. Br J Radiol 2016; 89:20160249. [PMID: 27266374 PMCID: PMC5124894 DOI: 10.1259/bjr.20160249] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/03/2022] Open
Abstract
Objective: The aim of this work was to assess the impact of greyscale inversion on nodule detection on posteroanterior chest X-ray images. Previous work has attempted this, with no consensus opinion formed. We assessed the value of “fast-flicking” between standard and inverted display modes for nodule detection. Methods: Six consultant radiologists (with 5–32 years' reporting experience) completed an observer task under the free-response paradigm. An anthropomorphic chest phantom was loaded with 50 different configurations of simulated nodules (1–4 nodules per case) measuring 5, 8, 10 and 12 mm in spherical diameter; each configuration represented a single case. In addition, 25 cases contained no nodules. Images were displayed in three modes: (i) standard, (ii) inverted and (iii) fast-flicking between standard and inverted display modes. Each observer completed the study in a different order of display (i, ii, iii) using a calibrated 5-megapixel monitor. Nodules were localized with mouse clicks and ratings assigned using a 1–10 discrete slider-bar confidence scale. Rjafroc (Pittsburgh, PA) was used for data analysis; differences in nodule detection performance were considered significant at 0.05. Results: The observer-averaged weighted jackknife alternative free-response receiver-operating characteristic figures of merit were 0.715 (standard), 0.684 (inverted) and 0.717 (fast-flicking). Random-reader fixed-case analysis revealed no statistically significant difference between any treatment pair [F(2,8) = 1.22; p = 0.345]. Conclusion: No statistically significant difference in nodule detection was found for the three display conditions. Advances in knowledge: We have investigated the impact of fast-flicking between standard and inverted display modes for the detection of nodules. We found no benefit.
Collapse
Affiliation(s)
- John D Thompson
- Directorate of Radiology, University of Salford, Manchester, UK.,Radiology, University Hospitals of Morecambe Bay NHS Foundation Trust, Barrow-in-Furness, UK
| | - Nigel B Thomas
- Directorate of Radiology, University of Salford, Manchester, UK
| | - David J Manning
- Directorate of Radiology, University of Salford, Manchester, UK.,Lancaster Medical School, Lancaster University, Lancaster, UK
| | - Peter Hogg
- Directorate of Radiology, University of Salford, Manchester, UK.,Karolinska Institute, Stockholm, Sweden
| |
Collapse
|
10
|
Littlefair S, Mello-Thoms C, Reed W, Pietryzk M, Lewis S, McEntee M, Brennan P. Increasing Prevalence Expectation in Thoracic Radiology Leads to Overcall. Acad Radiol 2016; 23:284-9. [PMID: 26774736 DOI: 10.1016/j.acra.2015.11.007] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/26/2015] [Revised: 11/01/2015] [Accepted: 11/03/2015] [Indexed: 11/18/2022]
Abstract
RATIONALE AND OBJECTIVES The aim of this study was to measure the effect of prevalence expectation as determined by clinical history on the diagnostic performance of radiologists during pulmonary nodule detection on adult chest radiographs. MATERIALS AND METHODS A multi-observer, counter-balanced study (having half the readers in each group read a different condition initially) was performed to assess the effect of abnormality expectation on experienced radiologists' performance. A total of 33 board-certified radiologists were divided into three groups and searched for evidence of malignancy on a single set of 47 postero-anterior (PA) chest radiographs, 10 of which contained a single pulmonary nodule. The radiologists were unaware of disease prevalence. Before each viewing of the same dataset, the radiologists were allocated to two of three conditions based on the differing clinical information (previous cancer, no history, visa applicant). Location sensitivity, specificity, and jack-knife free-response receiver operator characteristics figure of merit were used to compare radiologist performance between conditions. RESULTS A significant reduction in specificity was shown for the cancer compared to that for the visa condition (W = -41 P = 0.02). No other significant findings were demonstrated for this or the other condition comparisons. No significant difference in the performance of radiologists was noted when viewing images under the same conditions. CONCLUSIONS This study suggested that there is a reduction in specificity with high compared to low prevalence expectation following specific radiological contexts. A reduction in specificity can have important clinical consequences leading to unnecessary interventions. The results and their implications emphasize the caution that should be placed on providing accurate referral criteria.
Collapse
Affiliation(s)
- Stephen Littlefair
- Medical Image Optimisation and Perception Group (MIOPeG), Discipline of Medical Radiation Sciences, Faculty of Health Sciences, University of Sydney, Room M213, Cumberland Campus, East Street, Lidcombe, NSW 2141, Australia.
| | - Claudia Mello-Thoms
- Medical Image Optimisation and Perception Group (MIOPeG), Discipline of Medical Radiation Sciences, Faculty of Health Sciences, University of Sydney, Room M213, Cumberland Campus, East Street, Lidcombe, NSW 2141, Australia; National Imaging Facilities, Brain and Mind Research Institute (BMRI), Faculty of Health Sciences, University of Sydney, Sydney, New South Wales, Australia
| | - Warren Reed
- Medical Image Optimisation and Perception Group (MIOPeG), Discipline of Medical Radiation Sciences, Faculty of Health Sciences, University of Sydney, Room M213, Cumberland Campus, East Street, Lidcombe, NSW 2141, Australia; National Imaging Facilities, Brain and Mind Research Institute (BMRI), Faculty of Health Sciences, University of Sydney, Sydney, New South Wales, Australia
| | | | - Sarah Lewis
- Medical Image Optimisation and Perception Group (MIOPeG), Discipline of Medical Radiation Sciences, Faculty of Health Sciences, University of Sydney, Room M213, Cumberland Campus, East Street, Lidcombe, NSW 2141, Australia; National Imaging Facilities, Brain and Mind Research Institute (BMRI), Faculty of Health Sciences, University of Sydney, Sydney, New South Wales, Australia
| | - Mark McEntee
- Medical Image Optimisation and Perception Group (MIOPeG), Discipline of Medical Radiation Sciences, Faculty of Health Sciences, University of Sydney, Room M213, Cumberland Campus, East Street, Lidcombe, NSW 2141, Australia; National Imaging Facilities, Brain and Mind Research Institute (BMRI), Faculty of Health Sciences, University of Sydney, Sydney, New South Wales, Australia
| | - Patrick Brennan
- Medical Image Optimisation and Perception Group (MIOPeG), Discipline of Medical Radiation Sciences, Faculty of Health Sciences, University of Sydney, Room M213, Cumberland Campus, East Street, Lidcombe, NSW 2141, Australia; National Imaging Facilities, Brain and Mind Research Institute (BMRI), Faculty of Health Sciences, University of Sydney, Sydney, New South Wales, Australia
| |
Collapse
|
11
|
Suleiman WI, McEntee MF, Lewis SJ, Rawashdeh MA, Georgian-Smith D, Heard R, Tapia K, Brennan PC. In the digital era, architectural distortion remains a challenging radiological task. Clin Radiol 2015; 71:e35-40. [PMID: 26602930 DOI: 10.1016/j.crad.2015.10.009] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/27/2015] [Revised: 09/30/2015] [Accepted: 10/12/2015] [Indexed: 11/28/2022]
Abstract
AIM To compare readers' performance in detecting architectural distortion (AD) compared with other breast cancer types using digital mammography. MATERIALS AND METHODS Forty-one experienced breast screen readers (20 US and 21 Australian) were asked to read a single test set of 30 digitally acquired mammographic cases. Twenty cases had abnormal findings (10 with AD, 10 non-AD) and 10 cases were normal. Each reader was asked to locate and rate any abnormalities. Lesion and case-based performance was assessed. For each collection of readers (US; Australian; combined), jackknife free-response receiver operating characteristic (JAFROC), figure of merit (FOM), and inferred receiver operating characteristic (ROC), area under curve (Az) were calculated using JAFROC v.4.1 software. Readers' sensitivity, location sensitivity, JAFROC, FOM, ROC, Az scores were compared between cases groups using Wilcoxon's signed ranked test statistics. RESULTS For lesion-based analysis, significantly lower location sensitivity (p=0.001) was shown on AD cases compared with non-AD cases for all reader collections. The case-based analysis demonstrated significantly lower ROC Az values (p=0.02) for the first collection of readers, and lower sensitivity for the second collection of readers (p=0.04) and all-readers collection (p=0.008), for AD compared with non-AD cases. CONCLUSIONS The current work demonstrates that AD remains a challenging task for readers, even in the digital era.
Collapse
Affiliation(s)
- W I Suleiman
- Medical Image Optimisation and Perception Group (MIOPeG), and the Brain and Mind Research Institute, The Faculty of Health Sciences, The University of Sydney, Australia.
| | - M F McEntee
- Medical Image Optimisation and Perception Group (MIOPeG), and the Brain and Mind Research Institute, The Faculty of Health Sciences, The University of Sydney, Australia
| | - S J Lewis
- Medical Image Optimisation and Perception Group (MIOPeG), and the Brain and Mind Research Institute, The Faculty of Health Sciences, The University of Sydney, Australia
| | - M A Rawashdeh
- Medical Image Optimisation and Perception Group (MIOPeG), and the Brain and Mind Research Institute, The Faculty of Health Sciences, The University of Sydney, Australia; Faculty of Applied Medical Sciences, Jordan University of Science and Technology, P.O.Box 3030, Irbid 22110, Jordan
| | - D Georgian-Smith
- Department of Radiology, Brigham and Women's Hospital, Harvard Medical School, 75 Francis St, RA 020, Boston, MA 02115, USA
| | - R Heard
- Medical Image Optimisation and Perception Group (MIOPeG), and the Brain and Mind Research Institute, The Faculty of Health Sciences, The University of Sydney, Australia
| | - K Tapia
- Medical Image Optimisation and Perception Group (MIOPeG), and the Brain and Mind Research Institute, The Faculty of Health Sciences, The University of Sydney, Australia
| | - P C Brennan
- Medical Image Optimisation and Perception Group (MIOPeG), and the Brain and Mind Research Institute, The Faculty of Health Sciences, The University of Sydney, Australia
| |
Collapse
|
12
|
Analysing data from observer studies in medical imaging research: An introductory guide to free-response techniques. Radiography (Lond) 2014. [DOI: 10.1016/j.radi.2014.04.005] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
|
13
|
Suleiman WI, Lewis SJ, Georgian-Smith D, Evanoff MG, McEntee MF. Number of mammography cases read per year is a strong predictor of sensitivity. J Med Imaging (Bellingham) 2014; 1:015503. [PMID: 26158030 DOI: 10.1117/1.jmi.1.1.015503] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2013] [Revised: 03/26/2014] [Accepted: 04/07/2014] [Indexed: 11/14/2022] Open
Abstract
Early detection of breast cancers affects the 5-year recurrence rates and treatment options for diagnosed patients, and consequently, many countries have instituted nationwide screening programs. This study compared the performance of expert radiologists from Australia and the United States in detection of breast cancer. Forty-one radiologists, 21 from Australia and 20 from the United States, reviewed 30 mammographic cases containing two-view mammograms. Twenty cases had abnormal findings and 10 cases had normal findings. Radiologists were asked to locate malignancies and assign a level of confidence. A jackknife free-response receiver operating characteristic, figure of merit (JAFROC, FOM), inferred receiver operating characteristic, area under curve (ROC, AUC), specificity, sensitivity, and location sensitivity were calculated using Ziltron software and JAFROC v4.1. A Mann-Whitney U test was used to compare the performance of Australian and U.S. radiologists. The results showed that when experience and the number of mammograms read per year were taken into account, the Australian radiologists sampled showed significantly higher sensitivity and location sensitivity ([Formula: see text]). JAFROC (FOM) and inferred ROC (AUC) analysis showed no difference between the overall performance of the two countries. ROC (AUC) and location sensitivity were higher for the Australian radiologists who read the most cases per year.
Collapse
Affiliation(s)
- Wasfi I Suleiman
- University of Sydney , Medical Imaging Optimisation and Perception Group, and the Brain and Mind Institute, 94 Mallet Street, Camperdown, NSW 2050, Australia
| | - Sarah J Lewis
- University of Sydney , Medical Imaging Optimisation and Perception Group, and the Brain and Mind Institute, 94 Mallet Street, Camperdown, NSW 2050, Australia
| | - Dianne Georgian-Smith
- Harvard Medical School , Brigham and Women's Hospital, Department of Radiology, 75 Francis Street, RA 020, Boston, Massachusetts 02115
| | - Michael G Evanoff
- The American Board of Radiology , 5441 East William Boulevard, Suite 200, Tucson, Arizona 85711
| | - Mark F McEntee
- University of Sydney , Medical Imaging Optimisation and Perception Group, and the Brain and Mind Institute, 94 Mallet Street, Camperdown, NSW 2050, Australia
| |
Collapse
|
14
|
Haygood TM, Qing Liu MA, Galvan EM, Bassett R, Devine C, Lano E, Viswanathan C, Marom EM. Memory for previously viewed radiographs and the effect of prior knowledge of memory task. Acad Radiol 2013; 20:1598-603. [PMID: 24200488 DOI: 10.1016/j.acra.2013.08.015] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2013] [Revised: 08/20/2013] [Accepted: 08/21/2013] [Indexed: 02/03/2023]
Abstract
RATIONALE AND OBJECTIVES To investigate the effect of being forewarned that they would be asked to identify repeated images on radiologists' recognition of previously interpreted versus new chest radiographs. MATERIALS AND METHODS Thirteen radiologists viewed 60 posterior-anterior chest radiographs, 31 with and 29 without nodules, in two sets of 40 images each. Eight radiologists were forewarned and five radiologists were not forewarned of the memory task. Twenty images in each of the two sets were unique to each set and 20 images occurred in both sets. The readers indicated the presence or absence of any nodules during both readings, and in the second reading session they also indicated whether they thought each image had also occurred in the first reading. RESULTS There was no significant difference in recognition memory performance between forewarned and not-forewarned readers. Overall accuracy in distinguishing previously-viewed from new images was 60.7%. CONCLUSIONS Being forewarned of the memory task did not improve recognition memory.
Collapse
Affiliation(s)
- Tamara Miner Haygood
- The University of Texas, MD Anderson Cancer Center, Division of Diagnostic Radiology, Musculoskeletal Imaging, 1515 Holcombe Blvd, Unit 1475, Houston, TX 77030.
| | | | | | | | | | | | | | | |
Collapse
|
15
|
A brief history of free-response receiver operating characteristic paradigm data analysis. Acad Radiol 2013; 20:915-9. [PMID: 23583665 DOI: 10.1016/j.acra.2013.03.001] [Citation(s) in RCA: 44] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2013] [Revised: 03/01/2013] [Accepted: 03/07/2013] [Indexed: 11/23/2022]
Abstract
In the receiver operating characteristic paradigm the observer assigns a single rating to each image and the location of the perceived abnormality, if any, is ignored. In the free-response receiver operating characteristic paradigm the observer is free to mark and rate as many suspicious regions as are considered clinically reportable. Credit for a correct localization is given only if a mark is sufficiently close to an actual lesion; otherwise, the observer's mark is scored as a location-level false positive. Until fairly recently there existed no accepted method for analyzing the resulting relatively unstructured data containing random numbers of mark-rating pairs per image. This report reviews the history of work in this field, which has now spanned more than five decades. It introduces terminology used to describe the paradigm, proposed measures of performance (figures of merit), ways of visualizing the data (operating characteristics), and software for analyzing free-response receiver operating characteristic studies.
Collapse
|
16
|
Krupinski EA. The Medical Image Perception Society XIV conference. Br J Radiol 2013; 86:20120337. [PMID: 23239698 PMCID: PMC3615396 DOI: 10.1259/bjr.20120337] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2012] [Accepted: 07/05/2012] [Indexed: 11/05/2022] Open
|