Characterizing search, recognition, and decision in the detection of lung nodules on CT scans: elucidation with eye tracking

Radiology AI Lab: Evaluation of Radiology Applications with Clinical End-Users

Despite the approval of over 200 artificial intelligence (AI) applications for radiology in the European Union, widespread adoption in clinical practice remains limited. Current assessments of AI applications often rely on post-hoc evaluations, lacking the granularity to capture real-time radiologist-AI interactions. The purpose of the study is to realise the Radiology AI lab for real-time, objective measurement of the impact of AI applications on radiologists' workflows. We proposed the user-state sensing framework (USSF) to structure the sensing of radiologist-AI interactions in terms of personal, interactional, and contextual states. Guided by the USSF, a lab was established using three non-invasive biometric measurement techniques: eye-tracking, heart rate monitoring, and facial expression analysis. We conducted a pilot test with four radiologists of varying experience levels, who read ultra-low-dose (ULD) CT cases in (1) standard PACS and (2) manually annotated (to mimic AI) PACS workflows. Interpretation time, eye-tracking metrics, heart rate variability (HRV), and facial expressions were recorded and analysed. The Radiology AI lab was successfully realised as an initial physical iteration of the USSF at a tertiary referral centre. Radiologists participating in the pilot test read 32 ULDCT cases (mean age, 52 years ± 23 (SD); 17 male; 16 cases with abnormalities). Cases were read on average in 4.1 ± 2.2 min (standard PACS) and 3.9 ± 1.9 min (AI-annotated PACS), with no significant difference (p = 0.48). Three out of four radiologists showed significant shifts (p < 0.02) in eye-tracking metrics, including saccade duration, saccade quantity, fixation duration, fixation quantity, and pupil diameter, when using the AI-annotated workflow. These changes align with prior findings linking such metrics to increased competency and reduced cognitive load, suggesting a more efficient visual search strategy in AI-assisted interpretation. Although HRV metrics did not correlate with experience, when combined with facial expression analysis, they helped identify key moments during the pilot test. The Radiology AI lab was successfully realised, implementing personal, interactional, and contextual states of the user-state sensing framework, enabling objective analysis of radiologists' workflows, and effectively capturing relevant biometrics. Future work will focus on expanding sensing of the contextual state of the user-state sensing framework, refining baseline determination, and continuing investigation of AI-enabled tools in radiology workflows.

Non-Conscious Detection of "Missed" Lung Nodules by Radiologists: Expanding the Boundaries of Successful Processing During the Visual Assessment of Chest CT Scans

"Just Accepted" papers have undergone full peer review and have been accepted for publication in Radiology: Artificial Intelligence. This article will undergo copyediting, layout, and proof review before it is published in its final version. Please note that during production of the final copyedited article, errors may be discovered which could affect the content. Background Diagnostic error rates for detecting small lung nodules on chest CT scans remain high at 50%, despite advances in imaging technology and radiologist training. These failure rates may stem from limitations in conscious recognition processes. However, successful visual processes may be detecting the nodule independent of the radiologist's report. Purpose To investigate visual processing in radiologists during the assessment of chest nodules to determine if radiologists have successful non-conscious processes that detect lung nodules on chest CT examinations even when not consciously recognized or considered, as evidenced by changes in how long they look (dwell time) and pupil size to missed nodules. Materials and Methods This prospective study, conducted from [8/14] to [09/23], compared 6 experienced radiologists with 6 medically naïve control participants. Participants viewed 18 chest CTs (9 abnormal with 16 nodules, 9 normal) to detect lung nodules. High-speed video eye-tracking measured gaze duration and pupil size (indicating physiological arousal) at missed nodule locations and same locations on normal CTs. The reference standard was the known presence or absence of nodules (as determined by a 4-radiologist consensus panel) in abnormal and normal CTs, respectively. Primary outcome measures were detection rates of nodules, dwell time and pupil size at nodule locations versus normal tissue. Paired t-tests were used for statistical analysis. Results Twelve participants (6 radiologists [9.3 average years of radiological experience]) 6 controls (with no radiological experience) were evaluated. Radiologists missed on average 59% of these lung nodules. For missed nodules, radiologists exhibited longer dwell times (Mean: 228 milliseconds vs 175 milliseconds, P=.005) and larger pupil area (Mean: 1446 pixels vs. 1349 pixels, P=.04.) than normal tissue. Control participants showed no differences in dwell time (Mean: 197 milliseconds vs 180 milliseconds, P= .64) or pupil size (Mean: 1426 pixels vs. 1714 pixels, P=.23) for missed nodules than normal tissue locations. Conclusion Radiologists non-conscious processes during visual assessment of a CT examination can detect lung nodules on chest CTs even when conscious recognition fails, as evidenced by increased dwell time and larger pupil size. This successful non-conscious detection is a result of general radiology training.

Peripheral liver metastases are more frequently missed than central metastases in contrast-enhanced CT: insights from a 25-reader performance study

PURPOSE

Subtle liver metastases may be missed in contrast enhanced CT imaging. We determined the impact of lesion location and conspicuity on metastasis detection using data from a prior reader study.

METHODS

In the prior reader study, 25 radiologists examined 40 CT exams each and circumscribed all suspected hepatic metastases. CT exams were chosen to include a total of 91 visually challenging metastases. The detectability of a metastasis was defined as the fraction of radiologists that circumscribed it. A conspicuity index was calculated for each metastasis by multiplying metastasis diameter with its contrast, defined as the difference between the average of a circular region within the metastasis and the average of the surrounding circular region of liver parenchyma. The effects of distance from liver edge and of conspicuity index on metastasis detectability were measured using multivariable linear regression.

RESULTS

The median metastasis was 1.4 cm from the edge (interquartile range [IQR], 0.9-2.1 cm). Its diameter was 1.2 cm (IQR, 0.9-1.8 cm), and its contrast was 38 HU (IQR, 23-68 HU). An increase of one standard deviation in conspicuity index was associated with a 6.9% increase in detectability (p = 0.008), whereas an increase of one standard deviation in distance from the liver edge was associated with a 5.5% increase in detectability (p = 0.03).

CONCLUSION

Peripheral liver metastases were missed more frequently than central liver metastases, with this effect depending on metastasis size and contrast.

A multiscale 3D network for lung nodule detection using flexible nodule modeling

BACKGROUND

Lung cancer is the most common type of cancer. Detection of lung cancer at an early stage can reduce mortality rates. Pulmonary nodules may represent early cancer and can be identified through computed tomography (CT) scans. Malignant risk can be estimated based on attributes like size, shape, location, and density.

PURPOSE

Deep learning algorithms have achieved remarkable advancements in this domain compared to traditional machine learning methods. Nevertheless, many existing anchor-based deep learning algorithms exhibit sensitivity to predefined anchor-box configurations, necessitating manual adjustments to obtain optimal outcomes. Conversely, current anchor-free deep learning-based nodule detection methods normally adopt fixed-size nodule models like cubes or spheres.

METHODS

To address these technical challenges, we propose a multiscale 3D anchor-free deep learning network (M3N) for pulmonary nodule detection, leveraging adjustable nodule modeling (ANM). Within this framework, ANM empowers the representation of target objects in an anisotropic manner, with a novel point selection strategy (PSS) devised to accelerate the learning process of anisotropic representation. We further incorporate a composite loss function that combines the conventional L2 loss and cosine similarity loss, facilitating M3N to learn nodules' intensity distribution in three dimensions.

RESULTS

Experiment results show that the M3N achieves 90.6% competitive performance metrics (CPM) with seven predefined false positives per scan on the LUNA 16 dataset. This performance appears to exceed that of other state-of-the-art deep learning-based networks reported in their respective publications. Individual test results also demonstrate that M3N excels in providing more accurate, adaptive bounding boxes surrounding the contours of target nodules.

CONCLUSIONS

The newly developed nodule detection system reduces reliance on prior knowledge, such as the general size of objects in the dataset, thus it should enhance overall robustness and versatility. Distinct from traditional nodule modeling techniques, the ANM approach aligns more closely with the morphological characteristics of nodules. Time consumption and detection results demonstrate promising efficiency and accuracy which should be validated in clinical settings.

Signs of Nothing: Negotiations Over Semiotic Indeterminacy in Danish Lung Cancer Diagnostics

In Denmark, injunctions of "early" cancer diagnosis increasingly imply surveillance of small tissue changes, which may or may not develop into cancer. Based on fieldwork at diagnostic lung cancer clinics and with people in CT surveillance for tissue changes, I explore how detected tissue changes are ascribed meaning as signs of "nothing" or "something." Inspired by Peircean semiotics, I suggest that the semiotic indeterminacy of tissue changes points to how diagnostic socialities both expand medical semiotics and enable this expansion. The article, thereby, contributes to understandings of signs as diagnostic infrastructures.

Targeted Training Reduces Search Errors but Not Classification Errors for Hepatic Metastasis Detection at Contrast-Enhanced CT

RATIONALE AND OBJECTIVES

Methods are needed to improve the detection of hepatic metastases. Errors occur in both lesion detection (search) and decisions of benign versus malignant (classification). Our purpose was to evaluate a training program to reduce search errors and classification errors in the detection of hepatic metastases in contrast-enhanced abdominal computed tomography (CT).

MATERIALS AND METHODS

After Institutional Review Board approval, we conducted a single-group prospective pretest-posttest study. Pretest and posttest were identical and consisted of interpreting 40 contrast-enhanced abdominal CT exams containing 91 liver metastases under eye tracking. Between pretest and posttest, readers completed search training with eye-tracker feedback and coaching to increase interpretation time, use liver windows, and use coronal reformations. They also completed classification training with part-task practice, rating lesions as benign or malignant. The primary outcome was metastases missed due to search errors (<2 seconds gaze under eye tracker) and classification errors (>2 seconds). Jackknife free-response receiver operator characteristic (JAFROC) analysis was also conducted.

RESULTS

A total of 31 radiologist readers (8 abdominal subspecialists, 8 nonabdominal subspecialists, 15 senior residents/fellows) participated. Search errors were reduced (pretest 11%, posttest 8%, difference 3% [95% confidence interval, 0.3%-5.1%], P = .01), but there was no difference in classification errors (difference 0%, P = .97) or in JAFROC figure of merit (difference -0.01, P = .36). In subgroup analysis, abdominal subspecialists demonstrated no evidence of change.

CONCLUSION

Targeted training reduced search errors but not classification errors for the detection of hepatic metastases at contrast-enhanced abdominal CT. Improvements were not seen in all subgroups.

Appropriate Reliance on Artificial Intelligence in Radiology Education

Users of artificial intelligence (AI) can become overreliant on AI, negatively affecting the performance of human-AI teams. For a future in which radiologists use interpretive AI tools routinely in clinical practice, radiology education will need to evolve to provide radiologists with the skills to use AI appropriately and wisely. In this work, we examine how overreliance on AI may develop in radiology trainees and explore how this problem can be mitigated, including through the use of AI-augmented education. Radiology trainees will still need to develop the perceptual skills and mastery of knowledge fundamental to radiology to use AI safely. We propose a framework for radiology trainees to use AI tools with appropriate reliance, drawing on lessons from human-AI interactions research.

A 2D Synthesized Image Improves the 3D Search for Foveated Visual Systems

Current medical imaging increasingly relies on 3D volumetric data making it difficult for radiologists to thoroughly search all regions of the volume. In some applications (e.g., Digital Breast Tomosynthesis), the volumetric data is typically paired with a synthesized 2D image (2D-S) generated from the corresponding 3D volume. We investigate how this image pairing affects the search for spatially large and small signals. Observers searched for these signals in 3D volumes, 2D-S images, and while viewing both. We hypothesize that lower spatial acuity in the observers' visual periphery hinders the search for the small signals in the 3D images. However, the inclusion of the 2D-S guides eye movements to suspicious locations, improving the observer's ability to find the signals in 3D. Behavioral results show that the 2D-S, used as an adjunct to the volumetric data, improves the localization and detection of the small (but not large) signal compared to 3D alone. There is a concomitant reduction in search errors as well. To understand this process at a computational level, we implement a Foveated Search Model (FSM) that executes human eye movements and then processes points in the image with varying spatial detail based on their eccentricity from fixations. The FSM predicts human performance for both signals and captures the reduction in search errors when the 2D-S supplements the 3D search. Our experimental and modeling results delineate the utility of 2D-S in 3D search-reduce the detrimental impact of low-resolution peripheral processing by guiding attention to regions of interest, effectively reducing errors.

Discrimination tasks in simulated low-dose CT noise

BACKGROUND

This study reports the results of a set of discrimination experiments using simulated images that represent the appearance of subtle lesions in low-dose computed tomography (CT) of the lungs. Noise in these images has a characteristic ramp-spectrum before apodization by noise control filters. We consider three specific diagnostic features that determine whether a lesion is considered malignant or benign, two system-resolution levels, and four apodization levels for a total of 24 experimental conditions.

PURPOSE

The goal of the investigation is to better understand how well human observers perform subtle discrimination tasks like these, and the mechanisms of that performance. We use a forced-choice psychophysical paradigm to estimate observer efficiency and classification images. These measures quantify how effectively subjects can read the images, and how they use images to perform discrimination tasks across the different imaging conditions.

MATERIALS AND METHODS

The simulated CT images used as stimuli in the psychophysical experiments are generated from high-resolution objects passed through a modulation transfer function (MTF) before down-sampling to the image-pixel grid. Acquisition noise is then added with a ramp noise-power spectrum (NPS), with subsequent smoothing through apodization filters. The features considered are lesion size, indistinct lesion boundary, and a nonuniform lesion interior. System resolution is implemented by an MTF with resolution (10% max.) of 0.47 or 0.58 cyc/mm. Apodization is implemented by a Shepp-Logan filter (Sinc profile) with various cutoffs. Six medically naïve subjects participated in the psychophysical studies, entailing training and testing components for each condition. Training consisted of staircase procedures to find the 80% correct threshold for each subject, and testing involved 2000 psychophysical trials at the threshold value for each subject. Human-observer performance is compared to the Ideal Observer to generate estimates of task efficiency. The significance of imaging factors is assessed using ANOVA. Classification images are used to estimate the linear template weights used by subjects to perform these tasks. Classification-image spectra are used to analyze subject weights in the spatial-frequency domain.

RESULTS

Overall, average observer efficiency is relatively low in these experiments (10%-40%) relative to detection and localization studies reported previously. We find significant effects for feature type and apodization level on observer efficiency. Somewhat surprisingly, system resolution is not a significant factor. Efficiency effects of the different features appear to be well explained by the profile of the linear templates in the classification images. Increasingly strong apodization is found to both increase the classification-image weights and to increase the mean-frequency of the classification-image spectra. A secondary analysis of "Unapodized" classification images shows that this is largely due to observers undoing (inverting) the effects of apodization filters.

CONCLUSIONS

These studies demonstrate that human observers can be relatively inefficient at feature-discrimination tasks in ramp-spectrum noise. Observers appear to be adapting to frequency suppression implemented in apodization filters, but there are residual effects that are not explained by spatial weighting patterns. The studies also suggest that the mechanisms for improving performance through the application of noise-control filters may require further investigation.

Changes in Radiologists' Gaze Patterns Against Lung X-rays with Different Abnormalities: a Randomized Experiment

The workload of some radiologists increased dramatically in the last several, which resulted in a potentially reduced quality of diagnosis. It was demonstrated that diagnostic accuracy of radiologists significantly reduces at the end of work shifts. The study aims to investigate how radiologists cover chest X-rays with their gaze in the presence of different chest abnormalities and high workload. We designed a randomized experiment to quantitatively assess how radiologists' image reading patterns change with the radiological workload. Four radiologists read chest X-rays on a radiological workstation equipped with an eye-tracker. The lung fields on the X-rays were automatically segmented with U-Net neural network allowing to measure the lung coverage with radiologists' gaze. The images were randomly split so that each image was shown at a different time to a different radiologist. Regression models were fit to the gaze data to calculate the treads in lung coverage for individual radiologists and chest abnormalities. For the study, a database of 400 chest X-rays with reference diagnoses was assembled. The average lung coverage with gaze ranged from 55 to 65% per radiologist. For every 100 X-rays read, the lung coverage reduced from 1.3 to 7.6% for the different radiologists. The coverage reduction trends were consistent for all abnormalities ranging from 3.4% per 100 X-rays for cardiomegaly to 4.1% per 100 X-rays for atelectasis. The more image radiologists read, the smaller part of the lung fields they cover with the gaze. This pattern is very stable for all abnormality types and is not affected by the exact order the abnormalities are viewed by radiologists. The proposed randomized experiment captured and quantified consistent changes in X-ray reading for different lung abnormalities that occur due to high workload.

Multiple expressions of "expert" abnormality gist in novices following perceptual learning

With a brief half-second presentation, a medical expert can determine at above chance levels whether a medical scan she sees is abnormal based on a first impression arising from an initial global image process, termed "gist." The nature of gist processing is debated but this debate stems from results in medical experts who have years of perceptual experience. The aim of the present study was to determine if gist processing for medical images occurs in naïve (non-medically trained) participants who received a brief perceptual training and to tease apart the nature of that gist signal. We trained 20 naïve participants on a brief perceptual-adaptive training of histology images. After training, naïve observers were able to obtain abnormality detection and abnormality categorization above chance, from a brief 500 ms masked presentation of a histology image, hence showing "gist." The global signal demonstrated in perceptually trained naïve participants demonstrated multiple dissociable components, with some of these components relating to how rapidly naïve participants learned a normal template during perceptual learning. We suggest that multiple gist signals are present when experts view medical images derived from the tens of thousands of images that they are exposed to throughout their training and careers. We also suggest that a directed learning of a normal template may produce better abnormality detection and identification in radiologists and pathologists.

Understanding Reader Variability: A 25-Radiologist Study on Liver Metastasis Detection at CT

Background Substantial interreader variability exists for common tasks in CT imaging, such as detection of hepatic metastases. This variability can undermine patient care by leading to misdiagnosis. Purpose To determine the impact of interreader variability associated with (a) reader experience, (b) image navigation patterns (eg, eye movements, workstation interactions), and (c) eye gaze time at missed liver metastases on contrast-enhanced abdominal CT images. Materials and Methods In a single-center prospective observational trial at an academic institution between December 2020 and February 2021, readers were recruited to examine 40 contrast-enhanced abdominal CT studies (eight normal, 32 containing 91 liver metastases). Readers circumscribed hepatic metastases and reported confidence. The workstation tracked image navigation and eye movements. Performance was quantified by using the area under the jackknife alternative free-response receiver operator characteristic (JAFROC-1) curve and per-metastasis sensitivity and was associated with reader experience and image navigation variables. Differences in area under JAFROC curve were assessed with the Kruskal-Wallis test followed by the Dunn test, and effects of image navigation were assessed by using the Wilcoxon signed-rank test. Results Twenty-five readers (median age, 38 years; IQR, 31-45 years; 19 men) were recruited and included nine subspecialized abdominal radiologists, five nonabdominal staff radiologists, and 11 senior residents or fellows. Reader experience explained differences in area under the JAFROC curve, with abdominal radiologists demonstrating greater area under the JAFROC curve (mean, 0.77; 95% CI: 0.75, 0.79) than trainees (mean, 0.71; 95% CI: 0.69, 0.73) (P = .02) or nonabdominal subspecialists (mean, 0.69; 95% CI: 0.60, 0.78) (P = .03). Sensitivity was similar within the reader experience groups (P = .96). Image navigation variables that were associated with higher sensitivity included longer interpretation time (P = .003) and greater use of coronal images (P < .001). The eye gaze time was at least 0.5 and 2.0 seconds for 71% (266 of 377) and 40% (149 of 377) of missed metastases, respectively. Conclusion Abdominal radiologists demonstrated better discrimination for the detection of liver metastases on abdominal contrast-enhanced CT images. Missed metastases frequently received at least a brief eye gaze. Higher sensitivity was associated with longer interpretation time and greater use of liver display windows and coronal images. © RSNA, 2022 Online supplemental material is available for this article.

Use of "Diagnostic Yield" in Imaging Research Reports: Results from Articles Published in Two General Radiology Journals

OBJECTIVE

"Diagnostic yield," also referred to as the detection rate, is a parameter positioned between diagnostic accuracy and diagnosis-related patient outcomes in research studies that assess diagnostic tests. Unfamiliarity with the term may lead to incorrect usage and delivery of information. Herein, we evaluate the level of proper use of the term "diagnostic yield" and its related parameters in articles published in Radiology and Korean Journal of Radiology (KJR).

MATERIALS AND METHODS

Potentially relevant articles published since 2012 in these journals were identified using MEDLINE and PubMed Central databases. The initial search yielded 239 articles. We evaluated whether the correct definition and study setting of "diagnostic yield" or "detection rate" were used and whether the articles also reported companion parameters for false-positive results. We calculated the proportion of articles that correctly used these parameters and evaluated whether the proportion increased with time (2012-2016 vs. 2017-2022).

RESULTS

Among 39 eligible articles (19 from Radiology and 20 from KJR), 17 (43.6%; 11 from Radiology and 6 from KJR) correctly defined "diagnostic yield" or "detection rate." The remaining 22 articles used "diagnostic yield" or "detection rate" with incorrect meanings such as "diagnostic performance" or "sensitivity." The proportion of correctly used diagnostic terms was higher in the studies published in Radiology than in those published in KJR (57.9% vs. 30.0%). The proportion improved with time in Radiology (33.3% vs. 80.0%), whereas no improvement was observed in KJR over time (33.3% vs. 27.3%). The proportion of studies reporting companion parameters was similar between journals (72.7% vs. 66.7%), and no considerable improvement was observed over time.

CONCLUSION

Overall, a minority of articles accurately used "diagnostic yield" or "detection rate." Incorrect usage of the terms was more frequent without improvement over time in KJR than in Radiology. Therefore, improvements are required in the use and reporting of these parameters.

An interactive eye-tracking system for measuring radiologists' visual fixations in volumetric CT images: Implementation and initial eye-tracking accuracy validation

PURPOSE

Eye-tracking approaches have been used to understand the visual search process in radiology. However, previous eye-tracking work in computer tomography (CT) has been limited largely to single cross-sectional images or video playback of the reconstructed volume, which do not accurately reflect radiologists' visual search activities and their interactivity with three-dimensional image data at a computer workstation (e.g., scroll, pan, and zoom) for visual evaluation of diagnostic imaging targets. We have developed a platform that integrates eye-tracking hardware with in-house-developed reader workstation software to allow monitoring of the visual search process and reader-image interactions in clinically relevant reader tasks. The purpose of this work is to validate the spatial accuracy of eye-tracking data using this platform for different eye-tracking data acquisition modes.

METHODS

An eye-tracker was integrated with a previously developed workstation designed for reader performance studies. The integrated system captured real-time eye movement and workstation events at 1000 Hz sampling frequency. The eye-tracker was operated either in head-stabilized mode or in free-movement mode. In head-stabilized mode, the reader positioned their head on a manufacturer-provided chinrest. In free-movement mode, a biofeedback tool emitted an audio cue when the head position was outside the data collection range (general biofeedback) or outside a narrower range of positions near the calibration position (strict biofeedback). Four radiologists and one resident were invited to participate in three studies to determine eye-tracking spatial accuracy under three constraint conditions: head-stabilized mode (i.e., with use of a chin rest), free movement with general biofeedback, and free movement with strict biofeedback. Study 1 evaluated the impact of head stabilization versus general or strict biofeedback using a cross-hair target prior to the integration of the eye-tracker with the image viewing workstation. In Study 2, after integration of the eye-tracker and reader workstation, readers were asked to fixate on targets that were randomly distributed within a volumetric digital phantom. In Study 3, readers used the integrated system to scroll through volumetric patient CT angiographic images while fixating on the centerline of designated blood vessels (from the left coronary artery to dorsalis pedis artery). Spatial accuracy was quantified as the offset between the center of the intended target and the detected fixation using units of image pixels and the degree of visual angle.

RESULTS

The three head position constraint conditions yielded comparable accuracy in the studies using digital phantoms. For Study 1 involving the digital crosshairs, the median ± the standard deviation of offset values among readers were 15.2 ± 7.0 image pixels with the chinrest, 14.2 ± 3.6 image pixels with strict biofeedback, and 19.1 ± 6.5 image pixels with general biofeedback. For Study 2 using the random dot phantom, the median ± standard deviation offset values were 16.7 ± 28.8 pixels with use of a chinrest, 16.5 ± 24.6 pixels using strict biofeedback, and 18.0 ± 22.4 pixels using general biofeedback, which translated to a visual angle of about 0.8° for all three conditions. We found no obvious association between eye-tracking accuracy and target size or view time. In Study 3 viewing patient images, use of the chinrest and strict biofeedback demonstrated comparable accuracy, while the use of general biofeedback demonstrated a slightly worse accuracy. The median ± standard deviation of offset values were 14.8 ± 11.4 pixels with use of a chinrest, 21.0 ± 16.2 pixels using strict biofeedback, and 29.7 ± 20.9 image pixels using general biofeedback. These corresponded to visual angles ranging from 0.7° to 1.3°.

CONCLUSIONS

An integrated eye-tracker system to assess reader eye movement and interactive viewing in relation to imaging targets demonstrated reasonable spatial accuracy for assessment of visual fixation. The head-free movement condition with audio biofeedback performed similarly to head-stabilized mode.

The Influence of Experience on Gazing Patterns during Endovascular Treatment: Eye-Tracking Study

Objective

In various fields, differences in eye-gazing patterns during tasks between experts and novices have been evaluated. The aim of this study was to investigate gazing patterns during neuro-endovascular treatment using an eye-tracking device and assess whether gazing patterns depend on the physician's experience or skill.

Methods

Seven physicians performed coil embolization for a cerebral aneurysm in a silicone vessel model under biplane X-ray fluoroscopy, and their gazing patterns were recorded using an eye-tracking device. The subjects were divided into three groups according to experience, highly experienced (Expert) group, intermediately experienced (Trainee) group, and less experienced (Novice) group. The duration of fixation on the anterior-posterior (AP) view screen, lateral (LR) view, and out-of-screen were compared between each group.

Results

During microcatheter navigation, the Expert and Trainee groups spent a long time on fixation to AP, while the Novice group split their attention between each location. In coil insertion, the Expert group gazed at both the AP and the LR views with more saccades between screens. In contrast, the Trainee group spent most of their time only on the AP view screen and the Novice group spent longer out-of-screen.

Conclusion

An eye-tracking device can detect different gazing patterns among physicians with several experiences and skill levels of neuroendovascular treatment. Learning the gazing patterns of experts using eye tracking may be a good educational tool for novices and trainees.

How to Read an Abdominal CT: Insights from the Visual and Cognitive Sciences Translated for Clinical Practice

When first learning abdominal CT studies, residents are often given little concrete, practical direction. There is, however, a large literature from the visual and cognitive sciences that can provide guidance towards search strategies that maximize efficiency and comprehensiveness. This literature has not penetrated radiology teaching to any great extent. In this article, we will examine the current pedagogy (and why that falls short), why untutored search fails, where misses occur in abdomen/pelvis CT, why these misses occur where they do, how expert radiologists search 3d image stacks, and how novices might expedite the acquisition of expertise.

The cognitive and perceptual processes that affect observer performance in lung cancer detection: a scoping review

INTRODUCTION

Early detection of malignant pulmonary nodules through screening has been shown to reduce lung cancer-related mortality by 20%. However, perceptual and cognitive factors that affect nodule detection are poorly understood. This review examines the cognitive and visual processes of various observers, with a particular focus on radiologists, during lung nodule detection.

METHODS

Four databases (Medline, Embase, Scopus and PubMed) were searched to extract studies on eye-tracking in pulmonary nodule detection. Studies were included if they used eye-tracking to assess the search and detection of lung nodules in computed tomography or 2D radiographic imaging. Data were charted according to identified themes and synthesised using a thematic narrative approach.

RESULTS

The literature search yielded 25 articles and five themes were discovered: 1 - functional visual field and satisfaction of search, 2 - expert search patterns, 3 - error classification through dwell time, 4 - the impact of the viewing environment and 5 - the effect of prevalence expectation on search. Functional visual field reduced to 2.7° in 3D imaging compared to 5° in 2D radiographs. Although greater visual coverage improved nodule detection, incomplete search was not responsible for missed nodules. Most radiological errors during lung nodule detection were decision-making errors (30%-45%). Dwell times associated with false-positive (FP) decisions informed feedback systems to improve diagnosis. Interruptions did not influence diagnostic performance; however, it increased viewing time by 8% and produced a 23.1% search continuation accuracy. Comparative scanning was found to increase the detection of low contrast nodules. Prevalence expectation did not directly affect diagnostic accuracy; however, decision-making time increased by 2.32 seconds with high prevalence expectations.

CONCLUSION

Visual and cognitive factors influence pulmonary nodule detection. Insights gained from eye-tracking can inform advancements in lung screening. Further exploration of eye-tracking in lung screening, particularly with low-dose computed tomography (LDCT), will benefit the future of lung cancer screening.

Screen-detected solid nodules: from detection of nodule to structured reporting

Lung cancer screening (LCS) is gaining some interest worldwide after positive results from International trials. Unlike other screening practices, LCS is performed by an extremely sensitive test, namely low-dose computed tomography (LDCT) that can detect the smallest nodules in lung parenchyma. Up-to-date detection approaches, such as computer aided detection systems, have been increasingly employed for lung nodule automatic identification and are largely used in most LCS programs as a complementary tool to visual reading. Solid nodules of any size are represented in the vast majority of subjects undergoing LDCT. However, less than 1% of solid nodules will be diagnosed lung cancer. This fact calls for specific characterization of nodules to avoid false positives, overinvestigation, and reduce the risks associated with nodule work up. Recent research has been exploring the potential of artificial intelligence, including deep learning techniques, to enhance the accuracy of both detection and characterisation of lung nodule. Computer aided detection and diagnosis algorithms based on artificial intelligence approaches have demonstrated the ability to accurately detect and characterize parenchymal nodules, reducing the number of false positives, and to outperform some of the currently used risk models for prediction of lung cancer risk, potentially reducing the proportion of surveillance CT scans. These forthcoming approaches will eventually integrate a new reasoning for development of future guidelines, which are expected to evolve into precision and personalized stratification of lung cancer risk stratification by continuous fashion, as opposed to the current format with a limited number of risk classes within fixed thresholds of nodule size. This review aims to detail the standard of reference for optimal management of solid nodules by low-dose computed and its projection into the fine selection of candidates for work up.

The Search Patterns of Abdominal Imaging Subspecialists for Abdominal Computed Tomography: Toward a Foundational Pattern for New Radiology Residents

Objectives:

The routine search patterns used by subspecialty abdominal imaging experts to inspect the image volumes of abdominal/pelvic computed tomography (CT) have not been well characterized or rendered in practical or teachable terms. The goal of this study is to describe the search patterns used by experienced subspecialty imagers when reading a normal abdominal CT at a modern picture archiving and communication system workstation, and utilize this information to propose guidelines for residents as they learn to interpret CT during training.

Material and Methods:

Twenty-two academic subspecialists enacted their routine search pattern on a normal contrast-enhanced abdominal/pelvic CT study under standardized display parameters. Readers were told that the scan was normal and then asked to verbalize where their gaze centered and moved through the axial, coronal, and sagittal image stacks, demonstrating eye position with a cursor as needed. A peer coded the reported eye gaze movements and scrilling behavior. Spearman correlation coefficients were calculated between years of professional experience and the numbers of passes through the lung bases, liver, kidneys, and bowel.

Results:

All readers followed an initial organ-by-organ approach. Larger organs were examined by drilling, while smaller organs by oscillation or scanning. Search elements were classified as drilling, scanning, oscillation, and scrilling (scan drilling); these categories were parsed as necessary. The greatest variability was found in the examination the body wall and bowel/mesentery. Two modes of scrilling were described, and these classified as roaming and zigzagging. The years of experience of the readers did not correlated to number of passes made through the lung bases, liver, kidneys, or bowel.

Conclusion:

Subspecialty abdominal radiologists negotiate through the image stacks of an abdominal CT study in broadly similar ways. Collation of the approaches suggests a foundational search pattern for new trainees.

Image Annotation by Eye Tracking: Accuracy and Precision of Centerlines of Obstructed Small-Bowel Segments Placed Using Eye Trackers

Small-bowel obstruction (SBO) is a common and important disease, for which machine learning tools have yet to be developed. Image annotation is a critical first step for development of such tools. This study assesses whether image annotation by eye tracking is sufficiently accurate and precise to serve as a first step in the development of machine learning tools for detection of SBO on CT. Seven subjects diagnosed with SBO by CT were included in the study. For each subject, an obstructed segment of bowel was chosen. Three observers annotated the centerline of the segment by manual fiducial placement and by visual fiducial placement using a Tobii 4c eye tracker. Each annotation was repeated three times. The distance between centerlines was calculated after alignment using dynamic time warping (DTW) and statistically compared to clinical thresholds for diagnosis of SBO. Intra-observer DTW distance between manual and visual centerlines was calculated as a measure of accuracy. These distances were 1.1 ± 0.2, 1.3 ± 0.4, and 1.8 ± 0.2 cm for the three observers and were less than 1.5 cm for two of three observers (P < 0.01). Intra- and inter-observer DTW distances between centerlines placed with each method were calculated as measures of precision. These distances were 0.6 ± 0.1 and 0.8 ± 0.2 cm for manual centerlines, 1.1 ± 0.4 and 1.9 ± 0.6 cm for visual centerlines, and were less than 3.0 cm in all cases (P < 0.01). Results suggest that eye tracking-based annotation is sufficiently accurate and precise for small-bowel centerline annotation for use in machine learning-based applications.

Putting artificial intelligence (AI) on the spot: machine learning evaluation of pulmonary nodules

Lung cancer remains the leading cause of cancer related death world-wide despite advances in treatment. This largely relates to the fact that many of these patients already have advanced diseases at the time of initial diagnosis. As most lung cancers present as nodules initially, an accurate classification of pulmonary nodules as early lung cancers is critical to reducing lung cancer morbidity and mortality. There have been significant recent advances in artificial intelligence (AI) for lung nodule evaluation. Deep learning (DL) and convolutional neural networks (CNNs) have shown promising results in pulmonary nodule detection and have also excelled in segmentation and classification of pulmonary nodules. This review aims to provide an overview of progress that has been made in AI recently for pulmonary nodule detection and characterization with the ultimate goal of lung cancer prediction and classification while outlining some of the pitfalls and challenges that remain to bring such advancements to routine clinical use.

Observer Performance for Detection of Pulmonary Nodules at Chest CT over a Large Range of Radiation Dose Levels

Background There is a wide variation in radiation dose levels that can be used with chest CT in order to detect indeterminate pulmonary nodules. Purpose To compare the performance of lower-radiation-dose chest CT with that of routine dose in the detection of indeterminate pulmonary nodules 5 mm or greater. Materials and Methods In this retrospective study, CT projection data from 83 routine-dose chest CT examinations performed in 83 patients (120 kV, 70 quality reference mAs [QRM]) were collected between November 2013 and April 2014. Reference indeterminate pulmonary nodules were identified by two nonreader thoracic radiologists. By using validated noise insertion, five lower-dose data sets were reconstructed with filtered back projection (FBP) or iterative reconstruction (IR; 30 QRM with FBP, 10 QRM with IR, 5 QRM with FBP, 5 QRM with IR, and 2.5 QRM with IR). Three thoracic radiologists circled pulmonary nodules, rating confidence that the nodule was a 5-mm-or-greater indeterminate pulmonary nodule, and graded image quality. Analysis was performed on a per-nodule basis by using jackknife alternative free-response receiver operating characteristic figure of merit (FOM) and noninferiority limit of -0.10. Results There were 66 indeterminate pulmonary nodules (mean size, 8.6 mm ± 3.4 [standard deviation]; 21 part-solid nodules) in 42 patients (mean age, 51 years ± 17; 21 men and 21 women). Compared with the FOM for routine-dose CT (size-specific dose estimate, 6.5 mGy ± 1.8; FOM, 0.86 [95% confidence interval: 0.80, 0.91]), FOM was noninferior for all lower-dose configurations except for 2.5 QRM with IR. The sensitivity for subsolid nodules at 70 QRM was 60% (range, 48%-72%) and was significantly worse at a dose of 5 QRM and lower, whether or not IR was used (P < .05). Diagnostic image quality decreased with decreasing dose (P < .001) and was better with IR at 5 QRM (P < .05). Conclusion CT images reconstructed at dose levels down to 10 quality reference mAs (size-specific dose estimate, 0.9 mGy) had noninferior performance compared with routine dose in depicting pulmonary nodules. Iterative reconstruction improved subjective image quality but not performance at low dose levels. © RSNA, 2020 Online supplemental material is available for this article. See also the editorial by White and Kazerooni in this issue.

Radiomics: an overview in lung cancer management-a narrative review

Radiomics is a novel approach for optimizing the analysis massive data from medical images to provide auxiliary guidance in clinical issues. Quantitative feature extraction is one of the critical steps of radiomics. The association between radiomics features and the clinicopathological information of diseases can be identified by several statistics methods. For instance, although significant progress has been made in the field of lung cancer, too many questions remain, especially for the individualized decisions. Radiomics offers a new tool to encode the characteristics of lung cancer which is the leading cause of cancer-related deaths worldwide. Here, we reviewed the workflow and clinical utility of radiomics in lung cancer management, including pulmonary nodules detection, classification, histopathology and genetics evaluation, clinical staging, therapy response, and prognosis prediction. Most of these studies showed positive results, indicating the potential value of radiomics in clinical practice. The implementation of radiomics is both feasible and invaluable, and has aided clinicians in ascertaining the nature of a disease with greater precision. However, it should be noted that radiomics in its current state cannot completely replace the work of therapists or tissue examination. The potential future trends of this modality were also remarked. More efforts are needed to overcome the limitations identified above in order to facilitate the widespread application of radiomics in the reasonably near future.

Search of low-contrast liver lesions in abdominal CT: the importance of scrolling behavior

Purpose: Visual search using volumetric images is becoming the standard in medical imaging. However, we do not fully understand how eye movement strategies mediate diagnostic performance. A recent study on computed tomography (CT) images showed that the search strategies of radiologists could be classified based on saccade amplitudes and cross-quadrant eye movements [eye movement index (EMI)] into two categories: drillers and scanners. Approach: We investigate how the number of times a radiologist scrolls in a given direction during analysis of the images (number of courses) could add a supplementary variable to use to characterize search strategies. We used a set of 15 normal liver CT images in which we inserted 1 to 5 hypodense metastases of two different signal contrast amplitudes. Twenty radiologists were asked to search for the metastases while their eye-gaze was recorded by an eye-tracker device (EyeLink1000, SR Research Ltd., Mississauga, Ontario, Canada). Results: We found that categorizing radiologists based on the number of courses (rather than EMI) could better predict differences in decision times, percentage of image covered, and search error rates. Radiologists with a larger number of courses covered more volume in more time, found more metastases, and made fewer search errors than those with a lower number of courses. Our results suggest that the traditional definition of drillers and scanners could be expanded to include scrolling behavior. Drillers could be defined as scrolling back and forth through the image stack, each time exploring a different area on each image (low EMI and high number of courses). Scanners could be defined as scrolling progressively through the stack of images and focusing on different areas within each image slice (high EMI and low number of courses). Conclusions: Together, our results further enhance the understanding of how radiologists investigate three-dimensional volumes and may improve how to teach effective reading strategies to radiology residents.

The Radiologist's Gaze: Mapping Three-Dimensional Visual Search in Computed Tomography of the Abdomen and Pelvis

A radiologist's search pattern can directly influence patient management. A missed finding is a missed opportunity for intervention. Multiple studies have attempted to describe and quantify search patterns but have mainly focused on chest radiographs and chest CTs. Here, we describe and quantify the visual search patterns of 17 radiologists as they scroll through 6 CTs of the abdomen and pelvis. Search pattern tracings varied among individuals and remained relatively consistent per individual between cases. Attendings and trainees had similar eye metric statistics with respect to time to first fixation (TTFF), number of fixations in the region of interest (ROI), fixation duration in ROI, mean saccadic amplitude, or total number of fixations. Attendings had fewer numbers of fixations per second versus trainees (p < 0.001), suggesting efficiency due to expertise. In those cases that were accurately interpreted, TTFF was shorter (p = 0.04), the number of fixations per second and number of fixations in ROI were higher (p = 0.04, p = 0.02, respectively), and fixation duration in ROI was increased (p = 0.02). We subsequently categorized radiologists as "scanners" or "drillers" by both qualitative and quantitative methods and found no differences in accuracy with most radiologists being categorized as "drillers." This study describes visual search patterns of radiologists in interpretation of CTs of the abdomen and pelvis to better approach future endeavors in determining the effects of manipulations such as fatigue, interruptions, and computer-aided detection.

Measurement of the useful field of view for single slices of different imaging modalities and targets

Purpose: With three-dimensional (3-D) images displayed as stacks of 2-D images, radiologists rely more heavily on vision away from their fixation point to visually process information, guide eye movements, and detect abnormalities. Thus the ability to detect targets away from the fixation point, commonly characterized as the useful field of view (UFOV), becomes critical for these 3-D imaging modalities. We investigate how the UFOV, defined as the eccentricity, in which detection performance degrades to a given probability, varies across imaging modalities and targets. Approach: We measure the detectability of different targets at various distances from gaze locations for single slices of liver computed tomography (CT), 2-D digital mammograms (DM), and single slices of digital breast tomosynthesis (DBT) cases. Observers with varying expertise were instructed to maintain their gaze at a point while a short display of the image was flashed and an eye tracker verified observer's steady fixation. Display times were 200 and 1000 ms for CT images and 500 ms for DM and DBT images. Results: We find variations in the UFOV from 9 to 12 deg for liver CT to as small as 2.5 to 5 deg for calcification clusters in breast images (DM and DBT). We compare our results to those reported in the literature for lung nodules and discuss the differences across methods used to measure the UFOV, their dependence on case selection/task difficulty, viewing conditions, and observer expertise. We propose a complementary measure defined in terms of performance degradation relative to the peak foveal performance (relative-UFOV) to circumvent UFOV's variations with case selection/task difficulty. Conclusion: Our results highlight the variations in the UFOV across imaging modalities, target types, observer expertise, and measurement methods and suggest an additional relative-UFOV measure to more thoroughly characterize the detection performance away from point of fixation.

Influence of background lung characteristics on nodule detection with computed tomography

We sought to characterize local lung complexity in chest computed tomography (CT) and to characterize its impact on the detectability of pulmonary nodules. Forty volumetric chest CT scans were created by embedding between three and five simulated 5-mm lung nodules into one of three volumetric chest CT datasets. Thirteen radiologists evaluated 157 nodules, resulting in 2041 detection opportunities. Analyzing the substrate CT data prior to nodule insertion, 14 image features were measured within a region around each nodule location. A generalized linear mixed-effects statistical model was fit to the data to verify the contribution of each metric on detectability. The model was tuned for simplicity, interpretability, and generalizability using stepwise regression applied to the primary features and their interactions. We found that variables corresponding to each of five categories (local structural distractors, local intensity, global context, local vascularity, and contiguity with structural distractors) were significant ( p < 0.01 ) factors in a standardized model. Moreover, reader-specific models conveyed significant differences among readers with significant distraction (missed detections) influenced by local intensity- versus local-structural characteristics being mutually exclusive. Readers with significant local intensity distraction ( n = 10 ) detected substantially fewer lung nodules than those who were significantly distracted by local structure ( n = 2 ), 46.1% versus 65.3% mean nodules detected, respectively.

A comparison of visual identification of dental radiographic and nonradiographic images using eye tracking technology

OBJECTIVES

Eye tracking has been used in medical radiology to understand observers' gaze patterns during radiological diagnosis. This study examines the visual identification ability of junior hospital dental officers (JHDOs) and dental surgery assistants (DSAs) in radiographic and nonradiographic images using eye tracking technology and examines if there is a correlation.

MATERIAL AND METHODS

Nine JHDOs and nine DSAs examined six radiographic images and 16 nonradiographic images using eye tracking. The areas of interest (AOIs) of the radiographic images were rated as easy, medium, and hard, and the nonradiographic images were categorized as pattern recognition, face recognition, and image comparison. The participants were required to identify and locate the AOIs. Data analysis of the two domains, entire slide and AOI, was conducted by evaluating the eye tracking metrics (ETM) and the performance outcomes. ETM consisted of six parameters, and performance outcomes consisted of four parameters.

RESULTS

No significant differences were observed for ETMs for JHDOs and DSAs for both radiographic and nonradiographic images. The JHDOs showed significantly higher percentage in identifying AOIs than DSAs for all the radiographic images (72.7% vs. 36.4%, p = .004) and for the easy categorization of radiographic AOIs (85.7% vs. 42.9%, p = .012). JHDOs with higher correct identification percentage in face recognition had a shorter dwell time in AOIs.

CONCLUSIONS

Although no significant relation was observed between radiographic and nonradiographic images, there were some evidence that visual recognition skills may impact certain attributes of the visual search pattern in radiographic images.

A Review of Perceptual Expertise in Radiology-How it develops, How we can test it, and Why humans still matter in the era of Artificial Intelligence

As the first step in image interpretation is detection, an error in perception can prematurely end the diagnostic process leading to missed diagnoses. Because perceptual errors of this sort-"failure to detect"-are the most common interpretive error (and cause of litigation) in radiology, understanding the nature of perceptual expertise is essential in decreasing radiology's long-standing error rates. In this article, we review what constitutes a perceptual error, the existing models of radiologic image perception, the development of perceptual expertise and how it can be tested, perceptual learning methods in training radiologists, and why understanding perceptual expertise is still relevant in the era of artificial intelligence. Adding targeted interventions, such as perceptual learning, to existing teaching practices, has the potential to enhance expertise and reduce medical error.

What do we know about volumetric medical image interpretation?: a review of the basic science and medical image perception literatures

Interpretation of volumetric medical images represents a rapidly growing proportion of the workload in radiology. However, relatively little is known about the strategies that best guide search behavior when looking for abnormalities in volumetric images. Although there is extensive literature on two-dimensional medical image perception, it is an open question whether the conclusions drawn from these images can be generalized to volumetric images. Importantly, volumetric images have distinct characteristics (e.g., scrolling through depth, smooth-pursuit eye-movements, motion onset cues, etc.) that should be considered in future research. In this manuscript, we will review the literature on medical image perception and discuss relevant findings from basic science that can be used to generate predictions about expertise in volumetric image interpretation. By better understanding search through volumetric images, we may be able to identify common sources of error, characterize the optimal strategies for searching through depth, or develop new training and assessment techniques for radiology residents.

Analysis of Perceptual Expertise in Radiology - Current Knowledge and a New Perspective

Radiologists rely principally on visual inspection to detect, describe, and classify findings in medical images. As most interpretive errors in radiology are perceptual in nature, understanding the path to radiologic expertise during image analysis is essential to educate future generations of radiologists. We review the perceptual tasks and challenges in radiologic diagnosis, discuss models of radiologic image perception, consider the application of perceptual learning methods in medical training, and suggest a new approach to understanding perceptional expertise. Specific principled enhancements to educational practices in radiology promise to deepen perceptual expertise among radiologists with the goal of improving training and reducing medical error.

Eye Movements in Medical Image Perception: A Selective Review of Past, Present and Future

The eye movements of experts, reading medical images, have been studied for many years. Unlike topics such as face perception, medical image perception research needs to cope with substantial, qualitative changes in the stimuli under study due to dramatic advances in medical imaging technology. For example, little is known about how radiologists search through 3D volumes of image data because they simply did not exist when earlier eye tracking studies were performed. Moreover, improvements in the affordability and portability of modern eye trackers make other, new studies practical. Here, we review some uses of eye movements in the study of medical image perception with an emphasis on newer work. We ask how basic research on scene perception relates to studies of medical 'scenes' and we discuss how tracking experts' eyes may provide useful insights for medical education and screening efficiency.

Artificial intelligence and radiomics in pulmonary nodule management: current status and future applications

Artificial intelligence (AI) has been present in some guise within the field of radiology for over 50 years. The first studies investigating computer-aided diagnosis in thoracic radiology date back to the 1960s, and in the subsequent years, the main application of these techniques has been the detection and classification of pulmonary nodules. In addition, there have been other less intensely researched applications, such as the diagnosis of interstitial lung disease, chronic obstructive pulmonary disease, and the detection of pulmonary emboli. Despite extensive literature on the use of convolutional neural networks in thoracic imaging over the last few decades, we are yet to see these systems in use in clinical practice. The article reviews current state-of-the-art applications of AI and in detection, classification, and follow-up of pulmonary nodules and how deep-learning techniques might influence these going forward. Finally, we postulate the impact of these advancements on the role of radiologists and the importance of radiologists in the development and evaluation of these techniques.

Interchangeability between real and three-dimensional simulated lung tumors in computed tomography: an interalgorithm volumetry study

Using hybrid datasets consisting of patient-derived computed tomography (CT) images with digitally inserted computational tumors, we establish volumetric interchangeability between real and computational lung tumors in CT. Pathologically-confirmed malignancies from 30 thoracic patient cases from the RIDER database were modeled. Tumors were either isolated or attached to lung structures. Patient images were acquired on one of two CT scanner models (Lightspeed 16 or VCT; GE Healthcare) using standard chest protocol. Real tumors were segmented and used to inform the size and shape of simulated tumors. Simulated tumors developed in Duke Lesion Tool (Duke University) were inserted using a validated image-domain insertion program. Four readers performed volume measurements using three commercial segmentation tools. We compared the volume estimation performance of segmentation tools between real tumors in actual patient CT images and corresponding simulated tumors virtually inserted into the same patient images (i.e., hybrid datasets). Comparisons involved (1) direct assessment of measured volumes and the standard deviation between simulated and real tumors across readers and tools, respectively, (2) multivariate analysis, involving segmentation tools, readers, tumor shape, and attachment, and (3) effect of local tumor environment on volume measurement. Volume comparison showed consistent trends (9% volumetric difference) between real and simulated tumors across all segmentation tools, readers, shapes, and attachments. Across all cases, readers, and segmentation tools, an intraclass correlation coefficient = 0.99 indicates that simulated tumors correlated strongly with real tumors ( p = 0.95 ). In addition, the impact of the local tumor environment on tumor volume measurement was found to have a segmentation tool-related influence. Strong agreement between simulated tumors modeled in this study compared to their real counterparts suggests a high degree of similarity. This indicates that, volumetrically, simulated tumors embedded into patient CT data can serve as reasonable surrogates to real patient data.

Radiologist performance in the detection of lung cancer using CT

AIM

To measure the level of radiologists' performance in lung cancer detection, and to explore radiologists' performance in cancer specialised and non-specialised centres.

MATERIALS AND METHODS

Thirty radiologists read 60 chest computed tomography (CT) examinations. Thirty cases had surgically or biopsy-proven lung cancer and 30 were cancer-free cases. The cancer cases were validated by four expert radiologists who located the malignant lung nodules. Reader performance was evaluated by calculating sensitivity, location sensitivity, specificity, and area under the receiver operating characteristic (ROC) curve (AUC). In addition, sensitivity at fixed specificity (0.794) was computed from each reader's estimated ROC curve.

RESULTS

The radiologists had a mean sensitivity of 0.749, sensitivity at fixed specificity of 0.744, location sensitivity of 0.666, specificity of 0.81 and AUC of 0.846. Radiologists in the specialised and non-specialised cancer centres had the following (specialised, non-specialised) pairs of values: sensitivity=(0.80, 0.719); sensitivity for fixed 0.794 specificity=(0.752, 0.740); location sensitivity=(0.712, 0.637); specificity=(0.794, 0.82) and AUC=(0.846, 0.846).

CONCLUSION

The efficacy of radiologists was comparable to other studies. Furthermore, AUC outcomes were similar for specialised and non-specialised cancer centre radiologists, suggesting they have similar discriminatory ability and that the higher sensitivity and lower specificity for specialised-centre radiologists can be attributed to them being less conservative in interpreting case images.

Local complexity metrics to quantify the effect of anatomical noise on detectability of lung nodules in chest CT imaging

The purpose of this study is to (1) develop metrics to characterize the regional anatomical complexity of the lungs, and (2) relate these metrics with lung nodule detection in chest CT. A free-scrolling reader-study with virtually inserted nodules (13 radiologists × 157 total nodules = 2041 responses) is used to characterize human detection performance. Metrics of complexity based on the local density and orientation of distracting vasculature are developed for two-dimensional (2-D) and three-dimensional (3-D) considerations of the image volume. Assessed characteristics included the distribution of 2-D/3-D vessel structures of differing orientation (dubbed "2-D/3-D and dot-like/line-like distractor indices"), contiguity of inserted nodules with local vasculature, mean local gray-level surrounding each nodule, the proportion of lung voxels to total voxels in each section, and 3-D distance of each nodule from the trachea bifurcation. A generalized linear mixed-effects statistical model is used to determine the influence of each these metrics on nodule detectability. In order of decreasing effect size: 3-D line-like distractor index, 2-D line-like distractor index, 2-D dot-like distractor index, local mean gray-level, contiguity with 2-D dots, lung area, and contiguity with 3-D lines all significantly affect detectability ( P < 0.05 ). These data demonstrate that local lung complexity degrades detection of lung nodules.

CT Detectability of Small Low-Contrast Hypoattenuating Focal Lesions: Iterative Reconstructions versus Filtered Back Projection

Purpose To investigate performance in detectability of small (≤1 cm) low-contrast hypoattenuating focal lesions by using filtered back projection (FBP) and iterative reconstruction (IR) algorithms from two major CT vendors across a range of 11 radiation exposures. Materials and Methods A low-contrast detectability phantom consisting of 21 low-contrast hypoattenuating focal objects (seven sizes between 2.4 and 10.0 mm, three contrast levels) embedded into a liver-equivalent background was scanned at 11 radiation exposures (volume CT dose index range, 0.5-18.0 mGy; size-specific dose estimate [SSDE] range, 0.8-30.6 mGy) with four high-end CT platforms. Data sets were reconstructed by using FBP and varied strengths of image-based, model-based, and hybrid IRs. Sixteen observers evaluated all data sets for lesion detectability by using a two-alternative-forced-choice (2AFC) paradigm. Diagnostic performances were evaluated by calculating area under the receiver operating characteristic curve (AUC) and by performing noninferiority analyses. Results At benchmark exposure, FBP yielded a mean AUC of 0.79 ± 0.09 (standard deviation) across all platforms which, on average, was approximately 2% lower than that observed with the different IR algorithms, which showed an average AUC of 0.81 ± 0.09 (P = .12). Radiation decreases of 30%, 50%, and 80% resulted in similar declines of observer detectability with FBP (mean AUC decrease, -0.02 ± 0.05, -0.03 ± 0.05, and -0.05 ± 0.05, respectively) and all IR methods investigated (mean AUC decrease, -0.00 ± 0.05, -0.04 ± 0.05, and -0.04 ± 0.05, respectively). For each radiation level and CT platform, variance in performance across observers was greater than that across reconstruction algorithms (P = .03). Conclusion Iterative reconstruction algorithms have limited radiation optimization potential in detectability of small low-contrast hypoattenuating focal lesions. This task may be further complicated by a high degree of variation in radiologists' performances, seemingly exceeding real performance differences among reconstruction algorithms. © RSNA, 2018 Online supplemental material is available for this article.

Long-Term Active Surveillance of Screening Detected Subsolid Nodules is a Safe Strategy to Reduce Overtreatment

INTRODUCTION

Lung cancer presenting as subsolid nodule (SSN) can show slow growth, hence treating SSN is controversial. Our aim was to determine the long-term outcome of subjects with unresected SSNs in lung cancer screening.

METHODS

Since 2005, the Multicenter Italian Lung Detection (MILD) screening trial implemented active surveillance for persistent SSN, as opposed to early resection. Presence of SSNs was related to diagnosis of cancer at the site of SSN, elsewhere in the lung, or in the body. The risk of overall mortality and lung cancer mortality was tested by Cox proportional hazards model.

RESULTS

SSNs were found in 16.9% (389 of 2303) of screenees. During 9.3 ± 1.2 years of follow-up, the hazard ratio of lung cancer diagnosis in subjects with SSN was 6.77 (95% confidence interval: 3.39-13.54), with 73% (22 of 30) of cancers not arising from SSN (median time to diagnosis 52 months from SSN). Lung cancer-specific mortality in subjects with SSN was significantly increased (hazard ratio = 3.80; 95% confidence interval: 1.24-11.65) compared to subjects without lung nodules. Lung cancer arising from SSN did not lead to death within the follow-up period.

CONCLUSIONS

Subjects with SSN in the MILD cohort showed a high risk of developing lung cancer elsewhere in the lung, with only a minority of cases arising from SSN, and never representing the cause of death. These results show the safety of active surveillance for conservative management of SSN until signs of solid component growth and the need for prolonged follow-up because of high risk of other cancers.

The utilisation of convolutional neural networks in detecting pulmonary nodules: a review

Lung cancer is one of the leading causes of cancer-related fatality in the world. Patients display few or even no signs or symptoms in the early stages, resulting in up to 75% of patients diagnosed in the later stages of the disease. Consequently, there has been a call for lung cancer screening amongst at-risk populations. The early detection of malignant pulmonary nodules in CT is one of the suggested methods proposed to diagnose early-stage lung cancer; however, the reported sensitivity of radiologists' ability to accurately detect pulmonary nodules ranges widely from 30 to 97%. 2012 saw Alex Krizhevsky present a paper titled "ImageNet Classification with Deep Convolutional Networks" in which a multilayered convolutional computational model known as a convolutional neural network (CNN) was confirmed competent in identifying and classifying 1.2 million images to a previously unseen level of accuracy. Since then, CNNs have gained attention as a potential tool in aiding radiologists' detection of pulmonary nodules in CT imaging. This review found the use of CNN is a viable strategy to increase the overall sensitivity of pulmonary nodule detection. Small, non-validated data sets, computational constraints, and incomparable studies are currently limited factors of the existing research.

Emerging applications of eye-tracking technology in dermatology

Eye-tracking technology has been used within a multitude of disciplines to provide data linking eye movements to visual processing of various stimuli (i.e., x-rays, situational positioning, printed information, and warnings). Despite the benefits provided by eye-tracking in allowing for the identification and quantification of visual attention, the discipline of dermatology has yet to see broad application of the technology. Notwithstanding dermatologists' heavy reliance upon visual patterns and cues to discriminate between benign and atypical nevi, literature that applies eye-tracking to the study of dermatology is sparse; and literature specific to patient-initiated behaviors, such as skin self-examination (SSE), is largely non-existent. The current article provides a review of eye-tracking research in various medical fields, culminating in a discussion of current applications and advantages of eye-tracking for dermatology research.

Quantifying the costs of interruption during diagnostic radiology interpretation using mobile eye-tracking glasses

What are the costs and consequences of interruptions during diagnostic radiology? The cognitive psychology literature suggests that interruptions lead to an array of negative consequences that could hurt patient outcomes and lead to lower patient throughput. Meanwhile, observational studies have both noted a strikingly high rate of interruptions and rising number of interruptions faced by radiologists. There is some observational evidence that more interruptions could lead to worse patient outcomes: Balint et al. (2014) found that the shifts with more telephone calls received in the reading room were associated with more discrepant calls. The purpose of the current study was to use an experimental manipulation to precisely quantify the costs of two different types of interruption: telephone interruption and an interpersonal interruption. We found that the first telephone interruption led to a significant increase in time spent on the case, but there was no effect on diagnostic accuracy. Eye-tracking revealed that interruptions strongly influenced where the radiologists looked: they tended to spend more time looking at dictation screens and less on medical images immediately after interruption. Our results demonstrate that while radiologists’ eye movements are reliably influenced by interruptions, the behavioral consequences were relatively mild, suggesting effective compensatory mechanisms.

Evaluation of Search Strategies for Microcalcifications and Masses in 3D Images

Medical imaging is quickly evolving towards 3D image modalities such as computed tomography (CT), magnetic resonance imaging (MRI) and digital breast tomosynthesis (DBT). These 3D image modalities add volumetric information but further increase the need for radiologists to search through the image data set. Although much is known about search strategies in 2D images less is known about the functional consequences of different 3D search strategies. We instructed readers to use two different search strategies: drillers had their eye movements restricted to a few regions while they quickly scrolled through the image stack, scanners explored through eye movements the 2D slices. We used real-time eye position monitoring to ensure observers followed the drilling or the scanning strategy while approximately preserving the percentage of the volumetric data covered by the useful field of view. We investigated search for two signals: a simulated microcalcification and a larger simulated mass. Results show an interaction between the search strategy and lesion type. In particular, scanning provided significantly better detectability for microcalcifications at the cost of 5 times more time to search while there was little change in the detectability for the larger simulated masses. Analyses of eye movements support the hypothesis that the effectiveness of a search strategy in 3D imaging arises from the interaction of the fixational sampling of visual information and the signals' visibility in the visual periphery.

Even if I showed you where you looked, remembering where you just looked is hard

People know surprisingly little about their own visual behavior, which can be problematic when learning or executing complex visual tasks such as search of medical images. We investigated whether providing observers with online information about their eye position during search would help them recall their own fixations immediately afterwards. Seventeen observers searched for various objects in “Where's Waldo” images for 3 s. On two-thirds of trials, observers made target present/absent responses. On the other third (critical trials), they were asked to click twelve locations in the scene where they thought they had just fixated. On half of the trials, a gaze-contingent window showed observers their current eye position as a 7.5° diameter “spotlight.” The spotlight “illuminated” everything fixated, while the rest of the display was still visible but dimmer. Performance was quantified as the overlap of circles centered on the actual fixations and centered on the reported fixations. Replicating prior work, this overlap was quite low (26%), far from ceiling (66%) and quite close to chance performance (21%). Performance was only slightly better in the spotlight condition (28%, p = 0.03). Giving observers information about their fixation locations by dimming the periphery improved memory for those fixations modestly, at best.

Techniques for virtual lung nodule insertion: volumetric and morphometric comparison of projection-based and image-based methods for quantitative CT

Virtual nodule insertion paves the way towards the development of standardized databases of hybrid CT images with known lesions. The purpose of this study was to assess three methods (an established and two newly developed techniques) for inserting virtual lung nodules into CT images. Assessment was done by comparing virtual nodule volume and shape to the CT-derived volume and shape of synthetic nodules. 24 synthetic nodules (three sizes, four morphologies, two repeats) were physically inserted into the lung cavity of an anthropomorphic chest phantom (KYOTO KAGAKU). The phantom was imaged with and without nodules on a commercial CT scanner (SOMATOM Definition Flash, Siemens) using a standard thoracic CT protocol at two dose levels (1.4 and 22 mGy CTDI_vol). Raw projection data were saved and reconstructed with filtered back-projection and sinogram affirmed iterative reconstruction (SAFIRE, strength 5) at 0.6 mm slice thickness. Corresponding 3D idealized, virtual nodule models were co-registered with the CT images to determine each nodule's location and orientation. Virtual nodules were voxelized, partial volume corrected, and inserted into nodule-free CT data (accounting for system imaging physics) using two methods: projection-based Technique A, and image-based Technique B. Also a third Technique C based on cropping a region of interest from the acquired image of the real nodule and blending it into the nodule-free image was tested. Nodule volumes were measured using a commercial segmentation tool (iNtuition, TeraRecon, Inc.) and deformation was assessed using the Hausdorff distance. Nodule volumes and deformations were compared between the idealized, CT-derived and virtual nodules using a linear mixed effects regression model which utilized the mean, standard deviation, and coefficient of variation ([Formula: see text], [Formula: see text] and [Formula: see text] of the regional Hausdorff distance. Overall, there was a close concordance between the volumes of the CT-derived and virtual nodules. Percent differences between them were less than 3% for all insertion techniques and were not statistically significant in most cases. Correlation coefficient values were greater than 0.97. The deformation according to the Hausdorff distance was also similar between the CT-derived and virtual nodules with minimal statistical significance in the ([Formula: see text]) for Techniques A, B, and C. This study shows that both projection-based and image-based nodule insertion techniques yield realistic nodule renderings with statistical similarity to the synthetic nodules with respect to nodule volume and deformation. These techniques could be used to create a database of hybrid CT images containing nodules of known size, location and morphology.

How visual search relates to visual diagnostic performance: a narrative systematic review of eye-tracking research in radiology

Eye tracking research has been conducted for decades to gain understanding of visual diagnosis such as in radiology. For educational purposes, it is important to identify visual search patterns that are related to high perceptual performance and to identify effective teaching strategies. This review of eye-tracking literature in the radiology domain aims to identify visual search patterns associated with high perceptual performance. Databases PubMed, EMBASE, ERIC, PsycINFO, Scopus and Web of Science were searched using 'visual perception' OR 'eye tracking' AND 'radiology' and synonyms. Two authors independently screened search results and included eye tracking studies concerning visual skills in radiology published between January 1, 1994 and July 31, 2015. Two authors independently assessed study quality with the Medical Education Research Study Quality Instrument, and extracted study data with respect to design, participant and task characteristics, and variables. A thematic analysis was conducted to extract and arrange study results, and a textual narrative synthesis was applied for data integration and interpretation. The search resulted in 22 relevant full-text articles. Thematic analysis resulted in six themes that informed the relation between visual search and level of expertise: (1) time on task, (2) eye movement characteristics of experts, (3) differences in visual attention, (4) visual search patterns, (5) search patterns in cross sectional stack imaging, and (6) teaching visual search strategies. Expert search was found to be characterized by a global-focal search pattern, which represents an initial global impression, followed by a detailed, focal search-to-find mode. Specific task-related search patterns, like drilling through CT scans and systematic search in chest X-rays, were found to be related to high expert levels. One study investigated teaching of visual search strategies, and did not find a significant effect on perceptual performance. Eye tracking literature in radiology indicates several search patterns are related to high levels of expertise, but teaching novices to search as an expert may not be effective. Experimental research is needed to find out which search strategies can improve image perception in learners.

Use of Eye Tracking as an Innovative Instructional Method in Surgical Human Anatomy

OBJECTIVE

Tobii glasses can record corneal infrared light reflection to track pupil position and to map gaze focusing in the video recording. Eye tracking has been proposed for use in training and coaching as a visually guided control interface. The aim of our study was to test the potential use of these glasses in various situations: explanations of anatomical structures on tablet-type electronic devices, explanations of anatomical models and dissected cadavers, and during the prosection thereof. An additional aim of the study was to test the use of the glasses during laparoscopies performed on Thiel-embalmed cadavers (that allows pneumoinsufflation and exact reproduction of the laparoscopic surgical technique). The device was also tried out in actual surgery (both laparoscopy and open surgery).

DESIGN

We performed a pilot study using the Tobii glasses.

SETTING

Dissection room at our School of Medicine and in the operating room at our Hospital.

PARTICIPANTS

To evaluate usefulness, a survey was designed for use among students, instructors, and practicing physicians.

RESULTS

The results were satisfactory, with the usefulness of this tool supported by more than 80% positive responses to most questions. There was no inconvenience for surgeons and that patient safety was ensured in the real laparoscopy.

CONCLUSION

To our knowledge, this is the first publication to demonstrate the usefulness of eye tracking in practical instruction of human anatomy, as well as in teaching clinical anatomy and surgical techniques in the dissection and operating rooms.

Identifying error types in visual diagnostic skill assessment

BACKGROUND

Misinterpretation of medical images is an important source of diagnostic error. Errors can occur in different phases of the diagnostic process. Insight in the error types made by learners is crucial for training and giving effective feedback. Most diagnostic skill tests however penalize diagnostic mistakes without an eye for the diagnostic process and the type of error. A radiology test with stepwise reasoning questions was used to distinguish error types in the visual diagnostic process. We evaluated the additional value of a stepwise question-format, in comparison with only diagnostic questions in radiology tests.

METHODS

Medical students in a radiology elective (n=109) took a radiology test including 11-13 cases in stepwise question-format: marking an abnormality, describing the abnormality and giving a diagnosis. Errors were coded by two independent researchers as perception, analysis, diagnosis, or undefined. Erroneous cases were further evaluated for the presence of latent errors or partial knowledge. Inter-rater reliabilities and percentages of cases with latent errors and partial knowledge were calculated.

RESULTS

The stepwise question-format procedure applied to 1351 cases completed by 109 medical students revealed 828 errors. Mean inter-rater reliability of error type coding was Cohen's κ=0.79. Six hundred and fifty errors (79%) could be coded as perception, analysis or diagnosis errors. The stepwise question-format revealed latent errors in 9% and partial knowledge in 18% of cases.

CONCLUSIONS

A stepwise question-format can reliably distinguish error types in the visual diagnostic process, and reveals latent errors and partial knowledge.