Searching for nodules: what features attract attention and influence detection?

Modeling Eye Movements During Decision Making: A Review

This article reviews recent advances in the psychometric and econometric modeling of eye-movements during decision making. Eye movements offer a unique window on unobserved perceptual, cognitive, and evaluative processes of people who are engaged in decision making tasks. They provide new insights into these processes, which are not easily available otherwise, allow for explanations of fundamental search and choice phenomena, and enable predictions of future decisions. We propose a theoretical framework of the search and choice tasks that people commonly engage in and of the underlying cognitive processes involved in those tasks. We discuss how these processes drive specific eye-movement patterns. Our framework emphasizes the central role of task and strategy switching for complex goal attainment. We place the extant literature within that framework, highlight recent advances in modeling eye-movement behaviors during search and choice, discuss limitations, challenges, and open problems. An agenda for further psychometric modeling of eye movements during decision making concludes the review.

The cognitive and perceptual processes that affect observer performance in lung cancer detection: a scoping review

INTRODUCTION

Early detection of malignant pulmonary nodules through screening has been shown to reduce lung cancer-related mortality by 20%. However, perceptual and cognitive factors that affect nodule detection are poorly understood. This review examines the cognitive and visual processes of various observers, with a particular focus on radiologists, during lung nodule detection.

METHODS

Four databases (Medline, Embase, Scopus and PubMed) were searched to extract studies on eye-tracking in pulmonary nodule detection. Studies were included if they used eye-tracking to assess the search and detection of lung nodules in computed tomography or 2D radiographic imaging. Data were charted according to identified themes and synthesised using a thematic narrative approach.

RESULTS

The literature search yielded 25 articles and five themes were discovered: 1 - functional visual field and satisfaction of search, 2 - expert search patterns, 3 - error classification through dwell time, 4 - the impact of the viewing environment and 5 - the effect of prevalence expectation on search. Functional visual field reduced to 2.7° in 3D imaging compared to 5° in 2D radiographs. Although greater visual coverage improved nodule detection, incomplete search was not responsible for missed nodules. Most radiological errors during lung nodule detection were decision-making errors (30%-45%). Dwell times associated with false-positive (FP) decisions informed feedback systems to improve diagnosis. Interruptions did not influence diagnostic performance; however, it increased viewing time by 8% and produced a 23.1% search continuation accuracy. Comparative scanning was found to increase the detection of low contrast nodules. Prevalence expectation did not directly affect diagnostic accuracy; however, decision-making time increased by 2.32 seconds with high prevalence expectations.

CONCLUSION

Visual and cognitive factors influence pulmonary nodule detection. Insights gained from eye-tracking can inform advancements in lung screening. Further exploration of eye-tracking in lung screening, particularly with low-dose computed tomography (LDCT), will benefit the future of lung cancer screening.

What do radiologists look for? Advances and limitations of perceptual learning in radiologic search

Supported by guidance from training during residency programs, radiologists learn clinically relevant visual features by viewing thousands of medical images. Yet the precise visual features that expert radiologists use in their clinical practice remain unknown. Identifying such features would allow the development of perceptual learning training methods targeted to the optimization of radiology training and the reduction of medical error. Here we review attempts to bridge current gaps in understanding with a focus on computational saliency models that characterize and predict gaze behavior in radiologists. There have been great strides toward the accurate prediction of relevant medical information within images, thereby facilitating the development of novel computer-aided detection and diagnostic tools. In some cases, computational models have achieved equivalent sensitivity to that of radiologists, suggesting that we may be close to identifying the underlying visual representations that radiologists use. However, because the relevant bottom-up features vary across task context and imaging modalities, it will also be necessary to identify relevant top-down factors before perceptual expertise in radiology can be fully understood. Progress along these dimensions will improve the tools available for educating new generations of radiologists, and aid in the detection of medically relevant information, ultimately improving patient health.

What do we know about volumetric medical image interpretation?: a review of the basic science and medical image perception literatures

Interpretation of volumetric medical images represents a rapidly growing proportion of the workload in radiology. However, relatively little is known about the strategies that best guide search behavior when looking for abnormalities in volumetric images. Although there is extensive literature on two-dimensional medical image perception, it is an open question whether the conclusions drawn from these images can be generalized to volumetric images. Importantly, volumetric images have distinct characteristics (e.g., scrolling through depth, smooth-pursuit eye-movements, motion onset cues, etc.) that should be considered in future research. In this manuscript, we will review the literature on medical image perception and discuss relevant findings from basic science that can be used to generate predictions about expertise in volumetric image interpretation. By better understanding search through volumetric images, we may be able to identify common sources of error, characterize the optimal strategies for searching through depth, or develop new training and assessment techniques for radiology residents.

Cognitive processing differences of experts and novices when correlating anatomy and cross-sectional imaging

The ability to correlate anatomical knowledge and medical imaging is crucial to radiology and as such, should be a critical component of medical education. However, we are hindered in our ability to teach this skill because we know very little about what expert practice looks like, and even less about novices' understanding. Using a unique simulation tool, this research conducted cognitive clinical interviews with experts and novices to explore differences in how they engage in this correlation and the underlying cognitive processes involved in doing so. This research supported what has been known in the literature, that experts are significantly faster at making decisions on medical imaging than novices. It also offers insight into the spatial ability and reasoning that is involved in the correlation of anatomy to medical imaging. There are differences in the cognitive processing of experts and novices with respect to meaningful patterns, organized content knowledge, and the flexibility of retrieval. Presented are some novice-expert similarities and differences in image processing. This study investigated extremes, opening an opportunity to investigate the sequential knowledge acquisition from student to resident to expert, and where educators can help intervene in this learning process.

Visual adaptation and the amplitude spectra of radiological images

We examined how visual sensitivity and perception are affected by adaptation to the characteristic amplitude spectra of X-ray mammography images. Because of the transmissive nature of X-ray photons, these images have relatively more low-frequency variability than natural images, a difference that is captured by a steeper slope of the amplitude spectrum (~ − 1.5) compared to the ~ 1/f (slope of − 1) spectra common to natural scenes. Radiologists inspecting these images are therefore exposed to a different balance of spectral components, and we measured how this exposure might alter spatial vision. Observers (who were not radiologists) were adapted to images of normal mammograms or the same images sharpened by filtering the amplitude spectra to shallower slopes. Prior adaptation to the original mammograms significantly biased judgments of image focus relative to the sharpened images, demonstrating that the images are sufficient to induce substantial after-effects. The adaptation also induced strong losses in threshold contrast sensitivity that were selective for lower spatial frequencies, though these losses were very similar to the threshold changes induced by the sharpened images. Visual search for targets (Gaussian blobs) added to the images was also not differentially affected by adaptation to the original or sharper images. These results complement our previous studies examining how observers adapt to the textural properties or phase spectra of mammograms. Like the phase spectrum, adaptation to the amplitude spectrum of mammograms alters spatial sensitivity and visual judgments about the images. However, unlike the phase spectrum, adaptation to the amplitude spectra did not confer a selective performance advantage relative to more natural spectra.

Analyzing visual-search observers using eye-tracking data for digital breast tomosynthesis images

Visual-search (VS) model observers have the potential to provide reliable predictions of human-observer performance in detection-localization tasks. The purpose of this work was to examine some characteristics of human gaze on breast images with the goal of informing the design of our VS observers. Using a helmet-mounted eye-tracking system, we recorded the movement of gaze from human observers as they searched for masses in sets of 2D digital breast tomosynthesis (DBT) images. The masses in this study were of a single profile. The DBT images were extracted from image volumes reconstructed with the filtered backprojection method. Fixation times associated with observer points of interest were computed from the observer data. We used the k-mean clustering algorithm to get dwell times of gaze data. The dwell times were then compared to sets of morphological feature values extracted from the images. These features, extracted as cross correlations involving the mass profile and the test image, included the matched filter (MF), gradient MF, Laplacian MF, and adaptive MF. The adaptive MF combining four feature maps was computed using a hotelling discriminant generated from training data. For this investigation, we computed correlation coefficients between the fixation times and the feature values. We also conducted a significance test by computing p-values of correlation coefficients for five features. Of all these features, the adaptive MF provided the highest correlation coefficients for DBT images with different densities.

Variations in the functional visual field for detection of lung nodules on chest computed tomography: Impact of nodule size, distance, and local lung complexity

PURPOSE

To explore the characteristics that impact lung nodule detection by peripheral vision when searching for lung nodules on chest CT-scans.

METHODS

This study was approved by the local IRB and is HIPAA compliant. A simulated primary (1°) target mass (2 × 2 × 5 cm) was embedded into 5 cm thick subvolumes (SV) extracted from three unenhanced lung MDCT scans (64 row, 1.25 mm thickness, 0.7 mm increment). One of 30 solid, secondary nodules with either 3-4 mm and 5-8 mm diameters were embedded into 192 of 207 SVs. The secondary nodule was placed at a random depth within each SV, a transverse distance of 2.5, 5, 7.5, or 10 mm, and along one of eight rays cast every 45° from the center of the 1° mass. Video recordings of transverse paging in cranio-caudal direction were created for each SV (frame rate three sections/sec). Six radiologists observed each cine-loop once while gaze-tracking hardware assured that gaze was centered on the 1° mass. Each radiologist assigned a confidence rating (0-5) to the detection of a secondary nodule and indicated its location. Detection sensitivity was analyzed relative to secondary nodule size, transverse distance, radial orientation, and lung complexity. Lung complexity was characterized by the number of particles (connected pixels) and the sum of the area of all particles above a -500 HU threshold within regions of interest around the 1° mass and secondary nodule.

RESULTS

Using a proportional odds logistic regression model and eliminating redundant predictors, models fit individually to each reader resulted in the following decreasing order of association based on greatest reduction in Akaike Information Criterion: secondary nodule diameter (6/6 readers, P < 0.001), distance from central mass (6/6 readers, P < 0.001), lung complexity particle count (5/6 readers, P = 0.05), and lung complexity particle area (3/6 readers, P = 0.03). Substantial inter-reader differences in sensitivity to decreasing nodule diameter, distance, and complexity characteristics were observed.

CONCLUSIONS

Of the investigated parameters, secondary nodule size, distance from the gaze center and lung complexity (particle number and area) significantly impact nodule detection with peripheral vision.

Review of prospects and challenges of eye tracking in volumetric imaging

While eye tracking research in conventional radiography has flourished over the past decades, the number of eye tracking studies that looked at multislice images lags behind. A possible reason for the lack of studies in this area might be that the eye tracking methodology used in the context of conventional radiography cannot be applied one-on-one to volumetric imaging material. Challenges associated with eye tracking in volumetric imaging are particularly associated with the selection of stimulus material, the detection of events in the eye tracking data, the calculation of meaningful eye tracking parameters, and the reporting of abnormalities. However, all of these challenges can be addressed in the design of the experiment. If this is done, eye tracking studies using volumetric imaging material offer almost unlimited opportunity for perception research and are highly relevant as the number of volumetric images that are acquired and interpreted is rising.

Anatomical landmark position--can we trust what we see? Results from an online reliability and validity study of osteopaths

BACKGROUND

Practitioners traditionally use observation to classify the position of patients' anatomical landmarks. This information may contribute to diagnosis and patient management.

OBJECTIVES

To calculate a) Inter-rater reliability of categorising the sagittal plane position of four anatomical landmarks (lateral femoral epicondyle, greater trochanter, mastoid process and acromion) on side-view photographs (with landmarks highlighted and not-highlighted) of anonymised subjects; b) Intra-rater reliability; c) Individual landmark inter-rater reliability; d) Validity against a 'gold standard' photograph.

DESIGN

Online inter- and intra-rater reliability study.

SUBJECTS

Photographed subjects: convenience sample of asymptomatic students; raters: randomly selected UK registered osteopaths.

METHODS

40 photographs of 30 subjects were used, a priori clinically acceptable reliability was ≥0.4. Inter-rater arm: 20 photographs without landmark highlights plus 10 with highlights; Intra-rater arm: 10 duplicate photographs (non-highlighted landmarks). Validity arm: highlighted landmark scores versus 'gold standard' photographs with vertical line. Research ethics approval obtained.

RATERS

Osteopaths (n = 48) categorised landmark position relative to imagined vertical-line; Gwet's Agreement Coefficient 1 (AC1) calculated and chance-corrected coefficient benchmarked against Landis and Koch's scale; Validity calculation used Kendall's tau-B.

RESULTS

Inter-rater reliability was 'fair' (AC1 = 0.342; 95% confidence interval (CI) = 0.279-0.404) for non-highlighted landmarks and 'moderate' (AC1 = 0.700; 95% CI = 0.596-0.805) for highlighted landmarks. Intra-rater reliability was 'fair' (AC1 = 0.522); range was 'poor' (AC1 = 0.160) to 'substantial' (AC1 = 0.896). No differences were found between individual landmarks. Validity was 'low' (TB = 0.327; p = 0.104).

CONCLUSION

Both inter- and intra-rater reliability was 'fair' but below clinically acceptable levels, validity was 'low'. Together these results challenge the clinical practice of using observation to categorise anterio-posterior landmark position.

Temporal sequences quantify the contributions of individual fixations in complex perceptual matching tasks

Perceptual tasks such as object matching, mammogram interpretation, mental rotation, and satellite imagery change detection often require the assignment of correspondences to fuse information across views. We apply techniques developed for machine translation to the gaze data recorded from a complex perceptual matching task modeled after fingerprint examinations. The gaze data provide temporal sequences that the machine translation algorithm uses to estimate the subjects' assumptions of corresponding regions. Our results show that experts and novices have similar surface behavior, such as the number of fixations made or the duration of fixations. However, the approach applied to data from experts is able to identify more corresponding areas between two prints. The fixations that are associated with clusters that map with high probability to corresponding locations on the other print are likely to have greater utility in a visual matching task. These techniques address a fundamental problem in eye tracking research with perceptual matching tasks: Given that the eyes always point somewhere, which fixations are the most informative and therefore are likely to be relevant for the comparison task?

Accuracy of a remote eye tracker for radiologic observer studies: effects of calibration and recording environment

RATIONALE AND OBJECTIVES

To determine the accuracy and reproducibility of a remote eye-tracking system for studies of observer gaze while displaying volumetric chest computed tomography (CT) images.

MATERIALS AND METHODS

Four participants performed calibrations using three different gray-scale backgrounds (black, gray, and white). Each participant then observed a three-dimensional 10-point test pattern embedded in five Digital Imaging and Communications in Medicine (DICOM) datasets (test backgrounds): a full 190-section chest CT scan, 190 copies of a single chest CT section, and three 190-section datasets of homogeneous intensity (black, gray, and white).

RESULTS

Significant variances between participants, calibration backgrounds, and test backgrounds were observed. The least mean systematic error (deviation of recorded gaze position from target) was obtained when the calibration background and test background were black (27 pixels). Systematic error increased when displaying a test background that deviated from the calibration background intensity. Hence, the largest mean systematic error occurred when calibrating to a black background and displaying a white background (67 pixels). For complex chest CT volumes the white calibration background performed best (38 pixels). An angular analysis of the systematic error was performed and demonstrated that the systemic error primarily affects the vertical position of the estimated gaze position.

CONCLUSION

Our findings indicate a potential source of systematic error during gaze recording in a dynamic environment and highlight the importance of configuring the calibration procedure according to the brightness of the display. We recommend that investigators develop routines for postcalibration accuracy measurement and report the effective accuracy for the display environment in which the data are collected.

Optimizing analysis, visualization, and navigation of large image data sets: one 5000-section CT scan can ruin your whole day

The technology revolution in image acquisition, instrumentation, and methods has resulted in vast data sets that far outstrip the human observers' ability to view, digest, and interpret modern medical images by using traditional methods. This may require a paradigm shift in the radiologic interpretation process. As human observers, radiologists must search for, detect, and interpret targets. Potential interventions should be based on an understanding of human perceptual and attentional abilities and limitations. New technologies and tools already in use in other fields can be adapted to the health care environment to improve medical image analysis, visualization, and navigation through large data sets. This historical psychophysical and technical review touches on a broad range of disciplines but focuses mainly on the analysis, visualization, and navigation of image data performed during the interpretive process. Advanced postprocessing, including three-dimensional image display, multimodality image fusion, quantitative measures, and incorporation of innovative human-machine interfaces, will likely be the future. Successful new paradigms will integrate image and nonimage data, incorporate workflow considerations, and be informed by evidence-based practices. This overview is meant to heighten the awareness of the complexities and limitations of how radiologists interact with images, particularly the large image sets generated today. Also addressed is how human-machine interface and informatics technologies could combine to transform the interpretation process in the future to achieve safer and better quality care for patients and a more efficient and effective work environment for radiologists.

SUPPLEMENTAL MATERIAL

http://radiology.rsna.org/lookup/suppl/doi:10.1148/radiol.11091276/-/DC1.

Conspicuity of colorectal polyps at CT colonography: visual assessment, CAD performance, and the important role of polyp height

RATIONALE AND OBJECTIVES

The factors that influence the conspicuity of polyps on computed tomographic (CT) colonography (CTC) are poorly understood. The aim of this study is to compare radiologists' visual assessment of polyp conspicuity to quantitative image features and show the relationship between visual conspicuity and the detection of colonic polyps by computer-aided detection (CAD) on CTC.

METHODS

One polyp (size range 6-10 mm) was selected from the CTC examination of each of 29 patients from a larger cohort. All patients underwent oral contrast-enhanced CTC with same-day optical colonoscopy with segmental unblinding. The polyps were analyzed by a previously validated CAD system and placed into one of two groups (detected [n = 12] or not detected [n = 17] by CAD). The study population was intentionally enriched with polyps that were not detected by the CAD system. Four board-certified radiologists, blinded to the CAD results, reviewed two- and three-dimensional CTC images of the polyps and scored the conspicuity of the polyps using a 4-point scale (0 = least conspicuous, 3 = most conspicuous). Polyp height and width were measured by a trained observer. A t-test (two-tailed, unpaired equal variance) was done to determine statistical significance. Intra- and interobserver variabilities of the conspicuity scores were assessed using the weighted kappa test. Regression analysis was used to investigate the relationship of conspicuity to polyp height and width.

RESULTS

A statistically significant difference was found between the average conspicuity scores for polyps that were detected by CAD compared to those that were not (2.3 +/- 0.6 vs. 1.4 +/- 0.8) (P = .004). There was moderate intraobserver agreement of the conspicuity scores (weighted kappa 0.57 +/- 0.09). Interobserver agreement was fair (average weighted kappa for six pair-wise comparisons, 0.38 +/- 0.15). Conspicuity was correlated with manual measurement of polyp height (r(2) = 0.38-0.56, P < .001).

CONCLUSIONS

This CAD system tends to detect 6-10 mm polyps that are more visually conspicuous. Polyp height is a major determinant of visual conspicuity. The generalizability of these findings to other CAD systems is currently unknown. Nevertheless, CAD developers may need to specifically target flatter and less conspicuous polyps for CAD to better assist the radiologist to find polyps in this clinically important size category.

Characterization of radiologists' search strategies for lung nodule detection: slice-based versus volumetric displays

The goal of this study was to assess whether radiologists' search paths for lung nodule detection in chest computed tomography (CT) between different rendering and display schemes have reliable properties that can be exploited as an indicator of ergonomic efficiency for the purpose of comparing different display paradigms. Eight radiologists retrospectively viewed 30 lung cancer screening CT exams, containing a total of 91 nodules, in each of three display modes [i.e., slice-by-slice, orthogonal maximum intensity projection (MIP) and stereoscopic] for the purpose of detecting and classifying lung nodules. Radiologists' search patterns in the axial direction were recorded and analyzed along with the location, size, and shape for each detected feature, and the likelihood that the feature is an actual nodule. Nodule detection performance was analyzed by employing free-response receiver operating characteristic methods. Search paths were clearly different between slice-by-slice displays and volumetric displays but, aside from training and novelty effects, not between MIP and stereographic displays. Novelty and training effects were associated with the stereographic display mode, as evidenced by differences between the beginning and end of the study. The stereo display provided higher detection and classification performance with less interpretation time compared to other display modes tested in the study; however, the differences were not statistically significant. Our preliminary results indicate a potential role for the use of radiologists' search paths in evaluating the relative ergonomic efficiencies of different display paradigms, but systematic training and practice is necessary to eliminate training curve and novelty effects before search strategies can be meaningfully compared.

Visual search of mammographic images: influence of lesion subtlety

RATIONALE AND OBJECTIVES

The goal of this study is to determine whether mammographic lesion subtlety influences the detection accuracy and visual search parameters of radiologists.

MATERIALS AND METHODS

Six radiologists searched a set of 20 mammograms with at least two lesions per image (masses and/or microcalcifications). Eye position was recorded. False-negative (FN) and true-positive (TP) decisions were correlated with lesion subtlety and visual search parameters of time to first hit, total dwell time, and number of return fixations.

RESULTS

Lesions with lower subtlety ratings were detected later in the search than more obvious ones (FN later than TP decisions). When subtler lesions were detected (TP), dwell time was longer than for more obvious lesions, but FN decisions received shorter total dwell. Subtler lesions, when detected (TP), received more total fixation clusters than more obvious ones, but FN decisions received fewer.

CONCLUSION

Subtle mammographic lesions are associated with significantly different visual search parameters than obvious lesions, explaining in part why they are missed more often. Developing tools to improve lesion visibility may improve detection.