An analysis of perceptual errors in reading mammograms using quasi-local spatial frequency spectra

Ultrasonography-Based Qualitative and Quantitative Evaluation Approaches for Pompe Disease

Purpose

The aim of this study was to propose the qualitative and quantitative approaches to evaluate the skeletal muscle ultrasound images of 23 Pompe disease (i.e., acid maltase deficiency, AMD) patients and 14 normal subjects.

Methods

A cohort of 23 AMD patients and 14 normal subjects has been investigated. We compared the B-mode echo intensity of the rectus femoris muscle with that of its surrounding fat (subcutaneous fat) and proposed a qualitative grading method. Quantitative analysis of the region of interest (ROI) with the echo intensity and the segmented area was also performed.

Results

Qualitative results showed that AMD patients without clinical symptoms (without undergoing ERT) had the highest distribution of Grade 1, and AMD patients undergoing ERT had the widest distribution of Grade 2, and control group (n = 14) with the highest distribution of Grade 1. Using the segmented area approach, quantitative results showed that AMD patients undergoing ERT had the largest and widest distribution. Meanwhile the control subjects (normal subjects) had the lowest and the narrowest areas. The echo intensity of the segmented ROI of AMD patients undergoing ERT displayed the highest and widest (inhomogeneous) distributions. By contrast, the echo intensity of AMD patients without clinical symptoms was slightly increased and with low inhomogeneity.

Conclusion

The proposed ultrasonography-based qualitative and quantitative approach may be used to evaluate the severity of muscle destruction for AMD patients. Besides, the quantitative segmented area with regression analysis could help predict the incidence of onset of Pompe disease patients.

[Measurements errors in screening mammogram interpretation by radiologists]

The timely detection of breast cancer is achieved through mammography; however, the quality of the procedure should be addressed for proper performance and interpretation. Despite recent improvements in quality assurance in mammography, interpretation still depends on each reader; therefore, errors can be made when interpreting screening mammograms, leading to unnecessary biopsies and/or overdiagnosis, with sustained physical, economic and psychological consequences. Since interpretation is related to the perceptive and cognitive ability of the radiologist, it is necessary to have extensive knowledge about the possible errors that may occur during interpretation, as well as of the way how they can be reduced, prevented and/or corrected to provide the patient with the highest possible level of safety.

The Impact of Interruptions on Chest Radiograph Interpretation: Effects on Reading Time and Accuracy

RATIONALE AND OBJECTIVES

The objective of this study was to experimentally test the effect of interruptions on image interpretation by comparing reading time and response accuracy of interrupted case reads to uninterrupted case reads in resident and attending radiologists.

MATERIALS AND METHODS

Institutional review board approval was obtained before participant recruitment from an urban academic health-care system during January 2016-March 2016. Eleven resident and 12 attending radiologists examined 30 chest radiographs, rating their confidence regarding the presence or the absence of a pneumothorax. Ten cases were normal (ie, no pneumothorax present), 10 cases had an unsubtle pneumothorax (ie, readily perceivable by a nonexpert), and 10 cases had a subtle pneumothorax. During three reads of each case type, the participants were interrupted with 30 seconds of a secondary task. The total reading time and the accuracy of interrupted and uninterrupted cases were compared. A mixed-factors analysis of variance was run on reading time and accuracy with experience (resident vs attending) as a between-subjects factor and case type (normal, unsubtle, or subtle) and interruption (interruption vs no interruption) as within-subjects factors.

RESULTS

Interrupted tasks had significantly longer reading times than uninterrupted cases (P = .032). During subtle cases, interruptions reduced accuracy (P = .034), but during normal cases, interruptions increased accuracy (P = .038).

CONCLUSIONS

Interruptions increased reading times and increased the tendency for a radiologist to conclude that a case is normal for both resident and attending radiologists, demonstrating that interruptions reduce efficiency and introduce patient safety concerns during reads of abnormal cases.

Review of prospects and challenges of eye tracking in volumetric imaging

While eye tracking research in conventional radiography has flourished over the past decades, the number of eye tracking studies that looked at multislice images lags behind. A possible reason for the lack of studies in this area might be that the eye tracking methodology used in the context of conventional radiography cannot be applied one-on-one to volumetric imaging material. Challenges associated with eye tracking in volumetric imaging are particularly associated with the selection of stimulus material, the detection of events in the eye tracking data, the calculation of meaningful eye tracking parameters, and the reporting of abnormalities. However, all of these challenges can be addressed in the design of the experiment. If this is done, eye tracking studies using volumetric imaging material offer almost unlimited opportunity for perception research and are highly relevant as the number of volumetric images that are acquired and interpreted is rising.

Scanners and drillers: characterizing expert visual search through volumetric images

Modern imaging methods like computed tomography (CT) generate 3-D volumes of image data. How do radiologists search through such images? Are certain strategies more efficient? Although there is a large literature devoted to understanding search in 2-D, relatively little is known about search in volumetric space. In recent years, with the ever-increasing popularity of volumetric medical imaging, this question has taken on increased importance as we try to understand, and ultimately reduce, errors in diagnostic radiology. In the current study, we asked 24 radiologists to search chest CTs for lung nodules that could indicate lung cancer. To search, radiologists scrolled up and down through a "stack" of 2-D chest CT "slices." At each moment, we tracked eye movements in the 2-D image plane and coregistered eye position with the current slice. We used these data to create a 3-D representation of the eye movements through the image volume. Radiologists tended to follow one of two dominant search strategies: "drilling" and "scanning." Drillers restrict eye movements to a small region of the lung while quickly scrolling through depth. Scanners move more slowly through depth and search an entire level of the lung before moving on to the next level in depth. Driller performance was superior to the scanners on a variety of metrics, including lung nodule detection rate, percentage of the lung covered, and the percentage of search errors where a nodule was never fixated.

Investigating the link between radiologists' gaze, diagnostic decision, and image content

OBJECTIVE

To investigate machine learning for linking image content, human perception, cognition, and error in the diagnostic interpretation of mammograms.

METHODS

Gaze data and diagnostic decisions were collected from three breast imaging radiologists and three radiology residents who reviewed 20 screening mammograms while wearing a head-mounted eye-tracker. Image analysis was performed in mammographic regions that attracted radiologists' attention and in all abnormal regions. Machine learning algorithms were investigated to develop predictive models that link: (i) image content with gaze, (ii) image content and gaze with cognition, and (iii) image content, gaze, and cognition with diagnostic error. Both group-based and individualized models were explored.

RESULTS

By pooling the data from all readers, machine learning produced highly accurate predictive models linking image content, gaze, and cognition. Potential linking of those with diagnostic error was also supported to some extent. Merging readers' gaze metrics and cognitive opinions with computer-extracted image features identified 59% of the readers' diagnostic errors while confirming 97.3% of their correct diagnoses. The readers' individual perceptual and cognitive behaviors could be adequately predicted by modeling the behavior of others. However, personalized tuning was in many cases beneficial for capturing more accurately individual behavior.

CONCLUSIONS

There is clearly an interaction between radiologists' gaze, diagnostic decision, and image content which can be modeled with machine learning algorithms.

Association between time spent interpreting, level of confidence, and accuracy of screening mammography

OBJECTIVE

The objective of this study was to examine the effect of time spent viewing images and level of confidence on a screening mammography test set on interpretive performance.

MATERIALS AND METHODS

Radiologists from six mammography registries participated in this study and were randomized to interpret one of four test sets and complete 12 survey questions. Each test set had 109 cases of digitized four-view screening screen-film mammograms with prior comparison screening views. Viewing time for each case was defined as the cumulative time spent viewing all mammographic images before recording which visible feature, if any, was the "most significant finding." Log-linear regression fit via the generalized estimating equation was used to test the effect of viewing time and level of confidence in the interpretation on test set sensitivity and false-positive rate.

RESULTS

One hundred nineteen radiologists completed a test set and contributed data on 11,484 interpretations. The radiologists spent more time viewing cases that had significant findings or cases for which they had less confidence in their interpretation. Each additional minute of viewing time increased the probability of a true-positive interpretation among cancer cases by 1.12 (95% CI, 1.06-1.19; p < 0.001) regardless of confidence in the assessment. Among the radiologists who were very confident in their assessment, each additional minute of viewing time increased the adjusted risk of a false-positive interpretation among noncancer cases by 1.42 (95% CI, 1.21-1.68), and this viewing-time effect diminished with decreasing confidence.

CONCLUSION

Longer interpretation times and higher levels of confidence in an interpretation are both associated with higher sensitivity and false-positive rates in mammography screening.

Skeletal muscle ultrasonography: Visual versus quantitative evaluation

In this study, we compared the sensitivity and specificity of visual versus quantitative evaluation of skeletal muscle ultrasound in children suspected of having a neuromuscular disorder (NMD). Ultrasonography (US) scans of four muscles (biceps brachii, forearm flexors, quadriceps femoris, anterior tibial muscle) were made in 76 children. All images were visually evaluated using the Heckmatt criteria and quantitatively evaluated with computer-assisted grey-scale analysis of muscle echo intensity. Visual evaluation could achieve a sensitivity up to 71%, with a specificity of 92%. With quantification, a sensitivity of 87% accompanied by a specificity of 67% was found, but other diagnostic values could be achieved, depending on the cut-off point. Quantification resulted in a higher interobserver agreement (kappa 0.86) compared with visual evaluation (kappa 0.53). We conclude that quantification of echo intensity is a more objective and accurate method. Because it can achieve higher sensitivities, it is better-suited for the screening task in the diagnostic phase of children with a NMD.

Visual search of mammographic images: influence of lesion subtlety

RATIONALE AND OBJECTIVES

The goal of this study is to determine whether mammographic lesion subtlety influences the detection accuracy and visual search parameters of radiologists.

MATERIALS AND METHODS

Six radiologists searched a set of 20 mammograms with at least two lesions per image (masses and/or microcalcifications). Eye position was recorded. False-negative (FN) and true-positive (TP) decisions were correlated with lesion subtlety and visual search parameters of time to first hit, total dwell time, and number of return fixations.

RESULTS

Lesions with lower subtlety ratings were detected later in the search than more obvious ones (FN later than TP decisions). When subtler lesions were detected (TP), dwell time was longer than for more obvious lesions, but FN decisions received shorter total dwell. Subtler lesions, when detected (TP), received more total fixation clusters than more obvious ones, but FN decisions received fewer.

CONCLUSION

Subtle mammographic lesions are associated with significantly different visual search parameters than obvious lesions, explaining in part why they are missed more often. Developing tools to improve lesion visibility may improve detection.

A preliminary report on the role of spatial frequency analysis in the perception of breast cancers missed at mammography screening

RATIONALE AND OBJECTIVES

Because several factors are involved in cancer detection, a malignant lesion that is visible on a mammogram will not necessarily be reported by the radiologist reading the case. Indeed, a significant fraction of screening-detected cancers are visible in retrospect, and were perceived by the radiologist when the case was read, but were either reported as benign findings or dismissed as variations of normal breast tissue. In this preliminary report the spatial frequency characteristics of clinically missed lesions are investigated by analyzing the mammogram acquired when the lesion was sent for biopsy and the most recent prior mammogram. For control purposes, the contralateral breast is also analyzed, when this breast is lesion free.

MATERIALS AND METHODS

A database of 70 mammogram cases was assembled. Each case contained eight films: craniocaudal (CC) and mediolateral oblique (MLO) of the breast where a biopsy-proven lesion was found, CC and MLO of the contralateral breast, and CC and MLO of both breasts in the most recent prior mammogram. The dictated reports for all of these cases were obtained. Both benign and malignant lesions were used. The films were digitized and an region of interest surrounding each lesion was segmented from the image for processing using wavelet packets to extract spatial frequency information. The corresponding area was also segmented from the prior mammogram and from the contralateral breast, when this breast was lesion-free. Analysis of variance was used to determine if statistically significant differences existed between the derived features of cancer in the current and prior mammograms.

RESULTS

The data suggests that malignant lesions reported in the prior mammogram as being benign differed from correctly reported malignant lesions and from correctly reported benign lesions. They also differed from nonreported malignant lesions. In addition, the spatial frequency representation of cancer significantly differed in the current and prior cases from the representation of normal breast tissue.

CONCLUSION

Spatial frequency analysis may be useful to differentiate malignant lesions that are reported as benign and correctly reported benign lesions.

The perception of breast cancers--a spatial frequency analysis of what differentiates missed from reported cancers

The primary detector of breast cancer is the human eye. Radiologists read mammograms by mapping exogenous and endogenous factors, which are based on the image and observer, respectively, into observer-based decisions. These decisions rely on an internal schema that contains a representation of possible malignant and benign findings. Thus, to understand the hits and misses made by the radiologists, it is important to model the interactions between the measurable image-based elements contained in the mammogram and the decisions made. The image-based elements can be of two types, i.e., areas that attracted the visual attention of the radiologist, but did not yield a report, and areas where the radiologist indicated the presence of an abnormal finding. In this way, overt and covert decisions are made when reading a mammogram. In order to model this decision-making process, we use a system that is based upon the processing done by the human visual system, which decomposes the areas under scrutiny in elements of different sizes and orientations. In our system, this decomposition is done using wavelet packets (WPs). Nonlinear features are then extracted from the WP coefficients, and an artificial neural network is trained to recognize the patterns of decisions made by each radiologist. Afterwards, the system is used to predict how the radiologist will respond to visually selected areas in new mammogram cases.

Perception of breast cancer: eye-position analysis of mammogram interpretation

RATIONALE AND OBJECTIVE

The author performed this study to determine how image-based elements are translated into decisions by radiologists with different levels of experience in the reading of mammograms.

MATERIALS AND METHODS

Three full-time mammographers and four radiology residents read 40 two-view mammogram cases. The observers' eye position was tracked while they searched the mammograms for malignancies. Spatial frequency analysis was performed to relate what the observers reported with where they looked.

RESULTS

Statistically significant differences were found between lesion-containing areas that attracted visual attention and were correctly interpreted and those that were visually inspected but not reported. In addition, an artificial neural network was successfully trained to map the image characteristics in the visually selected areas on a mammogram and to linkthem to a likely decision by the observer.

CONCLUSION

Spatial frequency analysis can be used to derive trends for how mammographers and radiology residents will respond to mammograms.

The perception of breast cancer: what differentiates missed from reported cancers in mammography?

RATIONALE AND OBJECTIVES

Mammographers map endogenous and exogenous factors into decisions whether to report the presence of a malignant finding in a mammogram case. Thus, to understand how image-based elements are translated into observer-based decisions, the authors used spatial frequency analysis to model the areas on mammograms that attracted visual attention, in addition to the areas localized as abnormal.

MATERIALS AND METHODS

Four mammographers read 40 two-view mammogram cases, of which 30 contained at least one malignant lesion visible on one or two views. Their eye positions were recorded during visual search. Once the mammographer felt confident enough to provide an initial impression of the case ("normal" or "abnormal"), the eye position monitoring was turned off and the mammographer indicated, with a mouse-controlled cursor, the location and nature of any malignant findings. Regions that elicited an overt or a covert response by the mammographers were extracted for processing by means of wavelet packets and artificial neural networks.

RESULTS

Different decision outcomes yielded different energy representations, in the spatial frequency domain. These energy representations were used by an artificial neural network to predict decision outcome in areas of interest, derived from eye position analysis, on mammograms from new cases. Individual trends were observed for each mammographer.

CONCLUSION

Spatial frequency representation of regions that attracted a given mammographer's visual attention may be useful for characterizing how that mammographer will respond to the visually selected areas.

Machine classification of dental images with visual search

RATIONALE AND OBJECTIVES

The authors performed this study to assess the performance of a computer-based classification system that uses gaze locations of observers to define the subspace for machine learning.

MATERIALS AND METHODS

Thirty-two dental radiographs were classified by an expert viewer into four categories of disease of the periapical region: no disease (normal tooth), mild disease (widened periodontal ligament space), moderate disease (destruction of the lamina dura), and severe disease (resorption of bone in the periapical area). There were eight images in each category. Six observers independently viewed the images while their eye gaze position was recorded. They then classified the images into one of the four categories. A sample of image space was used as input to a machine learning routine to develop a machine classifier. Sample space was determined with three techniques: visual gaze, random selection, and constrained random selection. K analyses were used to compare classification accuracies with the three sampling techniques.

RESULTS

With use of the expert classification as a standard of reference, observers classified images with 57% accuracy, and the machine classified images with 84% accuracy by using the same gaze-selected features and image space. Results of kappa analyses revealed mean values of 0.78 for gaze-selected sampling, 0.69 for random sampling, 0.68 for constrained random selection, and 0.44 for observers. The use of sample space selected with the visual gaze technique was superior to that selected with both random-selection techniques and by the observers.

CONCLUSION

Machine classification of dental images improves the accuracy of individual observers using gaze-selected image space.