Patterns of saliency and semantic features distinguish gaze of expert and novice viewers of surveillance footage

When viewing the actions of others, we not only see patterns of body movements, but we also "see" the intentions and social relations of people. Experienced forensic examiners - Closed Circuit Television (CCTV) operators - have been shown to convey superior performance in identifying and predicting hostile intentions from surveillance footage than novices. However, it remains largely unknown what visual content CCTV operators actively attend to, and whether CCTV operators develop different strategies for active information seeking from what novices do. Here, we conducted computational analysis for the gaze-centered stimuli captured by experienced CCTV operators and novices' eye movements when viewing the same surveillance footage. Low-level image features were extracted by a visual saliency model, whereas object-level semantic features were extracted by a deep convolutional neural network (DCNN), AlexNet, from gaze-centered regions. We found that the looking behavior of CCTV operators differs from novices by actively attending to visual contents with different patterns of saliency and semantic features. Expertise in selectively utilizing informative features at different levels of visual hierarchy may play an important role in facilitating the efficient detection of social relationships between agents and the prediction of harmful intentions.

Non-monotonic developmental trend of holistic processing in visual expertise: the case of Chinese character recognition

Holistic processing has been identified as an expertise marker of face and object recognition. By contrast, reduced holistic processing is purportedly an expertise marker in recognising orthographic characters in Chinese. Does holistic processing increase or decrease in expertise development? Is orthographic recognition a domain-specific exception to all other kinds of recognition (e.g. face and objects)? In two studies, we examined the developmental trend of holistic processing in Chinese character recognition in Chinese and non-Chinese children, and its relationship with literacy abilities: Chinese first graders—with emergent Chinese literacy acquired in kindergarten—showed increased holistic processing perhaps as an inchoate expertise marker when compared with kindergartners and non-Chinese first graders; however, the holistic processing effect was reduced in higher-grade Chinese children. These results suggest a non-monotonic inverted U-shape trend of holistic processing in visual expertise development: An increase in holistic processing due to initial reading experience followed by a decrease in holistic processing due to literacy enhancement. This result marks the development of holistic and analytic processing skills, both of which can be essential for mastering visual recognition. This study is the first to investigate the developmental trend of holistic processing in Chinese character recognition using the composite paradigm.

Visual expertise modulates baseline brain activity: a preliminary resting-state fMRI study using expertise model of radiologists

Background

visual expertise and experience modulate evoked brain activity in response to training-related stimuli. However, few studies have considered how the visual experience is represented in the resting state brain activity. This study tried to investigate the way visual experience, i.e., visual recognition expertise, modulates baseline brain neuronal activity in the resting state using the model of radiologists.

Methods

The amplitude of low-frequency (< 0.08 Hz) fluctuation (ALFF) was used as the metric of baseline brain activity and a visual expertise model of radiologists to investigated this question. The visual recognition skill enables them to accurately identify pathological information in medical images. After the behavior measurement, a cohort group of radiology interns (n = 22) and a group of matched layperson (n = 22) were selected for inclusion in the study. The resting state functional magnetic resonance imaging (fMRI) scans were performed for all of the subjects.

Results

Higher ALFF in the right fusiform gyrus and the left orbitofrontal cortex were observed, and the ALFF in the fusiform gyrus was correlated with the intern radiologists’ behavioral expertise(all results corrected for multiple comparisons).

Conclusions

Visual experience modulates the baseline brain activity in both high-level visual cortex and high-order cognitive cortex, indicating the engagement of both top-down and bottom-up facilitation. We provide a novel perspective to how visual experience modulated cortical brain activity by introducing the resting state changes. Also, we propose that our current study may provide novel ideas for the development of new training protocols in medical school.

Music-reading expertise associates with face but not Chinese character processing ability

A growing number of behavioural and neuroimaging studies have investigated the cognitive mechanisms and neural substrates underlying various forms of visual expertise, such as face and word processing. However, it remains poorly understood whether and to what extent the acquisition of one form of expertise would be associated with that of another. The current study examined the relationship between music-reading expertise and face and Chinese character processing abilities. In a series of experiments, music experts and novices performed discrimination and recognition tasks of musical notations, faces, and words. Results consistently showed that musical experts responded more accurately to musical notations and faces, but not to words, than did musical novices. More intriguingly, the music expert's age of training onset could well predict their face but not word processing performance: the earlier musical experts began musical notation reading, the better their face processing performance. Taken together, our findings provide preliminary and converging evidence that music-reading expertise links with face, but not word, processing, and lend support to the notion that the development of different types of visual expertise may not be independent, but rather interact with each other during their acquisition.

Domain-specific and domain-general contributions to reading musical notation

Musical practice may benefit not only domain-specific abilities, such as pitch discrimination and music performance, but also domain-general abilities, like executive functioning and memory. Behavioral and neural changes in visual processing have been associated with music-reading experience. However, it is still unclear whether there is a domain-specific visual ability to process musical notation. This study investigates the specificity of the visual skills relevant to simple decisions about musical notation. Ninety-six participants varying in music-reading experience answered a short survey to quantify experience with musical notation and completed a test battery that assessed musical notation reading fluency and accuracy at the level of individual note or note sequence. To characterize how this ability may relate to domain-general abilities, we also estimated general intelligence (as measured with the Raven's Progressive Matrices) and general object-recognition ability (as measure by a recently proposed construct o). We obtained reliable measurements on our various tasks and found evidence for a domain-specific ability of the perception of musical notation. This music-reading ability and domain-general abilities were found to contribute to performance on specific tasks differently, depending on the level of experience reading music.

How to support dental students in reading radiographs: effects of a gaze-based compare-and-contrast intervention

In dental medicine, interpreting radiographs (i.e., orthopantomograms, OPTs) is an error-prone process, even in experts. Effective intervention methods are therefore needed to support students in improving their image reading skills for OPTs. To this end, we developed a compare-and-contrast intervention, which aimed at supporting students in achieving full coverage when visually inspecting OPTs and, consequently, obtaining a better diagnostic performance. The comparison entailed a static eye movement visualization (heat map) on an OPT showing full gaze coverage from a peer-model (other student) and another heat map showing a student's own gaze behavior. The intervention group (N = 38) compared five such heat map combinations, whereas the control group (N = 23) diagnosed five OPTs. Prior to the experimental variation (pre-test) and after it (post-test), students in both conditions searched for anomalies in OPTs while their gaze was recorded. Results showed that students in the intervention group covered more areas of the OPTs and looked less often and for a shorter amount of time at anomalies after the intervention. Furthermore, they fixated on low-prevalence anomalies earlier and high-prevalence anomalies later during the inspection. However, the students in the intervention group did not show any meaningful improvement in detection rate and made more false positive errors compared to the control group. Thus, the intervention guided visual attention but did not improve diagnostic performance substantially. Exploratory analyses indicated that further interventions should teach knowledge about anomalies rather than focusing on full coverage of radiographs.

Human factors in forensic science: The cognitive mechanisms that underlie forensic feature-comparison expertise

After a decade of critique from leading scientific bodies, forensic science research is at a crossroads. Whilst emerging research has shown that some forensic feature-comparison disciplines are not foundationally valid, others are moving towards establishing reliability and validity. Forensic examiners in fingerprint, face and handwriting comparison disciplines have skills and knowledge that distinguish them from novices. Yet our understanding of the basis of this expertise is only beginning to emerge. In this paper, we review evidence on the psychological mechanisms contributing to forensic feature-comparison expertise, with a focus on one mechanism: statistical learning, or the ability to learn how often things occur in the environment. Research is beginning to emphasise the importance of statistical learning in forensic feature-comparison expertise. Ultimately, this research and broader cognitive science research has an important role to play in informing the development of training programs and selection tools for forensic feature-comparison examiners.

Familiarity with visual forms contributes to a left-lateralized and increased N170 response for Chinese characters

While skilled readers produce an increased and left-lateralized event-related-potential (ERP) component, known as N170, for strings of letters compared to strings of less familiar units, it remains unclear whether perceptual familiarity plays an important role in driving the increased and left-lateralized N170 for print. The present study addressed this issue by examining N170 responses for regular Chinese characters and cursive Chinese characters which are visually less familiar regarding their form, yet with phonological and semantic properties. Stroke combinations, which are with unfamiliar visual form and without phonological or semantic properties, were used as low-level control stimuli. Twenty college students (22.6 ± 1.2 years) were examined. A content-irrelevant color matching task was used to control for differences of attention load across familiar and unfamiliar stimuli. A left-lateralized N170 was evoked only by regular characters, but not by cursive characters or stroke combinations. Moreover, cursive characters, which are principally readable but visually unfamiliar, produced a lower N170 than regular characters, and no N170 difference was found compared with stroke combinations. These results suggest that visual form familiarity serves as an important driver for the increased and left-lateralized N170 response.

Reversal of the hanging protocol of Contrast Enhanced Mammography leads to similar diagnostic performance yet decreased reading times

OBJECTIVES

Contrast-enhanced mammography (CEM) was found superior to Full-Field Digital Mammography (FFDM) for breast cancer detection. Current hanging protocols show low-energy (LE, similar to FFDM) images first, followed by recombined (RC) images. However, evidence regarding which hanging protocol leads to the most efficient reading process and highest diagnostic performance is lacking. This study investigates the effects of hanging-protocol ordering on the reading process and diagnostic performance of breast radiologists using eye-tracking methodology. Furthermore, it investigates differences in reading processes and diagnostic performance between LE, RC and FFDM images.

MATERIALS AND METHODS

Twenty-seven breast radiologists were randomized into three reading groups: LE-RC (commonly used hangings), RC-LE (reversed hangings) and FFDM. Thirty cases (nine malignant) were used. Fixation count, net dwell time and time-to-first fixation on malignancies as visual search measures were registered by the eye-tracker. Reading time per image was measured. Participants clicked on suspicious lesions to determine sensitivity and specificity. Area-under-the-ROC-curve (AUC) values were calculated.

RESULTS

RC-LE scored identical on visual search measures, t(16)= -1.45, p =  .17 or higher-p values, decreased reading time with 31%, t(16)= -2.20, p = .04, while scoring similar diagnostic performance compared to LE-RC, t(13.2) = -1.39, p - .20 or higher p-values. The reading process was more efficient on RC compared to LE. Diagnostic performance of CEM was superior to FFDM; F (2,26) = 16.1, p <  .001. Average reading time did not differ between the three groups, F (2,25) = 3.15, p =  .06.

CONCLUSION

The reversed CEM hanging protocol (RC-LE) scored similar on diagnostic performance compared to LE-RC, while reading time was a third faster. Abnormalities were interpreted quicker on RC images. A RC-LE hanging protocol is therefore recommended for clinical practice and training. Diagnostic performance of CEM was (again) superior to FFDM.

Visual expertise for print in schizophrenia: Analysis of the N170 component

Reading deficits have been reported for patients suffering from schizophrenia namely, specific phonological processing deficits. Phonological processing skills are crucial in the learning-to-read process as they are necessary to develop visual expertise for print, which reflects the neural specialization for print. The present study is the first to test visual expertise for print in patients suffering from schizophrenia by measuring the N170 component. Patients and pair-matched healthy control participants performed a lexical decision task, in which words and symbols were presented. As expected, larger N170 amplitudes to word than to control stimuli were observed at the left occipito-temporal site PO7 but not at the PO8. More importantly, the modulation of the N170 as a function of the stimulus and hemisphere did not vary between patients and controls. This result suggests preserved visual expertise for print processing in patients suffering from schizophrenia.

Domain-general and domain-specific neural changes underlying visual expertise

Visual expertise induces changes in neural processing for many different domains of expertise. However, it is unclear how expertise effects for different domains of expertise are related. In the present fMRI study, we combine large-scale univariate and multi-voxel analyses to contrast the expertise-related neural changes associated with two different domains of expertise, bird expertise (ornithology) and mineral expertise (mineralogy). Results indicated distributed expertise-related neural changes, with effects for both domains of expertise in high-level visual cortex and effects for bird expertise even extending to low-level visual regions and the frontal lobe. Importantly, a multivariate generalization analysis showed that effects in high-level visual cortex were specific to the domain of expertise. In contrast, the neural changes in the frontal lobe relating to expertise showed significant generalization, signaling the presence of domain-independent expertise effects. In conclusion, expertise is related to a combination of domain-specific and domain-general changes in neural processing.

A neural marker of medical visual expertise: implications for training

Researchers have identified a component of the EEG that discriminates visual experts from novices. The marker indexes a comprehensive model of visual processing, and if it is apparent in physicians, it could be used to investigate the development and training of their visual expertise. The purpose of this study was to determine whether a neural marker of visual expertise-the enhanced N170 event-related potential-is apparent in the EEGs of physicians as they interpret diagnostic images. We conducted a controlled trial with 10 cardiologists and 9 pulmonologists. Each participant completed 520 trials of a standard visual processing task involving the rapid evaluation of EKGs and CXRs-indicating-lung-disease. Ostensibly, each participant is expert with one type of image and competent with the other. We collected behavioral data on the participants' expertise with EKGs and CXRs and electrophysiological data on the magnitude, latency, and scalp location of their N170 ERPs as they interpreted the two types of images. Cardiologists demonstrated significantly more expertise with EKGs than CXRs, and this was reflected in an increased amplitude of their N170 ERPs while reading EKGs compared to CXRs. Pulmonologists demonstrated equal expertise with both types of images, and this was reflected in equal N170 ERP amplitudes for EKGs and CXRs. The results suggest provisionally that visual expertise has a similar substrate in medical practice as it does in other domains that have been studied extensively. This provides support for applying a sophisticated body of literature to questions about training and assessment of visual expertise among physicians.

Expertise in clinical pathology: combining the visual and cognitive perspective

Expertise studies in the medical domain often focus on either visual or cognitive aspects of expertise. As a result, characteristics of expert behaviour are often described as either cognitive or visual abilities. This study focuses on both aspects of expertise and analyses them along three overarching constructs: (1) encapsulations, (2) efficiency, and (3) hypothesis testing. This study was carried out among clinical pathologists performing an authentic task: diagnosing microscopic slides. Participants were 13 clinical pathologists (experts), 12 residents in pathology (intermediates), and 13 medical students (novices). They all diagnosed seven cases in a virtual microscope and gave post hoc explanations for their diagnoses. The collected data included eye movements, microscope navigation, and verbal protocols. Results showed that experts used lower magnifications and verbalized their findings as diagnoses. Also, their diagnostic paths were more efficient, including fewer microscope movements and shorter reasoning chains. Experts entered relevant areas later in their diagnostic process, and visited fewer of them. Intermediates used relatively high magnifications and based their diagnoses on specific abnormalities. Also, they took longer to reach their diagnosis and checked more relevant areas. Novices searched in detail, described findings by their appearances, and uttered long reasoning chains. These results indicate that overarching constructs can justly be identified: encapsulations and efficiency are apparent in both visual and cognitive aspects of expertise.