Narrative Comprehension Guides Eye Movements in the Absence of Motion

Narrative predicts cardiac synchrony in audiences

Audio-visual media possesses a remarkable ability to synchronise audiences' neural, behavioural, and physiological responses. This synchronisation is considered to reflect some dimension of collective attention or engagement with the stimulus. But what is it about these stimuli that drives such strong engagement? There are several properties of media stimuli which may lead to synchronous audience response: from low-level audio-visual features, to the story itself. Here, we present a study which separates low-level features from narrative by presenting participants with the same content but in separate modalities. In this way, the presentations shared no low-level features, but participants experienced the same narrative. We show that synchrony in participants' heart rate can be driven by the narrative information alone. We computed both visual and auditory perceptual saliency for the content and found that narrative was approximately 10 times as predictive of heart rate as low-level saliency, but that low-level audio-visual saliency has a small additive effect towards heart rate. Further, heart rate synchrony was related to a separate cohorts' continuous ratings of immersion, and that synchrony is likely to be higher at moments of increased narrative importance. Our findings demonstrate that high-level narrative dominates in the alignment of physiology across viewers.

Is a knife the same as a plunger? Comparing the attentional effects of weapons and non-threatening unusual objects in dynamic scenes

Observers' memory for a person's appearance can be compromised by the presence of a weapon, a phenomenon known as the weapon-focus effect (WFE). According to the unusual-item hypothesis, attention shifts from the perpetrator to the weapon because a weapon is an unusual object in many contexts. To test this assumption, we monitored participants' eye movements while they watched a mock-crime video. The video was presented with sound and featured a female perpetrator holding either a weapon, a non-threatening unusual object, or a neutral object. Contrary to the predictions of current theories, there were no significant differences in total viewing times for the three objects. For the perpetrator, total viewing time was reduced when she held the non-threatening unusual object, but not when she held the weapon. However, weapon presence led to an attentional shift from the perpetrator's face toward her body. Detailed time-course analyses revealed that the effects of object type were more pronounced during early scene viewing. Thus, our results do not support the idea of extended attentional shifts from the perpetrator toward the unusual objects, but instead suggest more complex attentional effects. Contrary to previous research, memory for the perpetrator's appearance was not affected by object type. Thus, there was no WFE. An additional online experiment using the same videos and methodology produced a WFE, but this effect disappeared when the videos were presented without sound.

Influence of prior knowledge on eye movements to scenes as revealed by hidden Markov models

Human visual experience usually provides ample opportunity to accumulate knowledge about events unfolding in the environment. In typical scene perception experiments, however, participants view images that are unrelated to each other and, therefore, they cannot accumulate knowledge relevant to the upcoming visual input. Consequently, the influence of such knowledge on how this input is processed remains underexplored. Here, we investigated this influence in the context of gaze control. We used sequences of static film frames arranged in a way that allowed us to compare eye movements to identical frames between two groups: a group that accumulated prior knowledge relevant to the situations depicted in these frames and a group that did not. We used a machine learning approach based on hidden Markov models fitted to individual scanpaths to demonstrate that the gaze patterns from the two groups differed systematically and, thereby, showed that recently accumulated prior knowledge contributes to gaze control. Next, we leveraged the interpretability of hidden Markov models to characterize these differences. Additionally, we report two unexpected and interesting caveats of our approach. Overall, our results highlight the importance of recently acquired prior knowledge for oculomotor control and the potential of hidden Markov models as a tool for investigating it.

Prior knowledge about events depicted in scenes decreases oculomotor exploration

The visual input that the eyes receive usually contains temporally continuous information about unfolding events. Therefore, humans can accumulate knowledge about their current environment. Typical studies on scene perception, however, involve presenting multiple unrelated images and thereby render this accumulation unnecessary. Our study, instead, facilitated it and explored its effects. Specifically, we investigated how recently-accumulated prior knowledge affects gaze behavior. Participants viewed sequences of static film frames that contained several 'context frames' followed by a 'critical frame'. The context frames showed either events from which the situation depicted in the critical frame naturally followed, or events unrelated to this situation. Therefore, participants viewed identical critical frames while possessing prior knowledge that was either relevant or irrelevant to the frames' content. In the former case, participants' gaze behavior was slightly more exploratory, as revealed by seven gaze characteristics we analyzed. This result demonstrates that recently-gained prior knowledge reduces exploratory eye movements.

Gaze patterns reflect and predict expertise in dynamic echocardiographic imaging

Purpose

Echocardiography is the most important modality in cardiac imaging. Rapid valid visual assessment is a critical skill for image interpretation. However, it is unclear how skilled viewers assess echocardiographic images. Therefore, guidance and implicit advice are needed for learners to achieve valid image interpretation.

Approach

Using a signal detection approach, we compared 15 certified experts with 15 medical students in their diagnostic decision-making and viewing behavior. To quantify attention allocation, we recorded eye movements while viewing dynamic echocardiographic imaging loops of patients with reduced ejection fraction and healthy controls. Participants evaluated left ventricular ejection fraction and image quality (as diagnostic and visual control tasks, respectively).

Results

Experts were much better at discriminating between patients and healthy controls (d ' of 2.58, versus 0.98 for novices). Eye tracking revealed that experts fixated diagnostically relevant areas earlier and more often, whereas novices were distracted by visually salient task-irrelevant stimuli. We show that expertise status can be almost perfectly classified either based on judgments or purely on eye movements and that an expertise score derived from viewing behavior predicts diagnostic quality.

Conclusions

Judgments and eye tracking revealed significant differences between echocardiography experts and novices that can be used to derive numerical expertise scores. Experts have implicitly learned to ignore the salient motion cue presented by the mitral valve and to focus on the diagnostically more relevant left ventricle. These findings have implications for echocardiography training, objective characterization of echocardiographic expertise, and the design of user-friendly interfaces for echocardiography.