Inappropriate rightward saccades after right hemisphere damage: Oculomotor analysis and anatomical correlates

Machine learning algorithms on eye tracking trajectories to classify patients with spatial neglect

BACKGROUND AND OBJECTIVE

Eye-movement trajectories are rich behavioral data, providing a window on how the brain processes information. We address the challenge of characterizing signs of visuo-spatial neglect from saccadic eye trajectories recorded in brain-damaged patients with spatial neglect as well as in healthy controls during a visual search task.

METHODS

We establish a standardized pre-processing pipeline adaptable to other task-based eye-tracker measurements. We use traditional machine learning algorithms together with deep convolutional networks (both 1D and 2D) to automatically analyze eye trajectories.

RESULTS

Our top-performing machine learning models classified neglect patients vs. healthy individuals with an Area Under the ROC curve (AUC) ranging from 0.83 to 0.86. Moreover, the 1D convolutional neural network scores correlated with the degree of severity of neglect behavior as estimated with standardized paper-and-pencil tests and with the integrity of white matter tracts measured from Diffusion Tensor Imaging (DTI). Interestingly, the latter showed a clear correlation with the third branch of the superior longitudinal fasciculus (SLF), especially damaged in neglect.

CONCLUSIONS

The study introduces new methods for both the pre-processing and the classification of eye-movement trajectories in patients with neglect syndrome. The proposed methods can likely be applied to other types of neurological diseases opening the possibility of new computer-aided, precise, sensitive and non-invasive diagnostic tools.

Visual and motor neglect: Clinical and neurocognitive aspects

Attention allows us to prioritize the processing of external information according to our goals, but also to cope with sudden, unforeseen events. Attention processes rely on the coordinated activity of large-scale brain networks. At the cortical level, these systems are mainly organized in fronto-parietal networks, with functional and anatomical asymmetries in favor of the right hemisphere. Dysfunction of these right-lateralized networks often produce severe deficit of spatial attention, such as visual neglect. Other brain-damaged patients avoid moving the limbs contralateral to their brain lesion, even in the absence of sensorimotor deficits (motor neglect). This paper first summarizes past and current evidence on brain networks of attention; then, it presents clinical and experimental findings on visual and motor neglect, and on the possible mechanisms of clinical recovery.

Virtual prism adaptation for spatial neglect: A double-blind study

ABSTRACTPrismatic adaptation (PA) with wedge prisms is a non-invasive technique used in the rehabilitation of patients suffering from spatial neglect. Unfortunately, as for many behavioural intervention techniques, it is nearly impossible to achieve adequate blinding using wedge prisms, and the potential benefit of PA in the rehabilitation of neglect remains controversial. In order to study an alternative to wedge prism, we examine whether virtual PA at different degrees of deviation may alleviate signs of neglect in a double-blind design. Fifteen neglect patients participated in three adaptation sessions, which differed by the degree of deviation (0°, 15°, or 30°). Performance in line bisection and item cancellation tasks was measured in virtual reality immediately before and after adaptation. Session allocation was concealed from patients and the examiner. Despite the presence of robust, dose-dependent effects of virtual PA on Open-Loop Pointing (OLP), no transfer to line bisection and item cancellation tests were observed. None of the patients were aware of differences between sessions. Virtual PA did not result in visuo-motor transfer effects despite inducing significant adaptation effects in OLP. Together with recent negative findings of randomized-controlled trials, these findings cast doubt on the general efficacy of PA as a rehabilitation method of spatial neglect.

Fronto-parietal networks underlie the interaction between executive control and conscious perception: Evidence from TMS and DWI

The executive control network is involved in novel situations or those in which prepotent responses need to be overridden. Previous studies have demonstrated that when control is exerted, conscious perception is impaired, and this effect is related to the functional connectivity of fronto-parietal regions. In the present study, we explored the causal involvement of one of the nodes of this fronto-parietal network (the right Supplementary Motor Area, SMA) in the interaction between executive control and conscious perception. Participants performed a dual task in which they responded to a Stroop task while detecting the presence/absence of a near-threshold Gabor stimulus. Concurrently, transcranial magnetic stimulation (TMS) was applied over the right SMA or a control site (vertex; Experiment 1). As a further control, the right Frontal Eye Field (FEF) was stimulated in Experiment 2. Diffusion-weighted imaging (DWI) tractography was used to isolate the three branches of the superior longitudinal fasciculus (SLF I, II and III), and the frontal aslant tract (FAT), and to explore if TMS effects were related to their micro- and macrostructural characteristics. Results demonstrated reduced perceptual sensitivity on incongruent as compared to congruent Stroop trials. A causal role of the right SMA on the modulation of perceptual sensitivity by executive control was only demonstrated when the microstructure of the right SLF III or the left FAT were taken into account. The volume of the right SLF III was also related to the modulation of response criterion by executive control when the right FEF was stimulated. These results add evidence in favor of shared neural correlates for attention and conscious perception in fronto-parietal regions and highlight the role of white matter in TMS effects.

Keeping an eye on visual search patterns in visuospatial neglect: A systematic review

Patients with visuospatial neglect exhibit a failure to detect, respond, or orient towards information located in the side of space opposite to their brain lesion. To extend our understanding of the underlying cognitive processes involved in neglect, some studies have used eye movement measurements to complement behavioural data. We provide a qualitative synthesis of studies that have used eye-tracking in patients with neglect, with a focus on highlighting the utility of examining eye movements and reporting what eye-tracking has revealed about visual search patterns in these patients. This systematic review includes twenty studies that met the eligibility criteria. We extracted information pertaining to patient characteristics (e.g., age, type of stroke, time since stroke), neglect test(s) used, type of stimuli (e.g., static, dynamic), eye-tracker specifications (e.g., temporal and spatial resolution), and eye movement measurements (e.g., saccade amplitude, fixation duration). Five key themes were identified. First, eye-tracking is a useful tool to complement pen-and-paper neglect tests. Second, the lateral asymmetrical bias in eye movement patterns observed during active exploration also occurred while at rest. Third, the lateral asymmetrical bias was evident not only in the horizontal plane but also in the vertical plane. Fourth, eye movement patterns were modulated by stimulus- and task-related factors (e.g., visual salience, local perceptual features, image content, stimulus duration, presence of distractors). Fifth, measuring eye movements in patients with neglect is useful for determining and understanding other cognitive impairments, such as spatial working memory. To develop a fuller, and a more accurate, picture of neglect, future research would benefit from eye movement measurements.

Visual Exploration Area in Neglect: A New Analysis Method for Video-Oculography Data Based on Foveal Vision

Video-oculography during free visual exploration (FVE) is a valuable tool to evaluate visual attention spatial allocation in neglect patients after right-hemispheric stroke. In conventional FVE analyses, the position of a visual fixation is conceived as a single point in space. Here, we describe a new complementary method to analyze FVE data based on foveal vision, leading to an accurate estimate of the portion of the picture that was effectively explored. In 15 neglect patients and 20 healthy controls, visual exploration areas (i.e., considering 1° visual angle around every single fixation) were computed. Furthermore, the proportion of single and overlapping fixations was analyzed. Overlapping fixations were further categorized into capture fixations (successive overlapping fixation, putatively reflecting problem of disengagement) and re-capture fixations (temporally distant overlapping fixations, putatively reflecting spatial working memory deficits). The results of this new analysis approach were compared to the ones of conventional approaches. Conventional analyses showed the typical visual attention deficits in neglect patients versus healthy controls: significantly less fixations and time spent within the left and significantly more fixations and time spent within the right screen half. According to the results of our new approach, patients showed a significantly smaller visual exploration area within the left screen half. However, the right visual exploration area did not differ between groups. Furthermore, in neglect patients, the proportion of overlapping fixations within the right screen half was significantly higher than within the left screen half, as well as significantly higher than in healthy controls within either screen halves. Whereas neglect patients showed significantly more capture fixations than healthy controls, the number of re-capture fixations did not differ between groups. These results suggest that, in neglect patients, the efficiency of visual exploration is also reduced within the right screen half and that impaired disengagement might be an important mechanism leading to overlapping fixations. Our new analysis of the visual exploration area, based on foveal vision, may be a promising additional approach in visual attention research. It allows to accurately measure the portion of the picture that was effectively explored, disentangle single from overlapping fixations, and distinguish between capture and re-capture fixations.

Connectivity of Frontoparietal Regions Reveals Executive Attention and Consciousness Interactions

The executive control network is involved in the voluntary control of novel and complex situations. Solving conflict situations or detecting errors have demonstrated to impair conscious perception of near-threshold stimuli. The aim of this study was to explore the neural mechanisms underlying executive control and its interaction with conscious perception using functional magnetic resonance imaging and diffusion-weighted imaging. To this end, we used a dual-task paradigm involving Stroop and conscious detection tasks with near-threshold stimuli. A set of prefrontal and frontoparietal regions were more strongly engaged for incongruent than congruent trials while a distributed set of frontoparietal regions showed stronger activation for consciously than nonconsciously perceived trials. Functional connectivity analysis revealed an interaction between executive control and conscious perception in frontal and parietal nodes. The microstructural properties of the middle branch of the superior longitudinal fasciculus were associated with neural measures of the interaction between executive control and consciousness. These results demonstrate that conscious perception and executive control share neural resources in frontoparietal networks, as proposed by some influential models.

Rehabilitation in chronic spatial neglect strengthens resting-state connectivity

OBJECTIVES

Rehabilitation of patients with chronic visuospatial neglect is underexplored, and little is known about neural mechanisms that can be exploited to promote recovery. In this study, we present data on resting-state functional connectivity within the dorsal attention network (DAN) in chronic neglect patients as they underwent training in a virtual reality (VR) environment that improved left-side awareness.

METHODS

The study included 13 patients with visuospatial neglect persisting more than six months after a right-sided stroke. The patients underwent resting-state functional magnetic resonance imaging (fMRI). Scans were collected at baseline and after five weeks of intense training. We specifically examined resting-state functional connectivity within the DAN. In addition, using spatial concordance correlation, we compared changes in the spatial topology of the DAN with that of other networks.

RESULTS

We found a longitudinal increase in interhemispheric functional connectivity between the right frontal eye field and the left intraparietal sulcus following training (before: 0.33 ± 0.17 [mean ± SD]; after: 0.45 ± 0.13; P = 0.004). The spatial concordance analyses indicated that training influenced the DAN connectivity more than any of the other networks.

CONCLUSION

Intense VR training that improved left-sided awareness in chronic stroke patients also increased sporadic interhemispheric functional connectivity within the DAN. Specifically, a region responsible for saccadic eye movement to the left became more integrated with the left posterior parietal cortex. These results highlight a mechanism that should be exploited in the training of patients with chronic visuospatial neglect.

Influence of reward learning on visual attention and eye movements in a naturalistic environment: A virtual reality study

Rewards constitute crucial signals that motivate approach behavior and facilitate the perceptual processing of objects associated with favorable outcomes in past encounters. Reward-related influences on perception and attention have been reliably observed in studies where a reward is paired with a unidimensional low-level visual feature, such as the color or orientation of a line in visual search tasks. However, our environment is drastically different and composed of multidimensional and changing visual features, encountered in complex and dynamic scenes. Here, we designed an immersive virtual reality (VR) experiment using a 4-frame CAVE system to investigate the impact of rewards on attentional orienting and gaze patterns in a naturalistic and ecological environment. Forty-one healthy participants explored a virtual forest and responded to targets appearing on either the left or right side of their path. To test for reward-induced biases in spatial orienting, targets on one side were associated with high reward, whereas those on the opposite side were paired with a low reward. Eye-movements recording showed that left-side high rewards led to subsequent increase of eye gaze fixations towards this side of the path, but no such asymmetry was found after exposure to right-sided high rewards. A milder spatial bias was also observed after left-side high rewards during subsequent exploration of a virtual castle yard, but not during route turn choices along the forest path. Our results indicate that reward-related influences on attention and behavior may be better learned in left than right space, in line with a right hemisphere dominance, and could generalize to another environment to some extent, but not to spatial choices in another decision task, suggesting some domain- or context-specificity. This proof-of-concept study also outlines the advantages and the possible drawbacks of the use of the 3D CAVE immersive platform for VR in neuroscience.

Value-driven attentional capture in neglect

OBJECTIVE

Recent studies suggest that motivational cues such as rewards may be a powerful determinant of attentional selection, both in healthy subjects and in brain-damaged patients suffering from neglect. However, the exact brain mechanisms underlying these effects and their relation to other well-known attentional systems are still poorly known.

METHODS

We designed a visual search paradigm to examine how value-based attentional priority could modulate spatial orienting in patients with pathological biases due to neglect after right hemispheric stroke. Targets were preceded by exogenous valid or invalid spatial cues, in the presence or absence of distractors that were associated with high reward values subsequent to an initial reinforcement training phase.

RESULTS

We found that the learned reward value of distractors interfered with spatial reorienting toward the left (neglected) side when neglect patients were invalidly cued to the right side. Moreover, the presence of reward-associated distractors in the contralesional field interfered most with the detection of task-relevant targets on the same side, and this interference was exaggerated with more severe neglect. Voxelwise anatomical lesion analysis indicated that damage to the right angular gyrus, as well as lateral occipital and inferior temporal areas of the right hemisphere, were associated with stronger value-driven attentional effects.

CONCLUSIONS

Visual stimuli previously associated with rewards receive higher attentional priority during visual search despite pathological spatial biases due to neglect, and thus interfere with orienting to contralesional targets, presumably by competing with top-down mechanisms controlling exogenous spatial attention. Reward signals may bias neural activity evoked by visual stimuli, independent of conscious control, through a common priority map integrating several different attentional influences. These results do not only provide novel insights to link spatial orienting and motivational signals within current models of attention, but also open new perspectives that may usefully be exploited for neurological rehabilitation strategies in patients suffering from attentional deficits and neglect.

The Computational Anatomy of Visual Neglect

Visual neglect is a debilitating neuropsychological phenomenon that has many clinical implications and-in cognitive neuroscience-offers an important lesion deficit model. In this article, we describe a computational model of visual neglect based upon active inference. Our objective is to establish a computational and neurophysiological process theory that can be used to disambiguate among the various causes of this important syndrome; namely, a computational neuropsychology of visual neglect. We introduce a Bayes optimal model based upon Markov decision processes that reproduces the visual searches induced by the line cancellation task (used to characterize visual neglect at the bedside). We then consider 3 distinct ways in which the model could be lesioned to reproduce neuropsychological (visual search) deficits. Crucially, these 3 levels of pathology map nicely onto the neuroanatomy of saccadic eye movements and the systems implicated in visual neglect.

White matter microstructure of attentional networks predicts attention and consciousness functional interactions

Attention is considered as one of the pre-requisites of conscious perception. Phasic alerting and exogenous orienting improve conscious perception of near-threshold information through segregated brain networks. Using a multimodal neuroimaging approach, combining data from functional MRI (fMRI) and diffusion-weighted imaging (DWI), we investigated the influence of white matter properties of the ventral branch of superior longitudinal fasciculus (SLF III) in functional interactions between attentional systems and conscious perception. Results revealed that (1) reduced integrity of the left hemisphere SLF III was predictive of the neural interactions observed between exogenous orienting and conscious perception, and (2) increased integrity of the left hemisphere SLF III was predictive of the neural interactions observed between phasic alerting and conscious perception. Our results combining fMRI and DWI data demonstrate that structural properties of the white matter organization determine attentional modulations over conscious perception.

When brain damage "improves" perception: neglect patients can localize motion-shifted probes better than controls

When we look at bars flashed against a moving background, we see them displaced in the direction of the upcoming motion (flash-grab illusion). It is still debated whether these motion-induced position shifts are low-level, reflexive consequences of stimulus motion or high-level compensation engaged only when the stimulus is tracked with attention. To investigate whether attention is a causal factor for this striking illusory position shift, we evaluated the flash-grab illusion in six patients with damaged attentional networks in the right hemisphere and signs of left visual neglect and six age-matched controls. With stimuli in the top, right, and bottom visual fields, neglect patients experienced the same amount of illusion as controls. However, patients showed no significant shift when the test was presented in their left hemifield, despite having equally precise judgments. Thus, paradoxically, neglect patients perceived the position of the flash more veridically in their neglected hemifield. These results suggest that impaired attentional processes can reduce the interaction between a moving background and a superimposed stationary flash, and indicate that attention is a critical factor in generating the illusory motion-induced shifts of location.