Preserved Haptic Shape Processing after Bilateral LOC Lesions

Reciprocal interactions among parietal and occipito-temporal representations support everyday object-directed actions

Everyday interactions with common manipulable objects require the integration of conceptual knowledge about objects and actions with real-time sensory information about the position, orientation and volumetric structure of the grasp target. The ability to successfully interact with everyday objects involves analysis of visual form and shape, surface texture, material properties, conceptual attributes such as identity, function and typical context, and visuomotor processing supporting hand transport, grasp form, and object manipulation. Functionally separable brain regions across the dorsal and ventral visual pathways support the processing of these different object properties and, in cohort, are necessary for functional object use. Object-directed grasps display end-state-comfort: they anticipate in form and force the shape and material properties of the grasp target, and how the object will be manipulated after it is grasped. End-state-comfort is the default for everyday interactions with manipulable objects and implies integration of information across the ventral and dorsal visual pathways. We propose a model of how visuomotor and action representations in parietal cortex interact with object representations in ventral and lateral occipito-temporal cortex. One pathway, from the supramarginal gyrus to the middle and inferior temporal gyrus, supports the integration of action-related information, including hand and limb position (supramarginal gyrus) with conceptual attributes and an appreciation of the action goal (middle temporal gyrus). A second pathway, from posterior IPS to the fusiform gyrus and collateral sulcus supports the integration of grasp parameters (IPS) with the surface texture and material properties (e.g., weight distribution) of the grasp target. Reciprocal interactions among these regions are part of a broader network of regions that support everyday functional object interactions.

Naturalistic Object Representations Depend on Distance and Size Cues

Egocentric distance and real-world size are important cues for object perception and action. Nevertheless, most studies of human vision rely on two-dimensional pictorial stimuli that convey ambiguous distance and size information. Here, we use fMRI to test whether pictures are represented differently in the human brain from real, tangible objects that convey unambiguous distance and size cues. Participants directly viewed stimuli in two display formats (real objects and matched printed pictures of those objects) presented at different egocentric distances (near and far). We measured the effects of format and distance on fMRI response amplitudes and response patterns. We found that fMRI response amplitudes in the lateral occipital and posterior parietal cortices were stronger overall for real objects than for pictures. In these areas and many others, including regions involved in action guidance, responses to real objects were stronger for near vs. far stimuli, whereas distance had little effect on responses to pictures-suggesting that distance determines relevance to action for real objects, but not for pictures. Although stimulus distance especially influenced response patterns in dorsal areas that operate in the service of visually guided action, distance also modulated representations in ventral cortex, where object responses are thought to remain invariant across contextual changes. We observed object size representations for both stimulus formats in ventral cortex but predominantly only for real objects in dorsal cortex. Together, these results demonstrate that whether brain responses reflect physical object characteristics depends on whether the experimental stimuli convey unambiguous information about those characteristics.

Significance Statement

Classic frameworks of vision attribute perception of inherent object characteristics, such as size, to the ventral visual pathway, and processing of spatial characteristics relevant to action, such as distance, to the dorsal visual pathway. However, these frameworks are based on studies that used projected images of objects whose actual size and distance from the observer were ambiguous. Here, we find that when object size and distance information in the stimulus is less ambiguous, these characteristics are widely represented in both visual pathways. Our results provide valuable new insights into the brain representations of objects and their various physical attributes in the context of naturalistic vision.

Similar object shape representation encoded in the inferolateral occipitotemporal cortex of sighted and early blind people

We can sense an object's shape by vision or touch. Previous studies suggested that the inferolateral occipitotemporal cortex (ILOTC) implements supramodal shape representations as it responds more to seeing or touching objects than shapeless textures. However, such activation in the anterior portion of the ventral visual pathway could be due to the conceptual representation of an object or visual imagery triggered by touching an object. We addressed these possibilities by directly comparing shape and conceptual representations of objects in early blind (who lack visual experience/imagery) and sighted participants. We found that bilateral ILOTC in both groups showed stronger activation during a shape verification task than during a conceptual verification task made on the names of the same manmade objects. Moreover, the distributed activity in the ILOTC encoded shape similarity but not conceptual association among objects. Besides the ILOTC, we also found shape representation in both groups' bilateral ventral premotor cortices and intraparietal sulcus (IPS), a frontoparietal circuit relating to object grasping and haptic processing. In contrast, the conceptual verification task activated both groups' left perisylvian brain network relating to language processing and, interestingly, the cuneus in early blind participants only. The ILOTC had stronger functional connectivity to the frontoparietal circuit than to the left perisylvian network, forming a modular structure specialized in shape representation. Our results conclusively support that the ILOTC selectively implements shape representation independently of visual experience, and this unique functionality likely comes from its privileged connection to the frontoparietal haptic circuit.

Functional relevance of the extrastriate body area for visual and haptic object recognition: a preregistered fMRI-guided TMS study

The extrastriate body area (EBA) is a region in the lateral occipito-temporal cortex (LOTC), which is sensitive to perceived body parts. Neuroimaging studies suggested that EBA is related to body and tool processing, regardless of the sensory modalities. However, how essential this region is for visual tool processing and nonvisual object processing remains a matter of controversy. In this preregistered fMRI-guided repetitive transcranial magnetic stimulation (rTMS) study, we examined the causal involvement of EBA in multisensory body and tool recognition. Participants used either vision or haptics to identify 3 object categories: hands, teapots (tools), and cars (control objects). Continuous theta-burst stimulation (cTBS) was applied over left EBA, right EBA, or vertex (control site). Performance for visually perceived hands and teapots (relative to cars) was more strongly disrupted by cTBS over left EBA than over the vertex, whereas no such object-specific effect was observed in haptics. The simulation of the induced electric fields confirmed that the cTBS affected regions including EBA. These results indicate that the LOTC is functionally relevant for visual hand and tool processing, whereas the rTMS over EBA may differently affect object recognition between the 2 sensory modalities.

Evidence for Independent Processing of Shape by Vision and Touch

Although visual object recognition is well studied and relatively well understood, much less is known about how shapes are recognized by touch and how such haptic stimuli might be compared with visual shapes. One might expect that the processes of visual and haptic object recognition engage similar brain structures given the advantages of avoiding redundant brain circuitry and indeed there is some evidence that this is the case. A potentially fruitful approach to understanding the differences in how shapes might be neurally represented is to find an algorithmic method of comparing shapes, which agrees with human behavior and determines whether that method differs between different modality conditions. If not, it would provide further evidence for a shared representation of shape. We recruited human participants to perform a one-back same-different visual and haptic shape comparison task both within (i.e., comparing two visual shapes or two haptic shapes) and across (i.e., comparing visual with haptic shapes) modalities. We then used various shape metrics to predict performance based on the shape, orientation, and modality of the two stimuli that were being compared on each trial. We found that the metrics that best predict shape comparison behavior heavily depended on the modality of the two shapes, suggesting differences in which features are used for comparing shapes depending on modality and that object recognition is not necessarily performed in a single, modality-agnostic region.

Visuomotor adaptation in the absence of input from early visual cortex

Prism adaptation is a time-honored tool for studying how the motor system adapts to sensory perturbations. Past research on the neural substrates of prism adaptation has implicated the posterior parietal cortex (PPC) and the cerebellum, under the assumption that these structures gain their visual input from the dominant retinogeniculate pathway to V1. Here we question whether this pathway is even required for visuomotor adaptation to occur. To investigate this, we examined prism adaptation in 'MC', someone who is blind to static stimuli following bilateral lesions that encompass much of her occipital cortex and the caudal-most areas of ventrotemporal cortex. Remarkably, MC shows evidence of prism adaptation that is similar to healthy control participants. First, when pointing with either the left or the right hand, MC shows spatial realignment; the classical after-effect exhibited by most people when adapting to displacing prisms. Second, MC demonstrates strategic recalibration - a reduction in her pointing error over time - that is similar in magnitude to healthy controls. These findings suggest that the geniculostriate pathway is not necessary for visuomotor adaptation to take place. Alternatively, we suggest that an extrageniculostriate pathway which provides visual inputs to the cerebellum from area MT and the PPC via the dorsolateral pons plays a significant and appreciable role in the guidance of unconscious automatic visuomotor adaptation.

Preserved Object Weight Processing after Bilateral Lateral Occipital Complex Lesions

Object interaction requires knowledge of the weight of an object, as well as its shape. The lateral occipital complex (LOC), an area within the ventral visual pathway, is well known to be critically involved in processing visual shape information. Recently, however, LOC has also been implicated in coding object weight before grasping-a result that is surprising because weight is a nonvisual object property that is more relevant for motor interaction than visual perception. Here, we examined the causal role of LOC in perceiving heaviness and in determining appropriate fingertip forces during object lifting. We studied perceptions of heaviness and lifting behavior in a neuropsychological patient (M.C.) who has large bilateral occipitotemporal lesions that include LOC. We compared the patient's performance to a group of 18 neurologically healthy age-matched controls. Participants were asked to lift and report the perceived heaviness of a set of equally weighted spherical objects of various sizes-stimuli which typically induce the size-weight illusion, in which the smaller objects feel heavier than the larger objects despite having identical mass. Despite her ventral stream lesions, M.C. experienced a robust size-weight illusion induced by visual cues to object volume, and the magnitude of the illusion in M.C. was comparable to age-matched controls. Similarly, M.C. evinced predictive fingertip force scaling to visual size cues during her initial lifts of the objects that were well within the normal range. These single-case neuropsychological findings suggest that LOC is unlikely to play a causal role in computing object weight.

Task- and domain-specific modulation of functional connectivity in the ventral and dorsal object-processing pathways

A whole-brain network of regions collectively supports the ability to recognize and use objects-the Tool Processing Network. Little is known about how functional interactions within the Tool Processing Network are modulated in a task-dependent manner. We designed an fMRI experiment in which participants were required to either generate object pantomimes or to carry out a picture matching task over the same images of tools, while holding all aspects of stimulus presentation constant across the tasks. The Tool Processing Network was defined with an independent functional localizer, and functional connectivity within the network was measured during the pantomime and picture matching tasks. Relative to tool picture matching, tool pantomiming led to an increase in functional connectivity between ventral stream regions and left parietal and frontal-motor areas; in contrast, the matching task was associated with an increase in functional connectivity among regions in ventral temporo-occipital cortex, and between ventral temporal regions and the left inferior parietal lobule. Graph-theory analyses over the functional connectivity data indicated that the left premotor cortex and left lateral occipital complex were hub-like (exhibited high betweenness centrality) during tool pantomiming, while ventral stream regions (left medial fusiform gyrus and left posterior middle temporal gyrus) were hub-like during the picture matching task. These results demonstrate task-specific modulation of functional interactions among a common set of regions, and indicate dynamic coupling of anatomically remote regions in task-dependent manner.

Abstract Representations of Object-Directed Action in the Left Inferior Parietal Lobule

Prior neuroimaging and neuropsychological research indicates that the left inferior parietal lobule in the human brain is a critical substrate for representing object manipulation knowledge. In the present functional MRI study we used multivoxel pattern analyses to test whether action similarity among objects can be decoded in the inferior parietal lobule independent of the task applied to objects (identification or pantomime) and stimulus format in which stimuli are presented (pictures or printed words). Participants pantomimed the use of objects, cued by printed words, or identified pictures of objects. Classifiers were trained and tested across task (e.g., training data: pantomime; testing data: identification), stimulus format (e.g., training data: word format; testing format: picture) and specific objects (e.g., training data: scissors vs. corkscrew; testing data: pliers vs. screwdriver). The only brain region in which action relations among objects could be decoded across task, stimulus format and objects was the inferior parietal lobule. By contrast, medial aspects of the ventral surface of the left temporal lobe represented object function, albeit not at the same level of abstractness as actions in the inferior parietal lobule. These results suggest compulsory access to abstract action information in the inferior parietal lobe even when simply identifying objects.

Psychophysical and neuroimaging responses to moving stimuli in a patient with the Riddoch phenomenon due to bilateral visual cortex lesions

Patients with injury to early visual cortex or its inputs can display the Riddoch phenomenon: preserved awareness for moving but not stationary stimuli. We provide a detailed case report of a patient with the Riddoch phenomenon, MC. MC has extensive bilateral lesions to occipitotemporal cortex that include most early visual cortex and complete blindness in visual field perimetry testing with static targets. Nevertheless, she shows a remarkably robust preserved ability to perceive motion, enabling her to navigate through cluttered environments and perform actions like catching moving balls. Comparisons of MC's structural magnetic resonance imaging (MRI) data to a probabilistic atlas based on controls reveals that MC's lesions encompass the posterior, lateral, and ventral early visual cortex bilaterally (V1, V2, V3A/B, LO1/2, TO1/2, hV4 and VO1 in both hemispheres) as well as more extensive damage to right parietal (inferior parietal lobule) and left ventral occipitotemporal cortex (VO1, PHC1/2). She shows some sparing of anterior occipital cortex, which may account for her ability to see moving targets beyond ~15 degrees eccentricity during perimetry. Most strikingly, functional and structural MRI revealed robust and reliable spared functionality of the middle temporal motion complex (MT+) bilaterally. Moreover, consistent with her preserved ability to discriminate motion direction in psychophysical testing, MC also shows direction-selective adaptation in MT+. A variety of tests did not enable us to discern whether input to MT+ was driven by her spared anterior occipital cortex or subcortical inputs. Nevertheless, MC shows rich motion perception despite profoundly impaired static and form vision, combined with clear preservation of activation in MT+, thus supporting the role of MT+ in the Riddoch phenomenon.

Recruitment of Foveal Retinotopic Cortex During Haptic Exploration of Shapes and Actions in the Dark

The role of the early visual cortex and higher-order occipitotemporal cortex has been studied extensively for visual recognition and to a lesser degree for haptic recognition and visually guided actions. Using a slow event-related fMRI experiment, we investigated whether tactile and visual exploration of objects recruit the same "visual" areas (and in the case of visual cortex, the same retinotopic zones) and if these areas show reactivation during delayed actions in the dark toward haptically explored objects (and if so, whether this reactivation might be due to imagery). We examined activation during visual or haptic exploration of objects and action execution (grasping or reaching) separated by an 18 s delay. Twenty-nine human volunteers (13 females) participated in this study. Participants had their eyes open and fixated on a point in the dark. The objects were placed below the fixation point and accordingly visual exploration activated the cuneus, which processes retinotopic locations in the lower visual field. Strikingly, the occipital pole (OP), representing foveal locations, showed higher activation for tactile than visual exploration, although the stimulus was unseen and location in the visual field was peripheral. Moreover, the lateral occipital tactile-visual area (LOtv) showed comparable activation for tactile and visual exploration. Psychophysiological interaction analysis indicated that the OP showed stronger functional connectivity with anterior intraparietal sulcus and LOtv during the haptic than visual exploration of shapes in the dark. After the delay, the cuneus, OP, and LOtv showed reactivation that was independent of the sensory modality used to explore the object. These results show that haptic actions not only activate "visual" areas during object touch, but also that this information appears to be used in guiding grasping actions toward targets after a delay.SIGNIFICANCE STATEMENT Visual presentation of an object activates shape-processing areas and retinotopic locations in early visual areas. Moreover, if the object is grasped in the dark after a delay, these areas show "reactivation." Here, we show that these areas are also activated and reactivated for haptic object exploration and haptically guided grasping. Touch-related activity occurs not only in the retinotopic location of the visual stimulus, but also at the occipital pole (OP), corresponding to the foveal representation, even though the stimulus was unseen and located peripherally. That is, the same "visual" regions are implicated in both visual and haptic exploration; however, touch also recruits high-acuity central representation within early visual areas during both haptic exploration of objects and subsequent actions toward them. Functional connectivity analysis shows that the OP is more strongly connected with ventral and dorsal stream areas when participants explore an object in the dark than when they view it.

The Sander parallelogram illusion dissociates action and perception despite control for the litany of past confounds

The two visual systems hypothesis proposes that human vision is supported by an occipito-temporal network for the conscious visual perception of the world and a fronto-parietal network for visually-guided, object-directed actions. Two specific claims about the fronto-parietal network's role in sensorimotor control have generated much data and controversy: (1) the network relies primarily on the absolute metrics of target objects, which it rapidly transforms into effector-specific frames of reference to guide the fingers, hands, and limbs, and (2) the network is largely unaffected by scene-based information extracted by the occipito-temporal network for those same targets. These two claims lead to the counter-intuitive prediction that in-flight anticipatory configuration of the fingers during object-directed grasping will resist the influence of pictorial illusions. The research confirming this prediction has been criticized for confounding the difference between grasping and explicit estimates of object size with differences in attention, sensory feedback, obstacle avoidance, metric sensitivity, and priming. Here, we address and eliminate each of these confounds. We asked participants to reach out and pick up 3D target bars resting on a picture of the Sander Parallelogram illusion and to make explicit estimates of the length of those bars. Participants performed their grasps without visual feedback, and were permitted to grasp the targets after making their size-estimates to afford them an opportunity to reduce illusory error with haptic feedback. The results show unequivocally that the effect of the illusion is stronger on perceptual judgments than on grasping. Our findings from the normally-sighted population provide strong support for the proposal that human vision is comprised of functionally and anatomically dissociable systems.

Single-case cognitive neuropsychology in the age of big data

Historically, single-case studies of brain-damaged individuals have contributed substantially to our understanding of cognitive processes. However, the role of single-case cognitive neuropsychology has diminished with the proliferation of techniques that measure neural activity in humans. Instead, large-scale informatics approaches in which data are gathered from hundreds of neuroimaging studies have become popular. It has been claimed that utilizing these informatics approaches can address problems found in single imaging studies. We first discuss reasons for why cognitive neuropsychology is thought to be in decline. Next, we note how these informatics approaches, while having benefits, are not particularly suited for understanding functional architectures. We propose that the single-case cognitive neuropsychological approach, which is focused on developing models of cognitive processing, addresses several of the weaknesses inherent in informatics approaches. Furthermore, we discuss how using neural data from brain-damaged individuals provides data that can inform both cognitive and neural models of cognitive processing.

Neuroanatomical correlates of haptic object processing: combined evidence from tractography and functional neuroimaging

Touch delivers a wealth of information already from birth, helping infants to acquire knowledge about a variety of important object properties using their hands. Despite the fact that we are touch experts as much as we are visual experts, surprisingly, little is known how our perceptual ability in touch is linked to either functional or structural aspects of the brain. The present study, therefore, investigates and identifies neuroanatomical correlates of haptic perceptual performance using a novel, multi-modal approach. For this, participants' performance in a difficult shape categorization task was first measured in the haptic domain. Using a multi-modal functional magnetic resonance imaging and diffusion-weighted magnetic resonance imaging analysis pipeline, functionally defined and anatomically constrained white-matter pathways were extracted and their microstructural characteristics correlated with individual variability in haptic categorization performance. Controlling for the effects of age, total intracranial volume and head movements in the regression model, haptic performance was found to correlate significantly with higher axial diffusivity in functionally defined superior longitudinal fasciculus (fSLF) linking frontal and parietal areas. These results were further localized in specific sub-parts of fSLF. Using additional data from a second group of participants, who first learned the categories in the visual domain and then transferred to the haptic domain, haptic performance correlates were obtained in the functionally defined inferior longitudinal fasciculus. Our results implicate SLF linking frontal and parietal areas as an important white-matter track in processing touch-specific information during object processing, whereas ILF relays visually learned information during haptic processing. Taken together, the present results chart for the first time potential neuroanatomical correlates and interactions of touch-related object processing.

Analysis of haptic information in the cerebral cortex

Haptic sensing of objects acquires information about a number of properties. This review summarizes current understanding about how these properties are processed in the cerebral cortex of macaques and humans. Nonnoxious somatosensory inputs, after initial processing in primary somatosensory cortex, are partially segregated into different pathways. A ventrally directed pathway carries information about surface texture into parietal opercular cortex and thence to medial occipital cortex. A dorsally directed pathway transmits information regarding the location of features on objects to the intraparietal sulcus and frontal eye fields. Shape processing occurs mainly in the intraparietal sulcus and lateral occipital complex, while orientation processing is distributed across primary somatosensory cortex, the parietal operculum, the anterior intraparietal sulcus, and a parieto-occipital region. For each of these properties, the respective areas outside primary somatosensory cortex also process corresponding visual information and are thus multisensory. Consistent with the distributed neural processing of haptic object properties, tactile spatial acuity depends on interaction between bottom-up tactile inputs and top-down attentional signals in a distributed neural network. Future work should clarify the roles of the various brain regions and how they interact at the network level.

Haptically Guided Grasping. fMRI Shows Right-Hemisphere Parietal Stimulus Encoding, and Bilateral Dorso-Ventral Parietal Gradients of Object- and Action-Related Processing during Grasp Execution

The neural bases of haptically-guided grasp planning and execution are largely unknown, especially for stimuli having no visual representations. Therefore, we used functional magnetic resonance imaging (fMRI) to monitor brain activity during haptic exploration of novel 3D complex objects, subsequent grasp planning, and the execution of the pre-planned grasps. Haptic object exploration, involving extraction of shape, orientation, and length of the to-be-grasped targets, was associated with the fronto-parietal, temporo-occipital, and insular cortex activity. Yet, only the anterior divisions of the posterior parietal cortex (PPC) of the right hemisphere were significantly more engaged in exploration of complex objects (vs. simple control disks). None of these regions were re-recruited during the planning phase. Even more surprisingly, the left-hemisphere intraparietal, temporal, and occipital areas that were significantly invoked for grasp planning did not show sensitivity to object features. Finally, grasp execution, involving the re-recruitment of the critical right-hemisphere PPC clusters, was also significantly associated with two kinds of bilateral parieto-frontal processes. The first represents transformations of grasp-relevant target features and is linked to the dorso-dorsal (lateral and medial) parieto-frontal networks. The second monitors grasp kinematics and belongs to the ventro-dorsal networks. Indeed, signal modulations associated with these distinct functions follow dorso-ventral gradients, with left aIPS showing significant sensitivity to both target features and the characteristics of the required grasp. Thus, our results from the haptic domain are consistent with the notion that the parietal processing for action guidance reflects primarily transformations from object-related to effector-related coding, and these mechanisms are rather independent of sensory input modality.