Multisensory integration, sensory substitution and visual rehabilitation

Sensory substitution has advanced remarkably over the past 35 years since first introduced to the scientific literature by Paul Bach-y-Rita. In this issue dedicated to his memory, we describe a collection of reviews that assess the current state of neuroscience research on sensory substitution, visual rehabilitation, and multisensory processes.

EyeMusic: Introducing a "visual" colorful experience for the blind using auditory sensory substitution

PURPOSE

Sensory-substitution devices (SSDs) provide auditory or tactile representations of visual information. These devices often generate unpleasant sensations and mostly lack color information. We present here a novel SSD aimed at addressing these issues.

METHODS

We developed the EyeMusic, a novel visual-to-auditory SSD for the blind, providing both shape and color information. Our design uses musical notes on a pentatonic scale generated by natural instruments to convey the visual information in a pleasant manner. A short behavioral protocol was utilized to train the blind to extract shape and color information, and test their acquired abilities. Finally, we conducted a survey and a comparison task to assess the pleasantness of the generated auditory stimuli.

RESULTS

We show that basic shape and color information can be decoded from the generated auditory stimuli. High performance levels were achieved by all participants following as little as 2-3 hours of training. Furthermore, we show that users indeed found the stimuli pleasant and potentially tolerable for prolonged use.

CONCLUSIONS

The novel EyeMusic algorithm provides an intuitive and relatively pleasant way for the blind to extract shape and color information. We suggest that this might help facilitating visual rehabilitation because of the added functionality and enhanced pleasantness.

Color improves "visual" acuity via sound

Visual-to-auditory sensory substitution devices (SSDs) convey visual information via sound, with the primary goal of making visual information accessible to blind and visually impaired individuals. We developed the EyeMusic SSD, which transforms shape, location, and color information into musical notes. We tested the “visual” acuity of 23 individuals (13 blind and 10 blindfolded sighted) on the Snellen tumbling-E test, with the EyeMusic. Participants were asked to determine the orientation of the letter “E.” The test was repeated twice: in one test, the letter “E” was drawn with a single color (white), and in the other test, with two colors (red and white). In the latter case, the vertical line in the letter, when upright, was drawn in red, with the three horizontal lines drawn in white. We found no significant differences in performance between the blind and the sighted groups. We found a significant effect of the added color on the “visual” acuity. The highest acuity participants reached in the monochromatic test was 20/800, whereas with the added color, acuity doubled to 20/400. We conclude that color improves “visual” acuity via sound.

Visual cortex extrastriate body-selective area activation in congenitally blind people "seeing" by using sounds

Vision is by far the most prevalent sense for experiencing others' body shapes, postures, actions, and intentions, and its congenital absence may dramatically hamper body-shape representation in the brain. We investigated whether the absence of visual experience and limited exposure to others' body shapes could still lead to body-shape selectivity. We taught congenitally fully-blind adults to perceive full-body shapes conveyed through a sensory-substitution algorithm topographically translating images into soundscapes [1]. Despite the limited experience of the congenitally blind with external body shapes (via touch of close-by bodies and for ~10 hr via soundscapes), once the blind could retrieve body shapes via soundscapes, they robustly activated the visual cortex, specifically the extrastriate body area (EBA; [2]). Furthermore, body selectivity versus textures, objects, and faces in both the blind and sighted control groups was not found in the temporal (auditory) or parietal (somatosensory) cortex but only in the visual EBA. Finally, resting-state data showed that the blind EBA is functionally connected to the temporal cortex temporal-parietal junction/superior temporal sulcus Theory-of-Mind areas [3]. Thus, the EBA preference is present without visual experience and with little exposure to external body-shape information, supporting the view that the brain has a sensory-independent, task-selective supramodal organization rather than a sensory-specific organization.

Flexibility and Stability in Sensory Processing Revealed Using Visual-to-Auditory Sensory Substitution

The classical view of sensory processing involves independent processing in sensory cortices and multisensory integration in associative areas. This hierarchical structure has been challenged by evidence of multisensory responses in sensory areas, and dynamic weighting of sensory inputs in associative areas, thus far reported independently. Here, we used a visual-to-auditory sensory substitution algorithm (SSA) to manipulate the information conveyed by sensory inputs while keeping the stimuli intact. During scan sessions before and after SSA learning, subjects were presented with visual images and auditory soundscapes. The findings reveal 2 dynamic processes. First, crossmodal attenuation of sensory cortices changed direction after SSA learning from visual attenuations of the auditory cortex to auditory attenuations of the visual cortex. Secondly, associative areas changed their sensory response profile from strongest response for visual to that for auditory. The interaction between these phenomena may play an important role in multisensory processing. Consistent features were also found in the sensory dominance in sensory areas and audiovisual convergence in associative area Middle Temporal Gyrus. These 2 factors allow for both stability and a fast, dynamic tuning of the system when required.

Sensory substitution: closing the gap between basic research and widespread practical visual rehabilitation

Sensory substitution devices (SSDs) have come a long way since first developed for visual rehabilitation. They have produced exciting experimental results, and have furthered our understanding of the human brain. Unfortunately, they are still not used for practical visual rehabilitation, and are currently considered as reserved primarily for experiments in controlled settings. Over the past decade, our understanding of the neural mechanisms behind visual restoration has changed as a result of converging evidence, much of which was gathered with SSDs. This evidence suggests that the brain is more than a pure sensory-machine but rather is a highly flexible task-machine, i.e., brain regions can maintain or regain their function in vision even with input from other senses. This complements a recent set of more promising behavioral achievements using SSDs and new promising technologies and tools. All these changes strongly suggest that the time has come to revive the focus on practical visual rehabilitation with SSDs and we chart several key steps in this direction such as training protocols and self-train tools.

Increasing accessibility to the blind of virtual environments, using a virtual mobility aid based on the "EyeCane": feasibility study

Virtual worlds and environments are becoming an increasingly central part of our lives, yet they are still far from accessible to the blind. This is especially unfortunate as such environments hold great potential for them for uses such as social interaction, online education and especially for use with familiarizing the visually impaired user with a real environment virtually from the comfort and safety of his own home before visiting it in the real world. We have implemented a simple algorithm to improve this situation using single-point depth information, enabling the blind to use a virtual cane, modeled on the “EyeCane” electronic travel aid, within any virtual environment with minimal pre-processing. Use of the Virtual-EyeCane, enables this experience to potentially be later used in real world environments with identical stimuli to those from the virtual environment. We show the fast-learned practical use of this algorithm for navigation in simple environments.

Cross-sensory transfer of sensory-motor information: visuomotor learning affects performance on an audiomotor task, using sensory-substitution

Visual-to-auditory sensory-substitution devices allow users to perceive a visual image using sound. Using a motor-learning task, we found that new sensory-motor information was generalized across sensory modalities. We imposed a rotation when participants reached to visual targets, and found that not only seeing, but also hearing the location of targets via a sensory-substitution device resulted in biased movements. When the rotation was removed, aftereffects occurred whether the location of targets was seen or heard. Our findings demonstrate that sensory-motor learning was not sensory-modality-specific. We conclude that novel sensory-motor information can be transferred between sensory modalities.

Fast, accurate reaching movements with a visual-to-auditory sensory substitution device

PURPOSE

Visual sensory substitution devices (SSDs) use sound or touch to convey information that is normally perceived by vision. The primary focus of prior research using SSDs was the perceptual components of learning to use SSDs and their neural correlates. However, sensorimotor integration is critical in the effort to make SSDs relevant for everyday tasks, like grabbing a cup of coffee efficiently. The purpose of this study was to test the use of a novel visual-to-auditory SSD to guide a fast reaching movement.

METHODS

Using sound, the SSD device relays location, shape and color information. Participants were asked to make fast reaching movements to targets presented by the SSD.

RESULTS

After only a short practice session, blindfolded sighted participants performed fast and accurate movements to presented targets, which did not differ significantly from movements performed with visual feedback in terms of movement time, peak speed, and path length. A small but significant difference was found between the endpoint accuracy of movements under the two feedback conditions; remarkably, in both cases the average error was smaller than 0.5 cm.

CONCLUSIONS

Our findings combine with previous brain-imaging studies to support a theory of a modality-independent representation of spatial information. Task-specificity, rather than modality-specificity, of brain functions is crucially important for the rehabilitative use of SSDs in the blind and the visually impaired. We present the first direct comparison between movement trajectories performed with an SSD and ones performed under visual guidance. The accuracy level reached in this study demonstrates the potential applicability of using the visual-to-auditory SSD for performance of daily tasks which require fast, accurate reaching movements, and indicates a potential for rehabilitative use of the device.

'Visual' acuity of the congenitally blind using visual-to-auditory sensory substitution

Sensory Substitution Devices (SSDs) convey visual information through sounds or touch, thus theoretically enabling a form of visual rehabilitation in the blind. However, for clinical use, these devices must provide fine-detailed visual information which was not yet shown for this or other means of visual restoration. To test the possible functional acuity conveyed by such devices, we used the Snellen acuity test conveyed through a high-resolution visual-to-auditory SSD (The vOICe). We show that congenitally fully blind adults can exceed the World Health Organization (WHO) blindness acuity threshold using SSDs, reaching the highest acuity reported yet with any visual rehabilitation approach. This demonstrates the potential capacity of SSDs as inexpensive, non-invasive visual rehabilitation aids, alone or when supplementing visual prostheses.

Cortical activity during tactile exploration of objects in blind and sighted humans

PURPOSE

Recent studies show evidence of multisensory representation in the functionally normal visual cortex, but this idea remains controversial. Occipital cortex activation is often claimed to be a reflection of mental visual imagery processes triggered by other modalities. However, if the occipital cortex is genuinely active during touch, this might be the basis for the massive cross-modal plasticity observed in the congenitally blind.

METHODS

To address these issues, we used fMRI to compare patterns of activation evoked by a tactile object recognition (TOR) task (right or left hand) in 8 sighted and 8 congenitally blind subjects, with several other control tasks.

RESULTS

TOR robustly activated object selective regions in the lateral occipital complex (LOC/LOtv) in the blind (similar to the patterns of activation found in the sighted), indicating that object identification per se (i.e. in the absence of visual imagery) is sufficient to evoke responses in the LOC/LOtv. Importantly, there was negligible occipital activation for hand movements (imitating object palpations) in the occipital cortex, in both groups. Moreover, in both groups, TOR activation in the LOC/LOtv was bilateral, regardless of the palpating hand (similar to the lack of strong visual field preference in the LOC/LOtv for viewed objects). Finally, the most prominent enhancement in TOR activation in the congenitally blind (compared to their sighted peers) was found in the posterior occipital cortex.

CONCLUSIONS

These findings suggest that visual imagery is not an obligatory condition for object activation in visual cortex. It also demonstrates the massive plasticity in visual cortex of the blind for tactile object recognition that involves both the ventral and dorsal occipital areas, probably to support the high demand for this function in the blind.

Multisensory visual-tactile object related network in humans: insights gained using a novel crossmodal adaptation approach

Neuroimaging techniques have provided ample evidence for multisensory integration in humans. However, it is not clear whether this integration occurs at the neuronal level or whether it reflects areal convergence without such integration. To examine this issue as regards visuo-tactile object integration we used the repetition suppression effect, also known as the fMRI-based adaptation paradigm (fMR-A). Under some assumptions, fMR-A can tag specific neuronal populations within an area and investigate their characteristics. This technique has been used extensively in unisensory studies. Here we applied it for the first time to study multisensory integration and identified a network of occipital (LOtv and calcarine sulcus), parietal (aIPS), and prefrontal (precentral sulcus and the insula) areas all showing a clear crossmodal repetition suppression effect. These results provide a crucial first insight into the neuronal basis of visuo-haptic integration of objects in humans and highlight the power of using fMR-A to study multisensory integration using non-invasinve neuroimaging techniques.

A putative model of multisensory object representation

This review surveys the recent literature on visuo-haptic convergence in the perception of object form, with particular reference to the lateral occipital complex (LOC) and the intraparietal sulcus (IPS) and discusses how visual imagery or multisensory representations might underlie this convergence. Drawing on a recent distinction between object- and spatially-based visual imagery, we propose a putative model in which LOtv, a subregion of LOC, contains a modality-independent representation of geometric shape that can be accessed either bottom-up from direct sensory inputs or top-down from frontoparietal regions. We suggest that such access is modulated by object familiarity: spatial imagery may be more important for unfamiliar objects and involve IPS foci in facilitating somatosensory inputs to the LOC; by contrast, object imagery may be more critical for familiar objects, being reflected in prefrontal drive to the LOC.

Neural and behavioral correlates of drawing in an early blind painter: a case study

Humans rely heavily on vision to identify objects in the world and can create mental representations of the objects they encounter. Objects can also be identified and mentally represented through haptic exploration. However, it is unclear whether prior visual experience is necessary to generate these internal representations. Subject EA, an early blind artist, provides insight into this question. Like other blind individuals, EA captures the external world by touch. However, he is also able to reveal his internal representations through highly detailed drawings that are unequivocally understandable by a sighted person. We employed fMRI to investigate the neural correlates associated with EA's ability to transform tactilely explored three-dimensional objects into drawings and contrasted these findings with a series of control conditions (e.g. nonsensical scribbling as a sensory-motor control). Activation during drawing (compared to scribbling) occurred in brain areas normally associated with vision, including the striate cortex along with frontal and parietal cortical regions. Some of these areas showed overlap when EA was asked to mentally imagine the pictures he had to draw (albeit to a lesser anatomical extent and signal magnitude). These results have important implications as regards our understanding of the ways in which tactile information can generate mental representations of shapes and scenes in the absence of normal visual development. Furthermore, these findings suggest the occipital cortex plays a key role in supporting mental representations even without prior visual experience.

Spontaneous fluctuations in posterior alpha-band EEG activity reflect variability in excitability of human visual areas

Neural activity fluctuates dynamically with time, and these changes have been reported to be of behavioral significance, despite occurring spontaneously. Through electroencephalography (EEG), fluctuations in alpha-band (8-14 Hz) activity have been identified over posterior sites that covary on a trial-by-trial basis with whether an upcoming visual stimulus will be detected or not. These fluctuations are thought to index the momentary state of visual cortex excitability. Here, we tested this hypothesis by directly exciting human visual cortex via transcranial magnetic stimulation (TMS) to induce illusory visual percepts (phosphenes) in blindfolded participants, while simultaneously recording EEG. We found that identical TMS-stimuli evoked a percept (P-yes) or not (P-no) depending on prestimulus alpha-activity. Low prestimulus alpha-band power resulted in TMS reliably inducing phosphenes (P-yes trials), whereas high prestimulus alpha-values led the same TMS-stimuli failing to evoke a visual percept (P-no trials). Additional analyses indicated that the perceptually relevant fluctuations in alpha-activity/visual cortex excitability were spatially specific and occurred on a subsecond time scale in a recurrent pattern. Our data directly link momentary levels of posterior alpha-band activity to distinct states of visual cortex excitability, and suggest that their spontaneous fluctuation constitutes a visual operation mode that is activated automatically even without retinal input.

Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex

The lateral-occipital tactile-visual area (LOtv) is activated when objects are recognized by vision or touch. We report here that the LOtv is also activated in sighted and blind humans who recognize objects by extracting shape information from visual-to-auditory sensory substitution soundscapes. Recognizing objects by their typical sounds or learning to associate specific soundscapes with specific objects do not activate this region. This suggests that LOtv is driven by the presence of shape information.

Visual phosphene perception modulated by subthreshold crossmodal sensory stimulation

Crossmodal sensory interactions serve to integrate behaviorally relevant sensory stimuli. In this study, we investigated the effect of modulating crossmodal interactions between visual and somatosensory stimuli that in isolation do not reach perceptual awareness. When a subthreshold somatosensory stimulus was delivered within close spatiotemporal congruency to the expected site of perception of a phosphene, a subthreshold transcranial magnetic stimulation pulse delivered to the occipital cortex evoked a visual percept. The results suggest that under subthreshold conditions of visual and somatosensory stimulation, crossmodal interactions presented in a spatially and temporally specific manner can sum up to become behaviorally significant. These interactions may reflect an underlying anatomical connectivity and become further enhanced by attention modulation mechanisms.

Combined Activation and Deactivation of Visual Cortex During Tactile Sensory Processing

The involvement of occipital cortex in sensory processing is not restricted solely to the visual modality. Tactile processing has been shown to modulate higher-order visual and multisensory integration areas in sighted as well as visually deprived subjects; however, the extent of involvement of early visual cortical areas remains unclear. To investigate this issue, we employed functional magnetic resonance imaging in normally sighted, briefly blindfolded subjects with well-defined visuotopic borders as they tactually explored and rated raised-dot patterns. Tactile task performance resulted in significant activation in primary visual cortex (V1) and deactivation of extrastriate cortical regions V2, V3, V3A, and hV4 with greater deactivation in dorsal subregions and higher visual areas. These results suggest that tactile processing affects occipital cortex via two distinct pathways: a suppressive top-down pathway descending through the visual cortical hierarchy and an excitatory pathway arising from outside the visual cortical hierarchy that drives area V1 directly.

Attentional modulation of emotional stimulus processing: an fMRI study using emotional expectancy

We used emotional expectancy to study attentional modulation in the processing of emotional stimuli. During functional magnetic resonance imaging (fMRI), volunteers saw emotional and neutral expectancy cues signaling the subsequent presentation of corresponding emotional or neutral pictorial stimuli. As a control, emotional and neutral pictures were presented without preceding expectancy cue, resulting in a 2 x 2 factorial design with the factors "expectancy" and "emotion." Statistical analysis revealed a significant positive interaction effect between these factors in the medial prefrontal cortex (MPFC, Brodmann area [BA] 9/10), amygdala, and dorsal midbrain. In all these regions, expectancy augmented the neural response to emotional but not to neutral pictures. Time course analysis of raw data suggests that this augmented activation was not preceded by baseline increases in MPFC and amygdala during the period of emotional expectancy. In a post-scanning session, the paradigm was presented for a second time to allow emotional intensity rating. Again, a significant interaction between expectancy and emotion was observed, with intensity ratings specifically enhanced in emotional photographs preceded by expectancy. There was a positive correlation between intensity ratings and blood oxygenation level-dependent (BOLD) signals in the left amygdala. We conclude that specific components of the emotion network show enhanced activation in response to emotional stimuli when these are preceded by expectancy. This enhancement effect is not present in neutral pictures and might parallel accentuated subjective feeling states.

Negative BOLD Differentiates Visual Imagery and Perception

Recent studies emphasize the overlap between the neural substrates of visual perception and visual imagery. However, the subjective experiences of imagining and seeing are clearly different. Here we demonstrate that deactivation of auditory cortex (and to some extent of somatosensory and subcortical visual structures) as measured by BOLD functional magnetic resonance imaging unequivocally differentiates visual imagery from visual perception. During visual imagery, auditory cortex deactivation negatively correlates with activation in visual cortex and with the score in the subjective vividness of visual imagery questionnaire (VVIQ). Perception of the world requires the merging of multisensory information so that, during seeing, information from other sensory systems modifies visual cortical activity and shapes experience. We suggest that pure visual imagery corresponds to the isolated activation of visual cortical areas with concurrent deactivation of "irrelevant" sensory processing that could disrupt the image created by our "mind's eye."

Dissociable networks for the expectancy and perception of emotional stimuli in the human brain

William James posited that comparable brain regions were implicated in the anticipation and perception of a stimulus; however, dissociable networks (at least in part) may also underlie these processes. Recent functional neuroimaging studies have addressed this issue by comparing brain systems associated with the expectancy and perception of visual, tactile, nociceptive, and reward stimuli. In the present fMRI study, we addressed this issue in the domain of pictorial emotional stimuli (IAPS). Our paradigm involved the experimental conditions emotional expectancy, neutral expectancy, emotional picture perception, and neutral picture perception. Specifically, the emotional expectancy cue was uncertain in that it did not provide additional information regarding the positive or negative valence of the subsequent picture. Neutral expectancy and neutral picture perception served as control conditions, allowing the identification of expectancy and perception effects specific for emotion processing. To avoid contamination of the perception conditions by the preceding expectancy periods, 50% of the pictorial stimuli were presented without preceding expectancy cues. We found that the emotional expectancy cue specifically produced activation in the supracallosal anterior cingulate, cingulate motor area, and parieto-occipital sulcus. These regions were not significantly activated by emotional picture perception which recruited a different neuronal network, including the amygdala, insula, medial and lateral prefrontal cortex, cerebellum, and occipitotemporal areas. This dissociation may reflect a distinction between anticipatory and perceptive components of emotional stimulus processing.

THE PLASTIC HUMAN BRAIN CORTEX

Plasticity is an intrinsic property of the human brain and represents evolution's invention to enable the nervous system to escape the restrictions of its own genome and thus adapt to environmental pressures, physiologic changes, and experiences. Dynamic shifts in the strength of preexisting connections across distributed neural networks, changes in task-related cortico-cortical and cortico-subcortical coherence and modifications of the mapping between behavior and neural activity take place in response to changes in afferent input or efferent demand. Such rapid, ongoing changes may be followed by the establishment of new connections through dendritic growth and arborization. However, they harbor the danger that the evolving pattern of neural activation may in itself lead to abnormal behavior. Plasticity is the mechanism for development and learning, as much as a cause of pathology. The challenge we face is to learn enough about the mechanisms of plasticity to modulate them to achieve the best behavioral outcome for a given subject.

Functional imaging of human crossmodal identification and object recognition

The perception of objects is a cognitive function of prime importance. In everyday life, object perception benefits from the coordinated interplay of vision, audition, and touch. The different sensory modalities provide both complementary and redundant information about objects, which may improve recognition speed and accuracy in many circumstances. We review crossmodal studies of object recognition in humans that mainly employed functional magnetic resonance imaging (fMRI). These studies show that visual, tactile, and auditory information about objects can activate cortical association areas that were once believed to be modality-specific. Processing converges either in multisensory zones or via direct crossmodal interaction of modality-specific cortices without relay through multisensory regions. We integrate these findings with existing theories about semantic processing and propose a general mechanism for crossmodal object recognition: The recruitment and location of multisensory convergence zones varies depending on the information content and the dominant modality.

What blindness can tell us about seeing again: merging neuroplasticity and neuroprostheses

Significant progress has been made in the development of visual neuroprostheses to restore vision in blind individuals. Appropriate delivery of electrical stimulation to intact visual structures can evoke patterned sensations of light in those who have been blind for many years. However, success in developing functional visual prostheses requires an understanding of how to communicate effectively with the visually deprived brain in order to merge what is perceived visually with what is generated electrically.

V1 activation in congenitally blind humans is associated with episodic retrieval

Recently we showed that the occipital cortex of congenitally blind humans is activated during verbal-memory tasks. Activation was found in regions corresponding to the retinotopic visual areas of sighted humans, including the calcarine sulcus (V1). No such occipital activation was found in sighted humans. One year later, the same blind subjects participated in a second fMRI scan, to study the contribution of semantic elements and episodic memory to the occipital activation. The subjects performed an episodic-memory task, requiring recognition of words that were originally presented in the first scan. We demonstrate here that the magnitude of V1 activation during the recognition task is correlated with memory performance, assessed during the scan. Across the blind, the better-remembered set of words elicited greater V1 activation than words from the poorly-remembered set, although the semantic components and the behavioral task were similar in the two sets. This indicates that on top of semantic processing (suggested previously), V1 activation in the blind is probably associated with long-term episodic memory. Indeed, within the blind, those who showed better recognition-memory performance had greater V1 activation compared with the poorer performers. We conclude that the posterior occipital cortex (including V1) of the congenitally blind is likely to be involved in episodic retrieval.

Transcranial magnetic stimulation of the occipital pole interferes with verbal processing in blind subjects

Recent neuroimaging studies in blind persons show that the occipital cortex, including the primary visual cortex (V1), is active during language-related and verbal-memory tasks. No studies, however, have identified a causal link between early visual cortex activity and successful performance on such tasks. We show here that repetitive transcranial magnetic stimulation (rTMS) of the occipital pole reduces accuracy on a verb-generation task in blind subjects, but not in sighted controls. An analysis of error types revealed that the most common error produced by rTMS was semantic; phonological errors and interference with motor execution or articulation were rare. Thus, in blind persons, a transient 'virtual lesion' of the left occipital cortex interferes with high-level verbal processing

Early 'visual' cortex activation correlates with superior verbal memory performance in the blind

The visual cortex may be more modifiable than previously considered. Using functional magnetic resonance imaging (fMRI) in ten congenitally blind human participants, we found robust occipital activation during a verbal-memory task (in the absence of any sensory input), as well as during verb generation and Braille reading. We also found evidence for reorganization and specialization of the occipital cortex, along the anterior-posterior axis. Whereas anterior regions showed preference for Braille, posterior regions (including V1) showed preference for verbal-memory and verb generation (which both require memory of verbal material). No such occipital activation was found in sighted subjects. This difference between the groups was mirrored by superior performance of the blind in various verbal-memory tasks. Moreover, the magnitude of V1 activation during the verbal-memory condition was highly correlated with the blind individual's abilities in a variety of verbal-memory tests, suggesting that the additional occipital activation may have a functional role.

Convergence of visual and tactile shape processing in the human lateral occipital complex

We have recently demonstrated using fMRI that a region within the human lateral occipital complex (LOC) is activated by objects when either seen or touched. We term this cortical region LOtv for the lateral occipital tactile-visual region. We report here that LOtv voxels tend to be located in sub-regions of LOC that show preference for graspable visual objects over faces or houses. We further examine the nature of object representation in LOtv by studying its response to stimuli in three modalities: auditory, somatosensory and visual. If objects activate LOtv, irrespective of the modality used, the activation is likely to reflect a highly abstract representation. In contrast, activation specific to vision and touch may reflect common and exclusive attributes shared by these senses. We show here that while object activation is robust in both the visual and the somatosensory modalities, auditory signals do not evoke substantial responses in this region. The lack of auditory activation in LOtv cannot be explained by differences in task performance or by an ineffective auditory stimulation. Unlike vision and touch, auditory information contributes little to the recovery of the precise shape of objects. We therefore suggest that LOtv is involved in recovering the geometrical shape of objects.

Visuo-haptic object-related activation in the ventral visual pathway

The ventral pathway is involved in primate visual object recognition. In humans, a central stage in this pathway is an occipito-temporal region termed the lateral occipital complex (LOC), which is preferentially activated by visual objects compared to scrambled images or textures. However, objects have characteristic attributes (such as three-dimensional shape) that can be perceived both visually and haptically. Therefore, object-related brain areas may hold a representation of objects in both modalities. Using fMRI to map object-related brain regions, we found robust and consistent somatosensory activation in the occipito-temporal cortex. This region showed clear preference for objects compared to textures in both modalities. Most somatosensory object-selective voxels overlapped a part of the visual object-related region LOC. Thus, we suggest that neuronal populations in the occipito-temporal cortex may constitute a multimodal object-related network.